My university uses AI detectors when marking papers, and almost any result higher than 0% can cause lost grades. As anyone whose used one of these tools will know, they have an extremely high false positive rate, meaning that while I don’t use AI (and can’t, given the technical nature of the papers) I still lose marks. Is there any way to decrease the rate of false positives without completely destroying the structure and flow of a paper?

  • mlg
    link
    fedilink
    English
    arrow-up
    4
    ·
    3 hours ago

    higher than 0% can cause lost grades.

    I still lose marks.

    Is this actual policy or are you just trying to make sure your AI usage can’t be detected?

    Because there is no reliable way to detect AI usage except for obvious styling, prompt traps, and just matching your comprehension with the material (grades on assignments vs closed notes test).

    I’ve never heard of any university using such a system and would bring this up to administration with other students if they ever used such an unreliable grading method.

  • Meatwagon@lemmy.dbzer0.com
    link
    fedilink
    arrow-up
    6
    ·
    4 hours ago

    Pay for the highest tier plan of Claude and ask it to make it look more like a human wrote it.

    Real answer: Your university is being ridiculous. They should be using a digital proctoring service while you guys do your papers if they’re that serious about it. Detectors will never work. We aren’t allowed to make flat accusations of AI use, but a high result would make us look at the paper and see if there were any other irregularities or talk to the student to make sure that can explain the paper. It’s not a free chance to just toss the paper into a cheating bin.

  • Dearth
    link
    fedilink
    arrow-up
    15
    ·
    8 hours ago

    Video record you writing your papers by hand with a clear view of your computer screen. Turn in hand written paper.

  • alakey@piefed.social
    link
    fedilink
    English
    arrow-up
    19
    ·
    10 hours ago

    Slop detectors might as well be random number generators, it will never give you 0% because it’s slop in and of itself.

  • teyrnon@sh.itjust.works
    link
    fedilink
    arrow-up
    14
    ·
    9 hours ago

    Sounds like someone should file a lawsuit against the school for using ai ai detectors that don’t work right.

    At the least there should be an appeals process, anonymously, where the decision goes to a jury of faculty or something if not settled otherwise.

  • chuckleslord
    link
    fedilink
    arrow-up
    19
    ·
    11 hours ago

    No. There is nothing you can do. AI detection is bullshit and there is no way to game it to reduce how much it determines you to be an AI. Your school is punishing you for it existing, and thus they remove the punishment for those using it.

  • Yerbouti@sh.itjust.works
    link
    fedilink
    arrow-up
    15
    ·
    edit-2
    7 hours ago

    My experience is the opposite. Even for a work that was 100% obviously written by AI, an AI detector only gave me 30% probability. That specific work was written with a PhD level and had bogus sources, one litteraly having copilot in the name. Coming from a 19 years old first year college student… On every work I presented, the AI evaluated the probability very low while for me (I know the students, I’ve spent 15 weeks with them) l, they obviously didn’t wrote the papers. When I pointed to the model that I was sceptical, it suddenly change the cheating probability to a much higher score. So basically it just tolds you what you want to hear. The college told us they wont consider cheating unless we have proof so we just end up giving good grades to everyone. Two scenarios here, either about 70% of students use some sort of AI to write these days, or we have the smartest generation ever in front of us, with 19 years old kids having the level of their teacher aftef a year in College. University and College administration want to give diplomas, the students want to graduate, and the teachers are now just a tool to get this done.

    Universities like Hardvard recently adopted internal rules preventing a class from having more than 40% A or A+ in the final result, because it was getting absurd, with some classes having 80% of the students with an A. I’m seriously considering leaving college teaching because I can’t understand what’s happening.

    • Hoimo@ani.social
      link
      fedilink
      arrow-up
      2
      ·
      5 hours ago

      When I pointed to the model that I was sceptical, it suddenly change the cheating probability to a much higher score.

      What did you use? Don’t ask the LLM if something is written by an LLM, it has no way of telling.

  • reksas@sopuli.xyz
    link
    fedilink
    arrow-up
    22
    ·
    12 hours ago

    i wonder if it eventually comes to having to resort to legal action because of this. Those detectors are unreliable yet you can be penalized based on what they spew. And lost grades is lasting consequence.

    • bigmclargehuge
      link
      fedilink
      arrow-up
      17
      ·
      12 hours ago

      Meanwhile the same schools that punish students for using AI (even when they didn’t) are then hiring speakers to cram the “AI is the future, get used to it or get left behind” message down students throats.

      • reksas@sopuli.xyz
        link
        fedilink
        arrow-up
        4
        ·
        9 hours ago

        its actually quite weird how strongly the ai is being pushed everywhere. Though I guess it just proves how deep the rot goes.

  • AllHailTheSheep@sh.itjust.works
    link
    fedilink
    arrow-up
    6
    ·
    11 hours ago

    years ago, we were told to make our writing formal and dry. now, the best way to make sure it doesn’t get flagged as false positives is tone and personality of your writing. an AI will never have a joke in a footnote, actual papers sometimes do.

      • AllHailTheSheep@sh.itjust.works
        link
        fedilink
        arrow-up
        1
        ·
        3 hours ago

        yes, 100%. and if your writing has the tone where a quick aside lands naturally, it doesn’t matter where it is anyways, you’ve already passed the detection.

  • GreenShimada
    link
    fedilink
    arrow-up
    73
    ·
    18 hours ago

    You do really need to make a stand on this. Personally, I would offer a test or demonstration where you write a paper in a place where you can be assured to not be using an AI, and have them check the false-positive rate to show the vendor they have is bad. Short of that, you’re really talking about pushback on an administration that clearly has only mediocre ideas, and is willing to let students suffer the brunt of their bad decision-making until you make it a problem for them.

    Students should have a way to seek real recourse about this. Use it now and use it early to establish a record of receiving false positives, Otherwise you risk some professor later really causing problems with a 5% false positive rate and it being a huge problem for you.

  • degenerate_neutron_matter@fedia.io
    link
    fedilink
    arrow-up
    38
    ·
    17 hours ago

    You could use something that automatically keeps track of revision history, like Google Docs. If your writing is flagged by their AI detectors, show them the full history.

  • pcouy@lemmy.pierre-couy.fr
    link
    fedilink
    arrow-up
    19
    arrow-down
    2
    ·
    15 hours ago

    As someone who teaches CS and grades assignments, the last few years have been really rough. Academic dishonesty has skyrocketed with models becoming smarter and students becoming more dependent on them. Any assignment that’s above average will make me suspicious, and when it appears to be 100% AI generated, the feeling that I spend more time grading than the student spent working on it is awful. Even when I’m almost sure a work is AI generated, unless there are some dumb leftovers such as “As an AI assistant […]”, I can never be 100% sure. This causes me a lot of headaches because the only thing worse than rewarding dishonesty would be not appropriately rewarding an outstanding assignment.

    As much as I’d love to have à software tell me with 100% certainty which parts (if any) of an assignment are AI written, AI detectors are all snake oil, no exception. They exploit teachers’ helplessness to make false promises that we really want to believe in.

    Moreover, I don’t think fully banning AI use is a sensible thing to do. LLMs are a thing, whether we like them or not, and using them in a sensible way is a useful skill to learn. There’s one big issue though : on one hand, assignments are made so that the problems students have to solve all have well-known solutions. This is required to make sure the assignment is doable in the first place, and that teachers will be able to help. On the other hand, LLMs are disproportionately good at classic assignment problems since there are so many published solutions online (which then end up in training datasets). Moreover, assignements are usually made to guide students through a larger problem by breaking it down into smaller problems, which is basically the perfect prompt for a LLM. This means students can get away with the laziest uses of LLMs (which usually won’t work with real world problems). In the worst cases, the only “skill” some students learn is to throw a PDF at whatever AI they paid for, ask it to solve the assignment, and copy paste the output without thoroughly reading it first.

    Teachers clearly need to adapt. There will always be a few students who fail to learn in every class, but when so many students don’t learn, it’s the teacher who is failing to teach.

    • agamemnonymous@sh.itjust.works
      cake
      link
      fedilink
      arrow-up
      1
      arrow-down
      1
      ·
      4 hours ago

      Maybe it would be worthwhile to explicitly build the assignment around using AI, and grading on editing the result? Kinda like how research papers are graded on properly citing and presenting information that’s not supposed to be original.

      If LLMs are going to be a lasting tool, maybe using them effectively is an important skill to teach. Encourage AI use in generating components, but force those components into a structure that AI struggles with, and grade based on how well the AI-generated components fit together in a coherent end product.

      I remember when I was studying math in college, the upper level courses regularly gave take-home exams because all the tools and resources in the world weren’t going to help if you didn’t understand the material.

      It’s not a great solution, if students are using AI to skirt learning the basics then they aren’t going to develop the skills to understand the work they’re editing. Kinda like calculators; they’re great when you’re being evaluated for more complex tasks where the arithmetic isn’t the important part, but kids still need to learn how to do the arithmetic in the first place before they automate it.

      But the genie’s out of the bottle. Fair or not, teachers are going to have to adapt to test the skills that can’t be automated yet. I was around for the tail end of teaching kids how to use the card catalog in the library to do research, but everyone just uses search engines now.

      I do not envy teachers right now. They have a Hurculean task before them, and I only see it getting worse as AI gets better.

    • abbadon420@sh.itjust.works
      link
      fedilink
      arrow-up
      17
      ·
      13 hours ago

      I’m also a teacher. I teach programming. I teach adults, some even 50+. I love teaching the older students, because they really care, they really want to learn, they put in effort and attention, all of them. Younger students can be just as awesome, but there are more bad apples in that category. It’s not really agism though, because it’s really just the people that want to get rich quick that are the bad apples. Those are inherently on the younger side.

      My university primarily teaches online, through Teams. Lessons are not mandatory. Lately it happens a lot that simeone hands in their finals assignment and it looks shiny. I’ve never heard of this student or talked to them. They never handed in any homework or asked any questions. They didn’t even watch the recorded lessons. So I look at the code and it’s all smells of AI.

      In this case our process is to fike it with the exam commission and they will have a meeting with the student to prove theyy actually did it themselves. I’ve sit in with a few of these meetings, but I’ve only seen one student be able to answer any of the questions we ask them about their own work.

      This works pretty well, because I don’t have to prove AI, I just have to point to suspicion. Than the students have to prove that they know what they have done. Which is a “guilty until proven innocent” kind of sitution, but it’s very easy to pass if you’re actually innocent and did your own work.

      I think this is the tried and tested way to test for plagiarism, but now applied to AI usage. Because in fact, AI usage in a graduation project is plagiarism. Time consuming, but effective.

  • FriendOfDeSoto@startrek.website
    link
    fedilink
    English
    arrow-up
    13
    arrow-down
    1
    ·
    17 hours ago

    Get a video camera to film you while you write and screen record what you do on your laptop. That way you have proof that you didn’t use models. And I’m only half joking.

    As much as I loathe to suggest this: maybe you should try to prompt some models on topics you write on. Just to see what it spits out so you can avoid sounding alike. Same for visuals it would come up with.

    Or you could see if some industrious fellow student has found a copy of the software the faculty use to do a trial run. It may have fallen off a truck somewhere. Happens all the time! Just make sure you protect yourself.

  • viertesauge@feddit.org
    link
    fedilink
    Deutsch
    arrow-up
    15
    ·
    19 hours ago

    I guess you could make a document history. Make a copy for each day you work on that document. Idk if they are going to accept it as proof though…

    • trxxruraxvr
      link
      fedilink
      arrow-up
      15
      ·
      17 hours ago

      This only proves you can write. You can still generate the text using an LLM and write it down with a pen.

      • schnurrito@discuss.tchncs.de
        link
        fedilink
        arrow-up
        10
        ·
        16 hours ago

        I remember not long ago seeing an image of handwritten homework where the student answered “as a large language model …”.

        • idiomaddict
          link
          fedilink
          arrow-up
          6
          ·
          15 hours ago

          I had a student give me handwritten homework for German class. He’d written directions, and there was no instance of the word “left,” but the word ”remaining” was in every other sentence. It was very obvious

            • idiomaddict
              link
              fedilink
              arrow-up
              6
              ·
              14 hours ago

              I did ask him about it and he admitted it, but if he had done it with his own brain, it would have been worse, because we went over it in class a bunch first and he was capable of it then.

              It was totally pointless though, because the homework was for extra credit. Luckily, I didn’t have to pursue it further, but there are some programs that would have expelled him for it. I can understand that, but I don’t think things would be much improved if he got expelled for it.

              I just told him to stop being an idiot, because he’s not learning anything and the other teachers are probably better at spotting it. He was relatively dismissive at first, but he really didn’t learn very much German during a study abroad year, which is actively difficult to do. Now that they’re leaving, he’s realized that he didn’t get much out of the year, academically. Not a big deal, but he’s going to delay his graduation and see some minor consequences. Hopefully it’ll get his head on straight.