My university uses AI detectors when marking papers, and almost any result higher than 0% can cause lost grades. As anyone whose used one of these tools will know, they have an extremely high false positive rate, meaning that while I don’t use AI (and can’t, given the technical nature of the papers) I still lose marks. Is there any way to decrease the rate of false positives without completely destroying the structure and flow of a paper?
As someone who teaches CS and grades assignments, the last few years have been really rough. Academic dishonesty has skyrocketed with models becoming smarter and students becoming more dependent on them. Any assignment that’s above average will make me suspicious, and when it appears to be 100% AI generated, the feeling that I spend more time grading than the student spent working on it is awful. Even when I’m almost sure a work is AI generated, unless there are some dumb leftovers such as “As an AI assistant […]”, I can never be 100% sure. This causes me a lot of headaches because the only thing worse than rewarding dishonesty would be not appropriately rewarding an outstanding assignment.
As much as I’d love to have à software tell me with 100% certainty which parts (if any) of an assignment are AI written, AI detectors are all snake oil, no exception. They exploit teachers’ helplessness to make false promises that we really want to believe in.
Moreover, I don’t think fully banning AI use is a sensible thing to do. LLMs are a thing, whether we like them or not, and using them in a sensible way is a useful skill to learn. There’s one big issue though : on one hand, assignments are made so that the problems students have to solve all have well-known solutions. This is required to make sure the assignment is doable in the first place, and that teachers will be able to help. On the other hand, LLMs are disproportionately good at classic assignment problems since there are so many published solutions online (which then end up in training datasets). Moreover, assignements are usually made to guide students through a larger problem by breaking it down into smaller problems, which is basically the perfect prompt for a LLM. This means students can get away with the laziest uses of LLMs (which usually won’t work with real world problems). In the worst cases, the only “skill” some students learn is to throw a PDF at whatever AI they paid for, ask it to solve the assignment, and copy paste the output without thoroughly reading it first.
Teachers clearly need to adapt. There will always be a few students who fail to learn in every class, but when so many students don’t learn, it’s the teacher who is failing to teach.
I’m also a teacher. I teach programming. I teach adults, some even 50+. I love teaching the older students, because they really care, they really want to learn, they put in effort and attention, all of them. Younger students can be just as awesome, but there are more bad apples in that category. It’s not really agism though, because it’s really just the people that want to get rich quick that are the bad apples. Those are inherently on the younger side.
My university primarily teaches online, through Teams. Lessons are not mandatory. Lately it happens a lot that simeone hands in their finals assignment and it looks shiny. I’ve never heard of this student or talked to them. They never handed in any homework or asked any questions. They didn’t even watch the recorded lessons. So I look at the code and it’s all smells of AI.
In this case our process is to fike it with the exam commission and they will have a meeting with the student to prove theyy actually did it themselves. I’ve sit in with a few of these meetings, but I’ve only seen one student be able to answer any of the questions we ask them about their own work.
This works pretty well, because I don’t have to prove AI, I just have to point to suspicion. Than the students have to prove that they know what they have done. Which is a “guilty until proven innocent” kind of sitution, but it’s very easy to pass if you’re actually innocent and did your own work.
I think this is the tried and tested way to test for plagiarism, but now applied to AI usage. Because in fact, AI usage in a graduation project is plagiarism. Time consuming, but effective.
Seems like you nailed it. Basically the solution to AI detection is “defend your thesis” meeting at the end of the year
Maybe it would be worthwhile to explicitly build the assignment around using AI, and grading on editing the result? Kinda like how research papers are graded on properly citing and presenting information that’s not supposed to be original.
If LLMs are going to be a lasting tool, maybe using them effectively is an important skill to teach. Encourage AI use in generating components, but force those components into a structure that AI struggles with, and grade based on how well the AI-generated components fit together in a coherent end product.
I remember when I was studying math in college, the upper level courses regularly gave take-home exams because all the tools and resources in the world weren’t going to help if you didn’t understand the material.
It’s not a great solution, if students are using AI to skirt learning the basics then they aren’t going to develop the skills to understand the work they’re editing. Kinda like calculators; they’re great when you’re being evaluated for more complex tasks where the arithmetic isn’t the important part, but kids still need to learn how to do the arithmetic in the first place before they automate it.
But the genie’s out of the bottle. Fair or not, teachers are going to have to adapt to test the skills that can’t be automated yet. I was around for the tail end of teaching kids how to use the card catalog in the library to do research, but everyone just uses search engines now.
I do not envy teachers right now. They have a Hurculean task before them, and I only see it getting worse as AI gets better.