Identifying AI-Generated Plagiarism

If you're truly concerned that a student didn't write an assignment, you can quiz your student on the contents of their own writing.  If you do this, here are a few recommendations:

• Conversations might work better.  If you're already talking with every student, then a quiz might not be necesssary.

• Quiz every student.  You must have uniform standards for all your students — it's not fair or ethical to single out only one or two students for a plagiarism check.  If you do suspect a couple students of using AI to commit plagiarism, I recommend doing an in-class quiz with all your students.  Just have them write about the work they've already done.  Hence...

• This can be time consuming.  After all, how do you quiz every single student on the individual assignments submitted for your course?  You can't draw up a multiple-choice test for every single paper you've received.  Instead, you'll have to ask broad questions with short answers, which means...

• It might not be accurate.  A student who read an AI submission could easily give correct answers about it.  You would need a deeper conversation with that student to see just how deep their knowledge goes.

• Keep it light.  You don't want to cast suspicions on a student without cause.  Assume your student is being honest until you have good evidence to the contrary.

• Remember that people can forget their own writing.  During my dissertation, I was under so much stress that I drafted 150 pages without any recollection of the contents.  Now, had you quizzed me on the specific facts I'd written, I'm sure I could have answered those questions, but that might not be the case for every student.

Yes, You Should Use AI Dectors...but with care.

Despite the bad rap, AI detectors do offer a valuable data point when investigating cases of suspected AI.  Most AI detectors will give a percentage of "human" versus "AI-generated" content.  The percentage works about as well as a weather forecast for rain — a 15% chance of rain means that rain is possible, but unlikely, whereas an 80% chance of rain means that rain is likely, but not guaranteed.  But if the AI detector is giving you 0% or 100%, then that serves as very strong evidence of either calm skies or an e-mail to the Dean of Students office.

Here's the main concern with AI detectors: since they can't possibly give black-and-white certainty of AI involvment, we're left wondering what percent of "possible AI content" constitutes reasonable cause for investigation.  As a teacher, you have to decide the degree to which they can help.  Personally, I've found AI detectors useful, but they are not a tool of certainty.

Following gut instinct and then using AI detectors, I caught three high school students using AI between January-July, 2023.  I identified the AI concern because I had papers that didn't at all match the writing styles or speech patterns of two of my students — AI detectors found a high liklihood of AI, and then I verified the situation by talking with the individual students directly.  The third student, however, provided a more interesting case.  This individual naturally spoke in a rather elevated style — the AI-generated text read in a way that sounded very much like the student.  But the content seemed off, and I never the student working in class, so I ran the pages through an AI detector — it indicated a high liklihood of AI use, and it wasn't wrong.

When confronted, two of the students confessed to using AI — though the third one denied using AI, he had no personal knowledge of the information in the work.

In all three of these case, the AI detector was simply one tool — I would never recommend using an AI detector alone.  I had another case where the AI detector registered "98% AI content" when the words were pretty clearly produced by the student.  Unless of course I'm wrong — and there's likely no way I will ever truly know.

Because of the three students mentioned above, I began a deeper investigation into whether I had other students using AI to commit plagiarism.  Another instructor had also heard rumors that certain students weren't doing their own work, and this led to tension with those students who were genuinely work hard to graduate early.

This created a serious problem — if students believe that my system allowed some students to succeed by cheating without effort, then cheating would become not only common, but encouraged.  Rumors alone are enough to undermine the integrity of a system, even when no actual cheating has previously taken place.  But I had found AI misuse — it was clumsy and easily spotted, but what if I had cases that were less clumsy?

So I ran AI detector spot checks on submits that had been submitted earlier.  Fortunately, I didn't find any more cases of AI — and the AI detectors indicating that the writing was by human hands.  Instead, I found the more typical forms of plagiarism — three cases of students  copying material other websites without attribution.

Honestly, I was pretty surprised — I had assumed that my high school students would have less incentive to cheat.  But the fact that six students had been cheating on their papers was not a secret to the student body.  Basically, one student had gotten away with it for a couple months, and then she gave "advice" to help others do the same.  Most didn't follow that advice, but one did.

In another case, the plagiarism came from a misunderstanding of the nature of writing — one of my students had essentially been taught that research writing meant looking up sources and "patching together" the information from those sources into a paper — her patch writing involved copying over paragraphs from a variety of sources and arranging them into a coherent whole.  No, I'm not saying that any teacher would specifically teach a student to compose an essay this way, but failing to catch and correct this phenomenon is essentially the same as teaching it.  This student learned from prior teachers that a paper with good paragraphs from good sources will get an A.

Clearly, I had to do something.  Given the student population I was working with, it didn't make sense to "drop the hammer" or anything like that — most of my students had never been exposed to a rigorous writing classroom, and most didn't have prior positive experiences with school.  The cheating made sense, but it had to be stopped — and further plagiarism had to be prevented.  So I had the students redo their papers, I ran the revisions through the same plagiarism checks, and everyone continued on with their lives.

But here's the moral: you can't stop academic dishonesty unless you can accurately and consistently determine when it has happened.  Students must know that there is a credible system in place to weed out cheating.  And this isn't simply to ensure that students learn — I have many students who would never cheat under any circumstances, even if they knew they could get away with it.  But if they saw others getting away with it, they would rightfully feel incensed.  Cheated, you might say.  And that kind of justifiable anger creates a very unhealthy atmosphere.