AI Detection and Grammarly
Hi everyone, We have faculty using the AI report and are appreciating that feature. An interesting question came up recently: the report showed 64% AI-generated text, but the student claimed he had only used Grammarly, which also helps to improve phrasing. Has anyone else experienced an AI report picking up on Grammarly, and is there a way of distinguishing between a "grammar help" tool and other generative AI?
Second, will the AI report be visible to students soon? Faculty assumed that students could see it, but the report is only visible to faculty in our LMS.
Thanks,
Jennifer Douglas, American Public University System
134 replies
-
So interesting to read this thread. I had a student yesterday state that he only used Grammarly on an assignment that came up with a 73% AI score. He also sent me a list of news articles about students who have been falsely accused of cheating based on Turnitin AI software detection. I would be interested in any progress you make Gill Rowell in working out if it's picking up Grammerly or Grammerly AI (I agree with Paul Rittman - I think it will be the latter.
-
ok and so what
-
Hi all, thank you for this forum. I have run into a similar predicament with a few of my students showing up with high AI scores. One of my students who I have personally seen writing during my office hours and asking questions about their essay turned in a paper with 90% AI detection score. After talking with them, she mentioned that she uses Grammarly for all of her writing and that this may be the cause. I would love to stay updated Gill Rowell about the prevalence of this situation. As it stands right now, I will not be giving any zeros based on AI scores until it is clear that Grammarly does not affect the report.
-
Amy Henderson and Fiona Heagerty I will update you as soon as I have further news on this. In the meantime, if you have specific papers which are demonstrating this behaviour, please message me directly with details and I will pass on to the AI development team to take a look.
-
We are finding the same thing - a student swears he only used Grammarly, but that comes up as 55% AI use. It would be great if the TurnItIn could give us clarity on whether this is even likely that their detector will detect Grammarly.
-
I also hear from students that they used Grammarly. But I am wondering if they are using GrammarlyGo. I did hear that GrammarlyGo uses GPT3 language so that is what ChatGPT is based off of, so would Turnitin AI then detect GrammarlyGo?
-
Our institution uses Grammarly premium and the GrammarlyGo is disabled, so we can’t use it. I conducted three different tests to find out if Turnitin’s AI detector is in fact flagging Grammarly. Based on my findings, Grammarly isn’t the issue. I typed an essay from scratch in Grammarly, accepted the changes/edits, and downloaded it and turned it in to canvas. 0% AI detection. I then took a different essay I typed in Word, copied it into Grammarly, accepted the changes and edits and downloaded and turned it in. 0% AI detection. THEN I copied an essay from chatgpt, pasted it into Grammarly, accepted any changes, downloaded it and then turned it into canvas. 100% AI detection. My findings show that it’s not Grammarly. I’ve only gotten this far, but I mean it’s something to note.
-
Our detector is not tuned to target Grammarly generated content but rather, other AI content written by LLMs such as GPT-3.5. Based on initial tests we conducted on human-written documents with no AI-generated content in them, in most cases, changes made by Grammarly and/or other grammar-checking tools were not flagged as AI-written by our detector.
-
If an essay is flagged with 50% or over for AI, wouldn't that pretty much toss out the "false positive" theory? I get 20% or less can be less reliable, but 50%?