7

AI Detection and Grammarly

Hi everyone, We have faculty using the AI report and are appreciating that feature. An interesting question came up recently: the report showed 64% AI-generated text, but the student claimed he had only used Grammarly, which also helps to improve phrasing. Has anyone else experienced an AI report picking up on Grammarly, and is there a way of distinguishing between a "grammar help" tool and other generative AI?

Second, will the AI report be visible to students soon? Faculty assumed that students could see it, but the report is only visible to faculty in our LMS.

Thanks,

Jennifer Douglas, American Public University System

134 replies

null
    • Customer Engagement Specialist, Turnitin
    • Gill_Rowell
    • 1 yr ago
    • Reported - view

    Hi Jennifer Douglas Many thanks for your feedback on our new AI Detection feature. We have received several similar reports of this behaviour and are running some tests. It might be worth us investigating this further for you. I will message you directly for more details.

    Regarding student access to the report, this is something we will review in future. 

      • Dean, Graduate Studies and Research
      • Jennifer_Douglas
      • 1 yr ago
      • Reported - view

      Gill Rowell Here is a redacted sample of the Originality report and the AI report for the student in question, who then claimed that he had only used Grammarly and not an AI chatbot tool.

      • reesejohnson1
      • 1 yr ago
      • Reported - view

      Jennifer Douglas Grammarly employs a combination of standard correction detection, which identifies frequently misused words based on patterns, and Chat GPT-4 to enhance its speed and accuracy. Similarly, the latest version of the Windows suite has integrated AI-powered correction tools, as has added the option Google Docs. It's worth noting that I've tested content I personally wrote years before AI tools were accessible to the public, and these detection tools still identified them as AI-generated. I've also noticed most bibliographies become flagged by the detection too aswell. The detection tools might not be advancing as rapidly as AI itself and are not very reliable.

    • Narelle_Hunter
    • 1 yr ago
    • Reported - view

    Our faculty have experienced a similar issue with high AI likelihood but students stating they have used Grammarly, as well as language translation tools, such as Google Translate and not tools such as Chat GPT. I would be interested to know if the Turnitin AI detection is known to identify text from these tools as AI generated also.

      • Customer Engagement Specialist, Turnitin
      • Gill_Rowell
      • 1 yr ago
      • Reported - view

      Narelle Hunter I believe Google Translate may also cause similar behaviour. Are you able to message me direct with paper IDs so we are able to investigate in more detail? Or alternatively you can use the in-product feedback tool.

    • Paul_Rittman
    • 1 yr ago
    • Reported - view

    Same thing here. I submitted student work and Turnitin gave it a fairly low similarity score, but 100% AI score. The student told me that he had Grammarly edit his work. 

    Unfortunately, he composed his work on Grammarly (at least, that is what he says), so there isn't any original document that was corrected by Grammarly. 

    I have assigned a few zeros to different students, and this is the only one who has come back and insisted it wasn't AI. But this is a problem. I am not going to fault a student for using Grammarly, but man there really is IMO a big difference between this and AI. Assuming that my student was telling the truth, it would be nice for turnitin to know the difference.

    I think I will put in my syllabus for future classes, something to the effect that if you do choose to use Grammarly, to write out your text ahead of time, and keep the pre-edited version. 

      • Dean, Graduate Studies and Research
      • Jennifer_Douglas
      • 1 yr ago
      • Reported - view

      Paul Rittman Good idea to advise students to keep their pre-edited draft and be aware of the amount of editing that grammarly is doing.

      • Professor of Economics
      • Mike_Sulhoff
      • 6 mths ago
      • Reported - view

       I could have written your exact post.  It perfectly echos what I am doing, have done, and am experiencing.  I am so frustrated that I am close to sitting in sackcloth and ashes.  I want my students to produce organic work resulting from having absorbed the material.  Otherwise, we are sitting here spinning our wheels and going backward.  When I have issued "zeros" I have rarely been challenged, adding to my suspicion that I was right in doing so.  Grammarly can be useful in detecting misspellings, typos, and punctuation issues (it's doing that for me right now...) but to use it to, for all practical purposes, "rewrite" your paper for you is to not do the work at all, in my opinion.  Sure, you did some research, but to be able to faithfully communicate that on your own is a large part of secondary education.  The number of A.I. "checks" that come back as "100% A.I." is alarming, to say the least, and I have a hard time believing that they are the result of Grammarly correcting the student's original work.  

      I also have used what I call "internal" tests by comparing a student's submission to something else the student has written, often it might be a response that HAS to be organic, or even an email.  

      Crazily, they are even using A.I. to write emails, though, and then we are back at ground zero.  

      I have had some students submit to me work that has been proctored so I could compare it to work that was performed unattended.  This, too has helped betray the notion that the submissions in question were organic.  

      I know instructors who have quit the profession over this, and no wonder... what is the point of our efforts if we are, at the end of the day, simply grading an algorithm's "work?"  

      Finally, the most effective method I have found is to use source material that is not digitized on the internet and therefore unknown to A.I. at all.  An excellent example of this is "I Want the Earth Plus Five Percent" by Larry Hannigan, written in 1971...  (I teach Economics).  The work is out there (I provide links) but primarily exists as images and not digital text.  When a student tries to have Chat GPT write a response to this material, the algorithm can't "find" it and then creates a work of nonsense out of whole cloth.  It's a dead giveaway to me that the work was never even looked at.

      Regardless, it has made the work of instruction much harder for those who care about the material and want the students to become wiser and better educated.  If I didn't care, I'd just slap a "100%" on everything and go get myself an espresso.  

      Unfortunately (or not), I'm just not that guy.

      • Manager
      • Shabdal
      • 3 mths ago
      • Reported - view

      Send me your any previous work or upload on AI detector same as u did with student let’s see did you wrote your own work or it was AI.. come one bro, write anything and see., surely your work will catch AI as well.. 

       

    • JoAnne_Rondell
    • 1 yr ago
    • Reported - view

    I just encountered the same problem. According to Turnitin, a student's essay was created by AI. The student, however, claims he used Grammarly only. This is a serious problem, needless to say. Can we know by when Turnitin will be able to distinguish Grammarly from AI? 

      • Paul_Rittman
      • 1 yr ago
      • Reported - view

      Jo-Anne Rondell I am suspecting what they are doing after logging in to Grammarly, is using Grammarly's AI function (called something like GrammarlyGo--you can see it on the right side of the window after you log in). The reason why I say that, is that I've taken texts that were NOT composed by AI, taken them to grammarly and ONLY made grammar suggestions by Grammarly. The edited document had a 0% or very low AI score. So for turnitin to give an AI score of 100% when the student claims they submitted it to Grammarly, tells me that they used Grammarly's AI function. See my post here: https://turnitin.forumbee.com/t/83h4qhz/grammarly-turnitin-coms-ai-score

      • JoAnne_Rondell
      • 1 yr ago
      • Reported - view

      Paul Rittman Thanks very much for the reply. Can you tell me if you tried using Grammarly's AI?

      • Paul_Rittman
      • 1 yr ago
      • Reported - view

      Jo-Anne Rondell At this point I don't remember. I've been pretty busy since that post. :(

      • JoAnne_Rondell
      • 1 yr ago
      • Reported - view

      Paul Rittman Thanks again. I'm pursuing this since I suspect the problems will multiply as we continue on. I'll post back here with whatever I discover, if anything.

      • Paul_Rittman
      • 1 yr ago
      • Reported - view

      Jo-Anne Rondell Sounds good. Look forward to your results.

      • Paul_Rittman
      • 1 yr ago
      • Reported - view

      Jo-Anne Rondell One more thing. If you get some concrete results about how turnitin detects Grammarly AI vs. Grammarly's normal grammar check, I would PM Gill Rowell, who has posted above here in this thread twice. She is a turnitin.com employee who has PMed me about issues I've discussed. I think she would be interested in looking at your files, if you can show how turnitin's AI generator is working.

      • Customer Engagement Specialist, Turnitin
      • Gill_Rowell
      • 1 yr ago
      • Reported - view

      Jo-Anne Rondell I am happy to pass on any feedback you have to our AI development team. Please DM me with any paper IDs that are exhibiting this behaviour. We are all learning about AI writing!

      • Turnitin Admin, University of Huddersfield, UK
      • s_d_bentley
      • 1 yr ago
      • Reported - view

      Paul Rittman I think this is part of the problem, that Grammarly has multiple services, some only available to paid accounts. Free accounts, as I understand it, offer suggestions, paid tiers will rewrite sentences to improve them. Much more of a grey area in terms of acceptability, I know our regulations wouldn't accept that from a human proofreader.

       

      GrammarlyGo as their AI offering is an extra layer on top of that.

      • lucy_hunt
      • 1 yr ago
      • Reported - view

      Jo-Anne Rondell I have just stumbled onto this thread because I've run into this same issue. I tried to test this theory (typing something from scratch in Grammarly, downloading it, and then submitting it in canvas), but I guess because I was in "student view" and not an actual student, the AI detection was unable to be processed. Literally, the box was gray and it never registered. I did this twice. I have since resorted to asking a student I know and trust (who is in my class) to test this as well so I can see if it is in fact Grammarly that's being detected. If so, the two essays written by another student that were detected to have been generated by AI (100%) will be graded as normal. I'm hoping Turnitin's AI detector will be able to discern Grammarly from the other AI generators, but who knows. Should we discourage our students from starting from scratch in Grammarly? 

      • lucy_hunt
      • 1 yr ago
      • Reported - view

      Paul Rittman So, I figured the same thing (with GrammarlyGO), and I went to my Grammarly account to check. Our institution subscribes to Grammarly premium, so when I clicked on "features," the GrammarlyGO was disabled by admin. So if it's disabled on my end, it would have to be disabled on my students' Grammarlys right? That would mean that the AI detector is literally just detecting essays started and completed in Grammarly (but not using GrammarlyGO) as AI-generated. 

      • Emma_DukeWilliams
      • 1 yr ago
      • Reported - view

      lucy hunt COuld the student also have a free a/c with GO enabled? You get so many prompts for free ... 

      • lucy_hunt
      • 1 yr ago
      • Reported - view

      Emma Duke-Williams Super probable! I tell them they are only allowed to use their Grammarly accounts through the school, but of course, I'm sure some will use their personal accounts to sign up with Grammarly. I also tell my students that Turnitin will flag any and all AI sites/features--even if they're only using them to revise or clean up their writing. I just say avoid all AI sites and features on your essays that you have to submit to canvas. 

      • Manager
      • Shabdal
      • 3 mths ago
      • Reported - view

       turn it in and Grammerly both useless AI detectors.. you can write your piece of paper simultaneously with other students .. after a couple of days upload that it will show high AI score.. I am dealing few times this issue. 

    • Fiona_Heagerty
    • 1 yr ago
    • Reported - view

    So interesting to read this thread.  I had a student yesterday state that he only used Grammarly on an assignment that came up with a 73% AI score.  He also sent me a list of news articles about students who have been falsely accused of cheating based on Turnitin AI software detection.  I would be interested in any progress you make Gill Rowell in working out if it's picking up Grammerly or Grammerly AI (I agree with Paul Rittman - I think it will be the latter.  

Content aside

  • 7 Likes
  • 2 mths agoLast active
  • 134Replies
  • 76992Views
  • 61 Following