1

Grammarly & Turnitin.com's AI score

I have been using turnitin.com's new AI detection score since it came out. I have seen several assignments get 100% on the AI score. 

One student emailed me saying that they used Grammarly to help edit their writing. I see (from this thread: https://turnitin.forumbee.com/t/g9hcsxv/ai-detection-and-grammarly) that I am not the only teacher dealing with this issue.

I did ask the student if they could email me their composition in its pre-grammarly stage, and they said that they don't have a copy, that they simply composed it while on the grammarly site (at least that is my understanding of their response). 

So I am wondering: how much work does grammarly have to do, to get 100% score? I think I might just upload some samples to see, before and then after letting grammarly make suggestions. But I'm wondering if anyone here has had experience seeing how much work grammarly did on a composition, to get that 100% score.

13 replies

null
    • Paul_Rittman
    • 10 mths ago
    • Reported - view

    Hmmm well I just uploaded 2 short documents of notes that I had composed. One got 0% on both Similarity and AI; the other got 13% similarity and 0% AI. I then pasted both files into Grammarly, and automatically accepted (well, by clicking on each one) all their grammar suggestions. I then uploaded the edited files onto turnitin.com. As I thought, both of the edited files got 99% on the similarity score (they were essentially copies of the files that I had uploaded previously), but they both got 0% AI score on turnitin.com. The shorter file, of 625 words, had something around 35 suggestions, all of which I incorporated (OK, a few foreign terms that grammarly wanted to turn into English words were not accepted, but that was it). The longer file was of 4509 words, and I accepted all 93 suggestions.

     

    So my conclusion based on this very short experiment, with a sample size of two short documents, is that you have to work very hard to get Grammarly to make so many corrections to your document, that turnitin.com says your AI score is 100%.

    One other thing--I see that Grammarly is offering its own AI generating tool. I am not sure how to get it to turn your own text into edited text. But the fact that someone says they just went to grammarly doesn't mean (to me) that AI wasn't used at all.

    If anyone else has anything to comment, feel free. My own experience here doesn't necessarily mean that the student I referenced in the OP was lying to me, or using AI to compose their assignment. 

    • Customer Engagement Specialist, Turnitin
    • Gill_Rowell
    • 10 mths ago
    • Reported - view

    Paul Rittman We are conducting extensive testing on our tool and Grammarly and will be able to update you once this is complete. In the meantime if you have specific papers you would like us to investigate in more detail (such as the ones you reference above) please message me directly with the paper IDs.

    • lucy_hunt
    • 9 mths ago
    • Reported - view

    I've tested three different theories with this - and the only one that came back 100% AI generated was when I copied from an AI generator and pasted into Grammarly. Made the suggested edits or whatever, downloaded it, and then uploaded it to canvas. Came back 100%. I started from scratch in Grammarly, accepted suggestions/changes, then turned it in - no AI detection. Same for when I copied a document from word and pasted it in Grammarly, made changes, etc., and turned it in--no AI detection. This makes me believe any student who says they just used Grammarly to write their essay (and it came back as 100% AI generated) is lying. 

      • simpsych
      • 2 mths ago
      • Reported - view

      lucy,  I am a student doing a Master's, and English is my 3rd language, so yes, I use the Grammarly app to correct my grammar, but I have been flagged by AI even though I wrote my ideas. It hurts me after putting so much effort into my papers, and I don't feel like writing anymore. It is discouraging me to write my research paper. No matter how hard I try, it is flagged by AI. If I used AI and got flagged, I wouldn't be doing this research, reading all the reviews, and replying to you. My student life was beautiful when I was doing my bachelor's in 2017 before turnitin and before ChatGPT. Even if I wrote my own words, AI flagged everything now, if I use Grammarly. One of my friends had a PhD; he uploaded his old research paper, which was AI-detected. His research paper was done way before ChatGPT.\ I even uploaded my research paper on chatgpt and asked whether it was written by humans or AI-generated, and it said written by a human. This Turnitin is ruining my life. 

      • Turnitin Admin, University of Huddersfield, UK
      • s_d_bentley
      • 2 mths ago
      • Reported - view

      simchi marak I think it's been pretty well proven that ChatGPT can't accurately detect whether text was AI produced or not.

      • simpsych
      • 2 mths ago
      • Reported - view

      Steve Bentley, Thank you for the response. It might be true, but it was written by me, and copied-paste my paper to the chatgpt and wrote whether it was written by humans or AI-generated, this was the result chatgpt provided “it appears to be human written” and I believe it because it was actually written by human. 

      • Turnitin Admin, University of Huddersfield, UK
      • s_d_bentley
      • 2 mths ago
      • Reported - view

      simchi marak But the fact is that you are asking a system which does not know how to distinguish between human written and AI written material, you may as well toss a coin.

      • simpsych
      • 2 mths ago
      • Reported - view

      Steve Bentley, I believe, even turnitin as well, because my paper was detected AI even though I wrote it myself. However, I used the Grammarly app to correct spelling and grammar. On my first paper, I was detected as 9% AI. I took this cautiously, and yes, I used the Grammarly app, on the second paper I was detected 81% which made me upset and hurt after I put so much effort into it and on my final paper, it was 20% AI detected, I used the Grammarly app as well, but I used it where it was necessary I don't use it for all the papers, but I lost points on my grammar still detected 20% so I begin to research on Google, quora and Reddit that if we use Grammarly app can turnitin detect AI or not.

      • Turnitin Admin, University of Huddersfield, UK
      • s_d_bentley
      • 2 mths ago
      • Reported - view

      simchi marak Turnitin themselves say that an AI score of 20% or lower is highly likely to be a false positive, so I wouldn't be concerned.

      There are certainly points about Grammarly which need to be clarified, and the AI detection has opened up a huge discussion at my institution about whether allowing Grammarly to rewrite large portions of your work (compared with having it provide suggestions that you choose whether to act upon) is acceptable or not. 

      If you are losing points for grammar errors - ie the grammar is part of the assessment criteria - I would be surprised if using Grammarly to rewrite the work was permitted, but that's a matter for your institution.

      • Carol_Bailey.1
      • 2 wk ago
      • Reported - view

      lucy When you say 'no AI detection', do you mean no AI use % or 0%? I've had a situation where Turnitin was returning no result for a test document I'd uploaded using an AI-generated article summary. Another colleague uploaded the same document and received a 50% AI score. The issue is fixed now (took about four weeks) but it does raise questions about the reliability of the tool.

    • Marjorie_Pereira
    • 9 mths ago
    • Reported - view

    I have determined that if you type in to Chat GPT the following: Did you write this: and then paste the section that is flagged up as AI generated- It will own up to its writing. 

      • Turnitin Admin, University of Huddersfield, UK
      • s_d_bentley
      • 2 wk ago
      • Reported - view

      Marjorie Pereira I know this is an old post, but this is incorrect, it just guesses. And certainly in my institution pasting a student's work into GPT would breach the student's IP rights.

    • Shiferaw_Safoyi
    • 7 mths ago
    • Reported - view
    When we are given an assignment question, we search for it from google and after finding it, write us a website that can correct us so that the answer does not look like a copy paste

Content aside

  • 1 Likes
  • 2 wk agoLast active
  • 13Replies
  • 3109Views
  • 9 Following