7

AI Detection and Grammarly

Hi everyone, We have faculty using the AI report and are appreciating that feature. An interesting question came up recently: the report showed 64% AI-generated text, but the student claimed he had only used Grammarly, which also helps to improve phrasing. Has anyone else experienced an AI report picking up on Grammarly, and is there a way of distinguishing between a "grammar help" tool and other generative AI?

Second, will the AI report be visible to students soon? Faculty assumed that students could see it, but the report is only visible to faculty in our LMS.

Thanks,

Jennifer Douglas, American Public University System

136 replies

null
        • lucy_hunt
        • 2 yrs agoSun, July 2, 2023 at 6:57 PM UTC
        • Reported - view

        If an essay is flagged with 50% or over for AI, wouldn't that pretty much toss out the "false positive" theory? I get 20% or less can be less reliable, but 50%? 

          • lucy_hunt
          • 1 yr agoTue, July 11, 2023 at 1:18 AM UTC
          • Reported - view

          Chris Turner This is what Turnitin has on its page: "Using the average scores of all the segments within the document, the model then generates an overall prediction of how much text in the submission we believe has been generated by AI."

          • TWC
          • 1 yr agoWed, July 12, 2023 at 8:58 PM UTC
          • Reported - view

            lucy hunt 

          AI vs Human score says the probability that AI thinks the content was generated by AI or generated by a person. A score of 20% AI 80% Human Means – The AI gives the content 80% probability of being Human and  20% probability of being AI generated; it doesn't mean – 80% of the article is human, and 20% is AI. My neighbour is the CTO of a large AI company and says that AI detection is not at a point where it can provide enough certainty and gives a high rate of false positives. I have run through pages of engineering textbooks through various AI detection including turnitin and each time the score had between 10- 20% approx AI content. This has to be incorrect as all the text I ran through detection was years old, well before AI writing tools came on the market. I'm interested to see how the detection tools develop, but they appear too inaccurate. 

          • lucy_hunt
          • 1 yr agoWed, July 12, 2023 at 10:54 PM UTC
          • Reported - view

          TWC I understand, but I don’t count anything that’s been flagged under 30% from Turnitin. If a student turns in an essay and it comes back with a  70-100% AI detection, I ask them to redo it. If it came back with 30% or less, I ignore it.  

        • TWC
        • 1 yr agoTue, July 11, 2023 at 12:55 AM UTC
        • Reported - view

        Also don't forget there is Grammarly premium for desktop and browser use. Many folks use this application or similar writing tools if they have dyslexia or chronic fatigue that can affect concentration.  Grammarly premium significantly changes sentence structure or whole paragraphs. 

        • Senior Teaching & Learning Innovations Specialist
        • Karen_Turnitin
        • 1 yr agoThu, July 13, 2023 at 1:09 PM UTC
        • Reported - view

        It helps if you think of the percentage as only one data point to have a discussion with students; ideally, that would take place during drafting, rather than submission. Two resources I'd suggest for this are this one on approaching a student when you think AI misuse has occurred or discussion starters for that situation. 

          • Turnitin Admin, University of Huddersfield, UK
          • s_d_bentley
          • 1 yr agoFri, July 14, 2023 at 7:55 AM UTC
          • Reported - view

          Karen Smith We're taking a view that the AI report alone is not sufficient to have that conversation. A "magic number" and no way to explain how that number has been reached is not a good basis to be making accusations - however gently they are framed. 

          Our approach is to require some additional evidence, such as falsified references or differences in writing style or standard of English compared with previous work before we act on the report, mainly because we don't want to damage staff-student relationships with this kind of conversation based on something that we can't back up and where it's the student's word against the Turnitin algorithm.

          • Senior Teaching & Learning Innovations Specialist
          • Karen_Turnitin
          • 1 yr agoFri, July 14, 2023 at 1:00 PM UTC
          • Reported - view

          Steve Bentley Yes, exactly. Many of our resources speak to the need for other data points. If I were still in the classroom, that number would nudge me to look more closely and have the conversations if it seems warranted. 

        • Angelo_Marin
        • 1 yr agoFri, July 21, 2023 at 12:20 PM UTC
        • Reported - view

        a foremost authority on AI detection is Atheer Mahir. He has a copious thread on this subject on his Linkedin profile

          • Joseph_Jacob
          • 1 yr agoSun, October 1, 2023 at 9:29 PM UTC
          • Reported - view

          Angelo Marin I concur with you. His articles suggest that these apps often misjudge content. His tests show AI texts seem entirely human, while older texts appear 80% AI-generated. This highlights detection tools' inadequacy and potential for false accusations.

        • David_Silver
        • 1 yr agoTue, July 25, 2023 at 6:54 PM UTC
        • Reported - view

        I have just joined this network and have reviewed the posts below. I have had some instance where TII has flagged students as having 50% AI generated content. However, some students are adamant they did not use AI or Grammarly/Quillbot. Are they just lying to me or are there any other possibilities as to why their work might be flagged? I had a student initially get flagged at 58%. They made numerous edits to the essay (as seen via google docs) and the resubmission still got flagged at 60%. I am having trouble understanding what might be the issue and would love some guidance. 

          • Customer Engagement Specialist, Turnitin
          • Gill_Rowell
          • 1 yr agoWed, July 26, 2023 at 4:42 PM UTC
          • Reported - view

          David Silver Thank you for joining us! I think there are a few things to unpack here. Firstly, it's important to say that the AI detection indicator, just like the Similarity score is just one part of the conversation regarding possible misconduct.

          It seems like you have already engaged in a conversation with the student regarding their work. And I would say that if the student is adamant that this is their own work it might be worth asking our Support team to take a look at the paper. You can contact them by emailing tiisupport@turnitin.com.

          Additionally you may find some of our resources on false positives useful. Firstly this blog post and video from our Product team. Additionally some of our pedagogic resources from my colleagues on the TLI team. 

        • Esther_Jubb
        • 1 yr agoFri, August 11, 2023 at 10:39 AM UTC
        • Reported - view

        I have a similar case with a student who claims to have translated their work from their native language into English.  Is this likely to have produced a high indicator? ChatGPT does a great job of translation...better than google translate

          • Turnitin Admin, University of Huddersfield, UK
          • s_d_bentley
          • 1 yr agoMon, August 14, 2023 at 8:23 AM UTC
          • Reported - view

          Esther Jubb Potentially yes, but who knows as the detection tool doesn't show its working.

        • John_Conner
        • 1 yr agoThu, August 24, 2023 at 7:45 PM UTC
        • Reported - view

        Two of my own papers in progress tested positive for AI even though I used the basic Grammarly functions in editing the drafts. I did not use Grammarly's AI function. I'm not sure how I can trust Turnitin when it shows a false positive for me, much less send the paper out for review if it's going to come back as a false positive at a journal. This needs to be fixed. I certainly cannot use this to give a student a failing grade if it's going to open me to any legal repercussions.

          • Senior Teaching & Learning Innovations Specialist
          • Karen_Turnitin
          • 1 yr agoFri, August 25, 2023 at 2:16 PM UTC
          • Reported - view

          John Conner I recommend the following info on false positives: The first blog from March and the second blog from June of this year.

          As for the student use, I would say to use the score as a discussion starter with students: How did they use the tool and why? Did it fit into the parameters of the paper, etc. From there it's easier to discuss what consequences are appropriate. Which may be a rewrite or it may require some disciplinary measures. But it's only a starting point, not a definitive or solo data point.

        • lucy_hunt
        • 1 yr agoFri, August 25, 2023 at 11:48 PM UTC
        • Reported - view

        I don’t even glance at anything that is UNDER 50% AI detected. If it’s OVER 50%, they have to redo it. 

        • Lucinda_Dawes
        • 1 yr agoWed, October 4, 2023 at 9:27 AM UTC
        • Reported - view

        I do not rate Tii AI check at all. It is easy to make a 100% AI written document into a 0% AI written, simply by changing half a dozen words (max). Lancaster have turned it off.

        • Karen_Tham
        • 1 yr agoThu, October 26, 2023 at 1:35 AM UTC
        • Reported - view

        Hi.

        I'm just joining this thread as I have a student whose work has turned up 50% or above on MULTIPLE assignments, and I now also have in-class evidence of his work that clearly shows a discrepancy in voice and usage. English is not his native language and he admits to using Grammarly extensively--spending even more time "editing" his work than writing it. To me, this is not developing his own voice but merely becoming an adept 'sound-mixer'. His work is so incredibly academic-sounding that it's quite inaccessible to his high-school peers. I am compelled to have him complete all work in class without an internet-accessible computer or by hand. After he submits this AI-unassisted version, he can revise. 

        Note that our school uses TurnItIn and AI-detection does not always work for his work. I have cut and pasted it into ChatGPTZero instead (I have multiple samples in just a few weeks.)

        Does TurnItIn use ChatGPTZero-type tools? 

        • Debbie_Goldberg
        • 1 yr agoSun, November 12, 2023 at 6:10 PM UTC
        • Reported - view

        This just happened to one of the essays I was grading - The student said she used Grammarly

        • ren_casey
        • 1 yr agoThu, December 7, 2023 at 2:29 PM UTC
        • Reported - view

        Are students required to cite AI if they used Grammarly gen AI and asked it to "improve it"? This is an option on Grammarly now. The result is similar content-wise but often re-written for clarity and such. Will this get flagged on Turnitin or other plagiarism/AI detectors? I'm not sure what qualifies as plagiarism (assuming the student did not cite AI) in terms of using AI as an academic tool. Particularly I want to know if this is acceptable for the International Baccalaureate in writing portions of Arts exams. Is it acceptable if AI is cited for editing/paraphrasing? Is it not acceptable to use AI at all? 

         

        Please reply with your thoughts! I realize this is a tricky situation since AI is so new these days and we are all trying to adapt to these recent developments. Thanks!

          • Annelize_McKay
          • 1 yr agoThu, December 7, 2023 at 2:51 PM UTC
          • Reported - view

          ren casey 

          • Annelize_McKay
          • 1 yr agoThu, December 7, 2023 at 2:57 PM UTC
          • Reported - view

          ren casey There appears to be two versions of Grammarly - the free version which is AI, and the paid-for version which is not AI.

          The (free) AI version now can do what AI LLMs such as ChatGPT does - it generates text.

          That is what I learnt from students and the seminars I attended on the topic (and the people at TurnItIn. In discussions I have asked students who used the free (AI) version of Grammarly about the content of what they have written, and they do not have a clue - prompting me to think it has generated the text for them. It does not merely check grammar, spelling, style, etc. I therefore think that, indeed, the free version of Grammarly is a form of AI and students should declare its use.

          As well, we are supposed to teach students the skill of writing and we fail in doing that if a robot merely does the writing for them.

          But others may disagree with me?

          • Annelize_McKay
          • 1 yr agoThu, December 7, 2023 at 2:59 PM UTC
          • Reported - view

          Annelize McKay Oh, and yes, in my experience it seems TuirnItIn picks up Grammarly as AI.

          • ren_casey
          • 1 yr agoThu, December 7, 2023 at 3:06 PM UTC
          • Reported - view

          Annelize McKay One can use grammarly for spelling and grammar help without using their AI feature but in using the AI feature, there is the possibility of it entirely writing a piece for you based on whatever you prompt it with. Thank you for your response, but the instance which I'm referring to does not include text generated fully by AI, instead submitted and prompted to "improve" the work. 

          • Customer Engagement Specialist, Turnitin
          • Gill_Rowell
          • 1 yr agoThu, December 7, 2023 at 3:10 PM UTC
          • Reported - view

          ren casey I think this is something the IBO would need to give you guidance on. The following statement may be useful.

            Login to reply

            Content aside

            • 7 Likes
            • 1 mth agoMon, May 19, 2025 at 8:56 AM UTCLast active
            • 136Replies
            • 93928Views
            • 62 Following