6

AI Detection and Grammarly

Hi everyone, We have faculty using the AI report and are appreciating that feature. An interesting question came up recently: the report showed 64% AI-generated text, but the student claimed he had only used Grammarly, which also helps to improve phrasing. Has anyone else experienced an AI report picking up on Grammarly, and is there a way of distinguishing between a "grammar help" tool and other generative AI?

Second, will the AI report be visible to students soon? Faculty assumed that students could see it, but the report is only visible to faculty in our LMS.

Thanks,

Jennifer Douglas, American Public University System

119 replies

null
    • TWC
    • 9 mths ago
    • Reported - view

    Also don't forget there is Grammarly premium for desktop and browser use. Many folks use this application or similar writing tools if they have dyslexia or chronic fatigue that can affect concentration.  Grammarly premium significantly changes sentence structure or whole paragraphs. 

    • Senior Teaching & Learning Innovations Specialist
    • Karen_Smith
    • 9 mths ago
    • Reported - view

    It helps if you think of the percentage as only one data point to have a discussion with students; ideally, that would take place during drafting, rather than submission. Two resources I'd suggest for this are this one on approaching a student when you think AI misuse has occurred or discussion starters for that situation. 

      • Turnitin Admin, University of Huddersfield, UK
      • s_d_bentley
      • 9 mths ago
      • Reported - view

      Karen Smith We're taking a view that the AI report alone is not sufficient to have that conversation. A "magic number" and no way to explain how that number has been reached is not a good basis to be making accusations - however gently they are framed. 

      Our approach is to require some additional evidence, such as falsified references or differences in writing style or standard of English compared with previous work before we act on the report, mainly because we don't want to damage staff-student relationships with this kind of conversation based on something that we can't back up and where it's the student's word against the Turnitin algorithm.

      • Senior Teaching & Learning Innovations Specialist
      • Karen_Smith
      • 9 mths ago
      • Reported - view

      Steve Bentley Yes, exactly. Many of our resources speak to the need for other data points. If I were still in the classroom, that number would nudge me to look more closely and have the conversations if it seems warranted. 

    • Angelo_Marin
    • 9 mths ago
    • Reported - view

    a foremost authority on AI detection is Atheer Mahir. He has a copious thread on this subject on his Linkedin profile

      • Joseph_Jacob
      • 6 mths ago
      • Reported - view

      Angelo Marin I concur with you. His articles suggest that these apps often misjudge content. His tests show AI texts seem entirely human, while older texts appear 80% AI-generated. This highlights detection tools' inadequacy and potential for false accusations.

    • David_Silver
    • 9 mths ago
    • Reported - view

    I have just joined this network and have reviewed the posts below. I have had some instance where TII has flagged students as having 50% AI generated content. However, some students are adamant they did not use AI or Grammarly/Quillbot. Are they just lying to me or are there any other possibilities as to why their work might be flagged? I had a student initially get flagged at 58%. They made numerous edits to the essay (as seen via google docs) and the resubmission still got flagged at 60%. I am having trouble understanding what might be the issue and would love some guidance. 

      • Customer Engagement Specialist, Turnitin
      • Gill_Rowell
      • 9 mths ago
      • Reported - view

      David Silver Thank you for joining us! I think there are a few things to unpack here. Firstly, it's important to say that the AI detection indicator, just like the Similarity score is just one part of the conversation regarding possible misconduct.

      It seems like you have already engaged in a conversation with the student regarding their work. And I would say that if the student is adamant that this is their own work it might be worth asking our Support team to take a look at the paper. You can contact them by emailing tiisupport@turnitin.com.

      Additionally you may find some of our resources on false positives useful. Firstly this blog post and video from our Product team. Additionally some of our pedagogic resources from my colleagues on the TLI team. 

    • Esther_Jubb
    • 8 mths ago
    • Reported - view

    I have a similar case with a student who claims to have translated their work from their native language into English.  Is this likely to have produced a high indicator? ChatGPT does a great job of translation...better than google translate

      • Turnitin Admin, University of Huddersfield, UK
      • s_d_bentley
      • 8 mths ago
      • Reported - view

      Esther Jubb Potentially yes, but who knows as the detection tool doesn't show its working.

    • John_Conner
    • 8 mths ago
    • Reported - view

    Two of my own papers in progress tested positive for AI even though I used the basic Grammarly functions in editing the drafts. I did not use Grammarly's AI function. I'm not sure how I can trust Turnitin when it shows a false positive for me, much less send the paper out for review if it's going to come back as a false positive at a journal. This needs to be fixed. I certainly cannot use this to give a student a failing grade if it's going to open me to any legal repercussions.

      • Senior Teaching & Learning Innovations Specialist
      • Karen_Smith
      • 8 mths ago
      • Reported - view

      John Conner I recommend the following info on false positives: The first blog from March and the second blog from June of this year.

      As for the student use, I would say to use the score as a discussion starter with students: How did they use the tool and why? Did it fit into the parameters of the paper, etc. From there it's easier to discuss what consequences are appropriate. Which may be a rewrite or it may require some disciplinary measures. But it's only a starting point, not a definitive or solo data point.

    • lucy_hunt
    • 8 mths ago
    • Reported - view

    I don’t even glance at anything that is UNDER 50% AI detected. If it’s OVER 50%, they have to redo it. 

    • Lucinda_Dawes
    • 6 mths ago
    • Reported - view

    I do not rate Tii AI check at all. It is easy to make a 100% AI written document into a 0% AI written, simply by changing half a dozen words (max). Lancaster have turned it off.

    • Karen_Tham
    • 6 mths ago
    • Reported - view

    Hi.

    I'm just joining this thread as I have a student whose work has turned up 50% or above on MULTIPLE assignments, and I now also have in-class evidence of his work that clearly shows a discrepancy in voice and usage. English is not his native language and he admits to using Grammarly extensively--spending even more time "editing" his work than writing it. To me, this is not developing his own voice but merely becoming an adept 'sound-mixer'. His work is so incredibly academic-sounding that it's quite inaccessible to his high-school peers. I am compelled to have him complete all work in class without an internet-accessible computer or by hand. After he submits this AI-unassisted version, he can revise. 

    Note that our school uses TurnItIn and AI-detection does not always work for his work. I have cut and pasted it into ChatGPTZero instead (I have multiple samples in just a few weeks.)

    Does TurnItIn use ChatGPTZero-type tools? 

    • Debbie_Goldberg
    • 5 mths ago
    • Reported - view

    This just happened to one of the essays I was grading - The student said she used Grammarly

    • ren_casey
    • 4 mths ago
    • Reported - view

    Are students required to cite AI if they used Grammarly gen AI and asked it to "improve it"? This is an option on Grammarly now. The result is similar content-wise but often re-written for clarity and such. Will this get flagged on Turnitin or other plagiarism/AI detectors? I'm not sure what qualifies as plagiarism (assuming the student did not cite AI) in terms of using AI as an academic tool. Particularly I want to know if this is acceptable for the International Baccalaureate in writing portions of Arts exams. Is it acceptable if AI is cited for editing/paraphrasing? Is it not acceptable to use AI at all? 

     

    Please reply with your thoughts! I realize this is a tricky situation since AI is so new these days and we are all trying to adapt to these recent developments. Thanks!

      • Annelize_McKay
      • 4 mths ago
      • Reported - view

      ren casey 

      • Annelize_McKay
      • 4 mths ago
      • Reported - view

      ren casey There appears to be two versions of Grammarly - the free version which is AI, and the paid-for version which is not AI.

      The (free) AI version now can do what AI LLMs such as ChatGPT does - it generates text.

      That is what I learnt from students and the seminars I attended on the topic (and the people at TurnItIn. In discussions I have asked students who used the free (AI) version of Grammarly about the content of what they have written, and they do not have a clue - prompting me to think it has generated the text for them. It does not merely check grammar, spelling, style, etc. I therefore think that, indeed, the free version of Grammarly is a form of AI and students should declare its use.

      As well, we are supposed to teach students the skill of writing and we fail in doing that if a robot merely does the writing for them.

      But others may disagree with me?

      • Annelize_McKay
      • 4 mths ago
      • Reported - view

      Annelize McKay Oh, and yes, in my experience it seems TuirnItIn picks up Grammarly as AI.

      • ren_casey
      • 4 mths ago
      • Reported - view

      Annelize McKay One can use grammarly for spelling and grammar help without using their AI feature but in using the AI feature, there is the possibility of it entirely writing a piece for you based on whatever you prompt it with. Thank you for your response, but the instance which I'm referring to does not include text generated fully by AI, instead submitted and prompted to "improve" the work. 

      • Customer Engagement Specialist, Turnitin
      • Gill_Rowell
      • 4 mths ago
      • Reported - view

      ren casey I think this is something the IBO would need to give you guidance on. The following statement may be useful.

      • ren_casey
      • 4 mths ago
      • Reported - view

      Gill Rowell Perhaps, yeah. They are quite vague in their statements about use of AI though.

      • Turnitin Admin, University of Huddersfield, UK
      • s_d_bentley
      • 4 mths ago
      • Reported - view

      ren casey As I understand it, Grammarly Go's GenAI is powered by ChatGPT, so the same rules and policies would apply, I would have thought. 

      I know we have written our policies and guidance not to refer to specific tools and to be at quite a generic level as principles to apply to any tool so it doesn't get out of date too quickly.

      Grammarly is just a slight special case because some students with Disability Support packages are provided with a license for it as a reasonably adjustment, so we need to set the boundaries a little more carefully there to avoid misunderstandings.

      • Hamza_Proviotech
      • 4 mths ago
      • Reported - view

      Steve Bentley could you shed some light on the issue that has been happening since 7th of  December where it has showing everything as ai detected. Even the papers i wrote before chat gpt etc. could you tell me the reason and how can we justify that to  our teachers and supervisors because it is hectic. Almost every institution is using Turnitin as their trusted vendor for AI and plag detection but this glitch is something that can lead many students like me in serious trouble. please sort this out asap. Thanks!

Content aside

  • 6 Likes
  • 2 wk agoLast active
  • 119Replies
  • 39445Views
  • 52 Following