Students submitted more than 22 million papers in the past year that may have used generative AI, new data released by plagiarism detection company Turnitin reveals.
A year ago, Turnitin rolled out an AI writing detection tool trained on a treasure trove of student-written papers and other AI-generated text. Since then, more than 200 million of his papers, mostly written by high school and college students, have been peer-reviewed by Detector. Turnitin found that 20% of content may contain 11% AI-written language, and his 3% of all reviewed papers were more than 80% AI-written. has been flagged for doing so. (Turnitin is owned by Advance, which also owns WIRED publisher Condé Nast.) Turnitin says its detector has a false positive rate of less than 1 percent when analyzing entire documents.
When ChatGPT was launched, there were understandable concerns that English class essays would disappear. Chatbots can synthesize and extract information almost instantly, but they don't always get it right. Generative AI is known to create hallucinations by creating its own facts or citing academic references that don't actually exist. Generative AI chatbots were also caught spewing out biased text about gender and race. Despite these shortcomings, students have used chatbots for research, organizing ideas, and as ghostwriters. Traces of chatbots have also been found in peer-reviewed published academic papers.
It is natural for teachers to want to hold students accountable for using generative AI without permission or disclosure. However, this requires a reliable way to prove that her AI was used in a specific mission. Teachers sometimes tried to find their own solutions to detecting AI in writing, using messy and untested methods to enforce rules and tormenting students. Further complicating matters is that some teachers are using generative AI in their grading processes.
gen AI usage is difficult to detect. The generated text is still the original text, so flagging plagiarism is not easy. Additionally, there are nuances in how students use Gen AI. Some ask chatbots to write most or all of their papers, while others use the tool as an aid or brainstorming partner.
And students aren't just seduced by ChatGPT and similar large-scale language models. So-called word spinners are another type of AI software that rewrites text, potentially making it difficult for teachers to tell when work is plagiarized or generated by AI. Turnitin's AI detector has also been updated to detect word spinners, said Annie Chechitelli, the company's chief product officer. You can also flag rewritten work by services like spell checker Grammarly. Grammarly has its own generative AI tools. As generative AI components are added to familiar software, what students can and cannot use becomes increasingly confusing.
Detection tools themselves have a risk of bias. English learners may be more likely to dislike English. A 2023 study found that when seven different AI detectors were used to evaluate the Test of English as a Foreign Language (TOEFL) exam, the false positive rate was 61.3 percent. In this study he version of Turnitin was not investigated. The company said it trained its detector on writing by English learners and native English speakers. A study published in October found that Turnitin was one of the most accurate of 16 AI language detectors in a test that examined undergraduate papers and AI-generated papers.
