University of Kansas Researchers Develop Near-Perfect AI Detector

(Photo credit: Mike MacKenzie/Flickr)
(Photo credit: Mike MacKenzie/Flickr)

Sign up for smart news, insights, and analysis on the biggest financial stories of the day.

If you think all you need to do to fly through college is master ChatGPT, think again.

Last week, scientists from the University of Kansas released a study on an algorithm that reportedly detects ChatGPT with a 99% success rate. So, students, no cheating. Everyone else, you’re in the clear — for now.

Now You’re Speaking My Large Language

A lot of AI content — like this wacky film script about a Tibetan monk who tries to stop Dave Chappelle from stealing the Pulitzer Prize — is easy enough to look at with the naked eye and say, “OK, clearly, a computer made this.” But something as ridiculous as Puffer Pope stumped plenty of viewers and even some programs specifically designed to identify fake images.

While AI has everyone from writers to coders to investment bankers fearing for their livelihoods, students happily embrace large language models. AI can be an extra tool in the learning process, but when a survey from Study.com revealed that roughly 90% of college students have used ChatGPT to help with a homework assignment, a high potential for cheating is suspected. LLMs are the new copy-and-paste, which itself was the new “I’ll just watch the movie,” and AI-detecting programs are slowly getting better at parsing out what’s real and what’s fake:

  • In the University of Kansas study, 64 legitimate scientific articles were fed to ChatGPT, which was asked to create 128 news articles, and then each paragraph was used as the base for the AI detection algorithm. Then they test that algorithm on 30 more real articles and 60 ChatGPT pieces.
  • When combed in their entirety, all of the 60 fake articles were flagged as AI content, researchers said. On the paragraph level, accuracy dropped to 92%. That might be more impressive than Shadowbane.

Those are some pretty impressive stats, but the algorithm is so geared toward scientific academic writing, that it won’t work in other settings. So any high school American lit teacher still needs to stay on their toes when reading essays on The Great Gatsby. Did the student really know the green light symbolized his hopes and dreams, or was it just ChatGPT?