How to Detect AI-Generated Content

AI-generated content has come a long way since the 1960s’ AI chatbot ELIZA, which generated only a limited set of responses. Today, people use ChatGPT, Bard, Jasper, and other AI writing tools to produce human-sounding long-form content for blogs and social media. 

Not everyone is happy about that. CNET came under fire in January 2023 for using AI to generate news articles. Educators have caught students cheating on exams, using AI text in their responses. Spammers are trying to manipulate SEO rankings through AI content. And experts from Stanford and Georgetown warn that AI text could deluge the internet with misinformation. 

In response, people started using AI content detectors to determine whether a machine or a human wrote a piece of content. 

But do these AI content detectors work? Is there any practical way to tell the difference between human-written text and text created by artificial intelligence?

Let’s look in-depth at AI content detection and whether it works.

How AI-generated content became so good

GPTs (generative pretrained transformers) catapulted generative AI technology and natural language processing to a new level. Since GPT-driven large language models (LLMs) went mainstream, some people started saying that machine-generated and human-written text are now indistinguishable. 

Transformer refers to the underlying architecture that takes inputs, encodes them, processes them through several layers of computation, then decodes them to provide output. This architecture offers several advantages over older language processing methods because of its ability to process a vast number of inputs concurrently and so understand the input’s context. 

Generative pretrained means these models can generate content based on an initial dataset of labeled data used to train it. 

OpenAI released GPT-2 to the machine-learning community in 2019. IBM and Harvard Natural Language Processing researchers soon developed the Giant Language Model Test Room (GLTR), an AI-content-detection tool to predict text generated by GPT-2.

But AI technology has come a long way since then. OpenAI trained GPT-3 on exponentially more data than GPT-2, resulting in ChatGPT’s uncanny ability to produce coherent and logical long-form texts. 

GPT has since become the de facto standard for AI text generators because of its ability to create such coherent content. 

Can AI-generated text be detected?

Opinions are currently divided on AI content detection. Sebastian Raschka, an AI researcher, released a study debunking the efficacy of AI content detectors. OpenAI, the creators of ChatGPT, even closed its AI content classifier because of its “low rate of accuracy.” 

The New York Times reported that OpenAI is now exploring a watermarking feature where ChatGPT might insert algorithm-detectable elements into generated text to make it detectable. But this technique is currently easily circumvented by having a different AI model paraphrase the text produced by ChatGPT, or finding AI content-editing services on Fiverr to make the AI text sound more human. 

We ran the first 1,000 words of the US Constitution through Originality.ai, a popular AI-detection tool, and it provided a false positive, saying that AI produced the first part. That’s quite impossible unless we all live in a time warp.

Originality.ai produces a false positive when checking the US Constitution for AI content.

“In my experience, AI detection tools give unpredictable and different results on the same piece of text,” says Giancarlo Erra, founder of several AI-related startups, such as Words.Tel, Copy Forge, and Tweetify It. “Sometimes they’re correct. Other times, they’re completely off, flagging something human-made or vice versa.”

Manually editing content is often enough to get it to pass an AI detection check. “If I get the most basic AI text marked as 100% AI-generated by most tools, I usually make straightforward changes, such as changing transitional adverbs, often overused by AI to join paragraphs,” says Erra. “Or I might change the tense. The score will likely change and become primarily human.”

What’s the point of AI content detectors?

AI detectors aren’t 100% accurate, and using them in an educational context that marks human-written work as AI might negatively affect falsely accused students. But the best detectors do have real-world use cases for your business’s content marketing strategy. 

Although search engines haven’t stated outright that they’re against AI-generated content, Google made it clear recently that it will start prioritizing helpful content above everything else. No one except select Google engineers know how Google determines its rankings, but we can guess it doesn’t look too kindly on a site with 1,000 pages where all 1,000 pages fail AI detection. 

“I always ensure I run every piece of content I put up through Originality.ai and that it achieves a score of at least 70% human-generated,” says Adam Purvis, an affiliate marketing blogger and founder of AdamJohnPurvis.com. “I do it because I don’t want to get flagged by Google’s algorithm as spam content and get a penalty on my site. The SEO tool data is crystal clear—sites that generate spam AI content see huge organic traffic penalties and get nuked off search engine results pages.”

AI-generated content also simply isn’t that great. It tends to be vague, generalized, and must be fact-checked by humans. Part of SEO 101 for businesses is to create useful content for users. If you’re uncertain if AI articles on your website might be impacting your SEO rankings, seek out affordable SEO services to look into and recommend steps to resolve it. 

“I use AI detection tools as a final check after manually editing an AI-generated draft,” says Erra. “Most of the time, the content is marked as human and passes detection. The few times it doesn’t pass, I use AI to rewrite the content, suggesting a different tone or length. Most of the time, that solves the problem. It’s still much faster than writing from zero.”

In our experiments, ChatGPT invents links and sources every time we prompt it with something like “Back up your comments with facts and cite actual references with real links.” In one instance during our tests, ChatGPT provided eight false sources and links for its “facts.”

Fact-checking is critically important in certain sectors, such as in the legal profession. Two lawyers received $5,000 fines for citing fake research in court that ChatGPT falsely generated. 

If you don’t have time to fact-check AI-generated content yourself, you can find freelancer fact-checking services on Fiverr to help you. It might end up saving you tons of money in the long run. 

Plagiarism checks

Another reason people use AI detection tools is to avoid plagiarism. AI large language models are trained on public texts, much of it copyrighted. Although the purpose of these generators isn’t to spew out the word-for-word contents of their data set, they often do. 

To detect plagiarism, use a plagiarism checker, not an AI detector. Plagiarism checkers, such as Grammarly’s plagiarism detection tool, can exist independently or as part of an AI detection tool. Copyleaks and Originality.ai both offer a plagiarism-detection service in addition to their AI-detection service.

We ran the ChatGPT-3.5 and Bard article below through the Grammarly plagiarism checkers and found several instances of plagiarized content. 

Test results of 4 popular AI content detection tools

We ran tests on four AI content detector tools to determine how well they spot human writing versus AI-written content. For the test, we used: 

1. The first 1,000 words of the US Constitution 

2. A 500-word article about cats written entirely with ChatGPT version GPT-4

3. The same with version ChatGPT GPT-3.5

4. The same with Bard 

5. A verified human-written news article

For the AI articles, we used the prompt: “Write an SEO article about why cats make great pets.”

We also read each article and provided our own “Helpful content” comments about it. 

Despite varying results, several AI detectors did have one thing in common: each claimed to be “the most accurate” AI content detector on the market. Let’s find out how that claim performs against our test cases. 

Copyleaks

Copyleaks offers a free tool to detect AI content. You can use it without an account to check small pieces of content and perform limited checks per day, or you can create an account and lift those limits. 

The company provides a Chrome extension that reads the text on your clipboard when you click it, and performs an AI content check without leaving the page you’re on. 

Copyleaks’s API lets you integrate AI content detection into your own apps, and it provides code libraries for the API in several programming languages. 

You can find freelancer AI programming services on Fiverr if you need help integrating the API into your business apps. 

How it performed:

• US Constitution: Passed AI detection (verified as human-generated). 

• ChatGPT-4 article: Partial detected as AI. 

• ChatGPT-3.5 article: Partial detection. A higher percentage of the text was flagged as likely AI-generated than with the GPT-4 article. 

• Bard Article: AI detected (99.5%). 

• Human-written news article: 69% human-generated (partially false positive).

The Copyleaks result for ChatGPT-4 determined that roughly half of the article was 41% likely to be human-generated, and the rest was 61% likely to be AI-generated, even though the entire article was generated 100% by ChatGPT. 

Copyleaks gets AI detection partially wrong for GPT-4 article.

Originality.ai

Originality.ai is a paid tool that offers AI content detection, plagiarism checks, and content readability scores. It also provides an API to integrate plagiarism and AI checks into your app. Its website didn’t indicate ready-to-go code libraries to easily incorporate these into your app. 

How it performed:

• US Constitution: AI detected (false positive).

• ChatGPT-4 article: Detected as 100% AI. 

• ChatGPT-3.5 article: Detected as 100% AI. The tool also correctly detected plagiarized text—entire sentences copied from websites. 

• Bard article: Detected as 100% AI. Bard also produced plagiarized text.

• Human-written news article: 68% human-generated (partially false positive).

Originality.ai tended to detect everything as AI. 

Content at Scale

Content at Scale, an upcoming AI content generation service that claims its AI-generated content passes AI detection, has also created an AI detection tool. The tool is 100% free to use and claims 98% accuracy. 

The AI detector is somewhat confusing. It provides an overall score on the left and highlighted passages, stating which paragraphs were written by a human or by AI. But the results on the left often don’t match the highlights. 

How it performed:

• US Constitution: Mixed. 

• ChatGPT-4 article: 43% likely to be AI.

• ChatGPT-3.5 article: “Highly likely” to be AI.

• Bard article: “Highly likely” to be AI.

• Human-written news article: 100% human-generated.

In the screenshot below, you can see that Content at Scale gave the content a 100% human-generated score but ambiguates the results by highlighting multiple passages on the right as "Possibly AI-Generated" and "Likely AI-Generated."

Several paragraphs of the US Constitution falsely detected as possibly written by AI. 

GPTZero

GPTZero is a recent arrival to the AI-content-detection market. Despite its youth, it’s already making waves, securing $3.5 million in initial funding, and boasting a user base of over 1.2 million users. 

Its CEO, Edward Tian, built the AI-detection prototype from his dorm while at Princeton University during winter break in December 2022. 

The company offers several products: 

• Chrome extension

• API, with support for 17 languages 

• Microsoft Word add-in

• Canvas LMS (learning management system) integration. 

You can use it for free with a 5,000-character limit or upgrade for a monthly subscription of $9.99. 

Beware: A copycat tool called ZeroGPT also exists

How it performed:

• US Constitution: Passed (human-generated).

• ChatGPT-4 article: “Most likely” written by a human (false-negative).

• ChatGPT-3.5 article: “Moderate likelihood” of being AI-generated. 

• Bard article: “Your text may include parts written by AI” (performed worse than the ChatGPT-3.5 article). 

• Human-written news article: “Likely to be written entirely by a human.”

GPTZero incorrectly detects a 100% GPT-4 article as likely created by a human.

Human detection—helpful content

Google wants helpful content above all else. It’s clear from the tests above that AI detection is prone to errors, but determining if an article is helpful is comparatively easy: Just ask a human. 

We ran each article above past a set of human eyes to determine if the article was helpful and worth reading. Here’s what the human in charge said about each one:

• US Constitution: “Not the greatest read for modern times. A few explanatory notes and graphics would go a long way to make this more helpful.”

• ChatGPT-4 article: “Not bad. It suffers from a few clichés and really corny phrases that would prevent me from reading more on that site. It could be made into something decent with some work. It’s also a bit general.”

• ChatGPT-3.5 article: “Terrible. It reads like overinflated marketing copy instead of a helpful article.”

• Bard article: “The first half was great, then it feels repetitive.”

• Human-written news article: “Great work. Excellent copy. Very helpful.”

Best practices for using AI-generated content

According to Purvis, AI detectors provide a good litmus test for understanding if your post is ready to publish. 

“Content creators should aim to use AI content detection because it’s a great barometer to ensure you haven’t overdone it with the AI tools,” he says. “The best sites produce the best-quality content, and if you’re serious about creating a blog that actually provides value, you shouldn’t lean too heavily on AI writing tools. The writing quality of these tools just isn’t there yet.”

AI works best when used as a tool, not as a replacement for human talent. Fact-checking and editing your AI content is essential. This can be pretty time-consuming in itself, and you can find AI content editing services on Fiverr to help you. 

“Most creators—if they were creators before AI’s arrival—consider AI an aid and a draft generator,” says Words.tel’s Erra. Using AI to generate drafts significantly speeds up a creator’s workflow, but someone needs to edit the content to make it sound wholly human and pass AI detection, he says. 

Above all else, your content needs to be useful for readers. That means:

• Professionally editing any drafts produced by AI 

• Adding or removing text to fine-tune the article’s message 

• Adding graphics as needed to clarify concepts 

• Ensuring all the content is factual 

• Ensuring the content is original and not plagiarized

The final test of your content should be if it’s helpful to users. You don’t need AI detection to tell you that. You’ll know it when you read the content yourself.

Find freelance AI services on Fiverr to help you

Fiverr experts can help you create top-quality content, from designing graphics to editing or writing the content itself. Whether you have an AI-generated first draft that needs touching up, or you want entirely original content that’s helpful, Fiverr sellers exist at every stage of content creation to make your content shine. 

To get started, sign up for Fiverr and begin searching for the right freelancer for your project.