How to Stop AI From Making Things Up: Verify First

Key takeaways

AI states a fabricated fact with the exact same confidence as a real one, so confidence is useless as a signal. Verify the claim, not the tone.
Asking a second AI to confirm a fact isn't verification. Two models can agree on a fiction. Open the primary source yourself, in a real browser, with your own eyes.
Step three is the whole workflow: open the exact URL, read the address bar, then Cmd-F on Mac or Ctrl-F on Windows for the exact phrase. Zero results means the fact doesn't ship.
Damien Charlotin's AI Hallucination Cases database held 1,761 documented court cases as of July 16, 2026, growing from just six through Q2 2023. It tracks courts, so read it as a canary for how common the failure is, not a full count.
Fabricated citations are the most recorded failure mode at 2,989 instances, ahead of misrepresented sources and false quotes, so hunt citations first.
If you can't confirm a claim against a real primary source, cut it or rewrite the sentence without it. Dropping an unverifiable fact costs nothing; shipping a fabricated one can cost the client.
As a solo operator you can't blame a junior or the tool the way sanctioned firms tried to. The only safe move is a ten-minute check before anything client-facing ships.

The Solo Creators AI Studio

Get this skill, and every one I build.

The whole vaultA new drop monthlyBuild live with meYours to keep, forever

See inside the Studio

30-day money-back guarantee. Cancel anytime.

Questions people ask

How do I actually stop AI from making things up?

You can't stop the model from generating plausible-but-false content, that's inherent to how it works. What you can control is whether a fabrication reaches your client. The fix is a checking step: pull every specific claim (prices, stats, quotes, sources) out of the draft, then open the primary source for each one in a real browser and confirm the exact sentence exists. If you can't find it, cut it.

Can't I just ask another AI to fact-check the first one?

No, and this is the trap that fooled me. A second model is another confident pattern-matcher, and two tools will happily agree on a made-up number. Their agreement feels like confirmation but it's an echo. Real verification means checking against the actual primary source, loaded in your own browser, not against a second opinion from the same kind of tool.

What's the fastest way to check a claim on a page I've opened?

Press Cmd-F on a Mac or Ctrl-F on Windows and paste in the exact number or the quoted words. If the browser finds zero matches, the claim isn't on that page. Before you search, glance at the address bar to confirm the link didn't redirect you to a homepage or a login wall, and on pricing pages toggle monthly and annual, because a discounted annual rate divided by twelve is a very common source of a wrong price.

What do I do if the source is paywalled, dead, or now says something different?

Paywalled: look for the same figure in the organization's own press release, annual report, or public summary page. Dead or redirected: treat the claim as unverifiable and cut it, because a source you can't open isn't a source. Changed since the model was trained: the current page wins, so use the current number and note the date you checked in the deliverable.

Why does opening the page in a real browser matter more than an AI-fetched summary?

The ways AI reads the web are the ways it gets fooled. Pages can show one thing to an automated reader and another to a human, split a price across separate elements so a text grab glues '$8' and '49' into '849', or redirect a fetcher to the wrong page entirely, after which the model fills the gap with a convincing fake. Your eyes on the real page catch what the extraction misses.

Is the risk of AI fabrication really that common, or just a lawyer problem?

The public database that documents this only tracks court cases, so it captures the fabrications serious enough to reach a judge. That's the visible tip. Every fabrication in an everyday coaching email, sales page, or client one-pager, where no court is watching, goes uncounted. Read the 1,761 documented cases as proof the mechanism is real and frequent, not as the full total. If anything, the everyday risk is larger.

What happens to people who get caught relying on a hallucination?

In the court database, consequences included 302 monetary sanctions and 125 disciplinary referrals. When caught, 76 blamed the tool, 57 blamed a junior, and 24 doubled down and defended the fabrication. None of those escape routes work for a solo operator. You chose the tool and put your name on the work, so the accountability lands on you, which is exactly why the ten-minute upfront check is worth it.

Which claims in a draft actually need verifying?

Anything specific and falsifiable: prices, dates, percentages and statistics, named people or companies, direct quotes, and any 'according to [source]' citation. Vague statements don't need a source. The rule of thumb: if it has a number or sits inside quotation marks, it gets checked against a primary source before it ships.

Found this useful? Share itPost Share

Written by Ronnie Nijmeh

Ronnie spent 18 years building a SaaS with a team of 20 that served over 650,000 customers, generated over $14M in sales, and sent over 550M emails. Now he's solo, solving real business bottlenecks and turning them into working AI skills, workflows, and automations. He teaches all of it, with direct access to him, inside the Solo Creators AI Studio Skool community. See what he's built →