At the end of my . The content survives, but the source doesn't, so a good line drifts toward whichever speaker is the most famous. What AI does with plausible-sounding completions is, mechanically, remarkably close to what humans already do. What differs is scale, speed, and fluency.
Which means the countermeasures can be the same ones used for fact-checking a human. No special AI-specific defense is required — just the same unglamorous procedures that have always existed, applied without skipping steps.
The fix: a checklist for the reviewer
Here are the four rules I've put into practice:
Treat quotation marks and "there's a saying that goes..." as mechanically suspect. Before publishing, run a full-text search against the primary source for whatever is inside the quotation marks (grep for local files, site search for the web). If it doesn't come back as an exact, word-for-word match, it isn't a quotation.
If it doesn't hold up, rewrite it as indirect speech. "They express something to the effect of..." is an honest summary. Quotation marks are a claim of "exactly these words, verbatim" — not decoration.
Demand a clickable source for every AI conclusion. URL, page, line number — any of these will do. A plausible-sounding claim with no source to point to should be treated as suspect on that basis alone, independent of whether it sounds right.
Don't settle for asking "did you really say that?" Re-confirming with the AI is not verification. The actual cross-check has to be built into a human step (or some other independent verification step) in the process.
Catching both of these in the same day wasn't because I'm smarter than the AI — it's simply that this one narrow domain (talk around investing) happens to be somewhere I have some accumulated ground-level familiarity from long exposure. Which also means: in a field where I don't have that familiarity, running this checklist mechanically is the only defense there is.
Closing
If I had to compress the reviewer's posture into one line, it would be this:
Whenever a passage is landing a little too smoothly, that's exactly when to ask, "wait, where did that actually come from?"
Because the spot where prose lands most cleanly is the same spot where fabrication slides in most easily.
One more disclosure, since it would be embarrassing if this very article fell into the trap it's describing: every quoted AI output referenced above was cross-checked against the actual logs from that day, and the Matsuura Seizan attribution was cross-checked against multiple sources, before publication. And yes — this article, too, was written in collaboration with AI and reviewed by a human. The review process caught things to fix here as well, not zero of them.
Originally published in Japanese on Zenn: https://zenn.dev/idonthaveapen/articles/ai-fabricates-maxims-caught-twice
SOCIAL SHARE CARD GENERATOR