machine minds

I Think, Therefore I Might Be True

August 25, 202610 min read

When answers become cheap, verification becomes valuable.

A few days ago I was studying an AI-generated image, one of those dense collages crowded with references to companies, systems, and the general machinery of the moment. In one corner, occupying maybe five percent of the frame, stood a small cartoon robot at a whiteboard. It had written: “I think, therefore I might be true.”

It was almost certainly a throwaway joke. I have not been able to throw it away.

Article content

Descartes, to be fair to him, never claimed that whatever we think must be true. The cogito runs the other direction: the act of doubting proves that at least one thing exists, namely a doubter. It is the most defensive statement in the history of philosophy, the single claim that survives when you distrust everything else. Generative AI hands us the inverted problem. We now have machines that produce reasoning-shaped, knowledge-shaped output in industrial quantities, while every individual claim sits in the category of might be true until something establishes why anyone should believe it.

That gap between what can be asserted and what can be trusted is on its way to becoming one of the defining economic facts of the AI era.

When answers were expensive

For most of the history of knowledge work, answers cost something. To produce a market analysis, someone gathered sources, read them, compared them, wrote the thing, and then defended it in a room. To produce working software, someone understood the requirements, wrote the code, ran it, and debugged it. To write a legal brief, someone found the cases. The effort was annoying, and it was also a feature: the cost of producing an answer acted as a natural throttle on the number of assertions entering the system.

Generative AI removes the throttle. A model can produce twenty candidate architectures, fifty market observations, a hundred code changes, or a confident policy summary in the time it once took a human to open a blank document and feel bad about it still being blank. This is not a weakness. It is the entire source of the technology’s value.

Article content

But it creates an asymmetry that most productivity conversations skip past: the marginal cost of producing a plausible answer is collapsing much faster than the marginal cost of establishing that the answer deserves confidence. The code still has to run. The citation still has to exist. The financial figure still has to reconcile. The regulation still has to say what the summary claims it says. Reality remains stubbornly involved, and reality does not scale at the rate of token generation.

This is not a hypothetical shift. It is already visible in how people work. A Microsoft Research study presented at CHI 2025 surveyed 319 knowledge workers and analyzed 936 real examples of generative AI use. The researchers found that critical-thinking effort moved from information gathering toward information verification, from problem-solving toward integrating the AI’s responses, and from executing tasks toward stewarding them. The labor AI removes from the front of the workflow does not vanish. It reappears later, wearing the less glamorous name of judgment.

Uncertainty in a very good suit

Verification would be a manageable engineering problem if humans evaluated claims the way auditors evaluate ledgers. We don’t. We use shortcuts, and the shortcuts are precisely what fluent AI output exploits, not maliciously but structurally. It’s an exercise in persuasion.

Article content

Cognitive scientists have spent decades documenting the illusory truth effect: repetition and ease of processing make statements feel more true. A 2010 meta-analysis pooled 51 studies on it, and a 2026 systematic review in Nature Communications revisited the effect and found it robust. The literature is more nuanced than “smooth talk convinces everyone,” and it has not specifically proven that eloquent LLM prose manufactures belief. But it establishes the uncomfortable baseline: when humans judge truth, coherence, familiarity, and confidence all get counted as evidence, even though none of them are.

Now consider what a language model produces at near-zero marginal cost: grammatical precision, clean structure, confident tone, plausible detail, instant responsiveness. Every surface cue we associate with competence, none of which is provenance.

Before generative AI, bad information usually carried warning labels. It was badly written, internally inconsistent, oddly formatted, or visibly speculative, the intellectual equivalent of a phishing email with typos. AI removes the typos. Uncertainty can now arrive wearing a very good suit, and the doorman waves it through.

The standard reassurance is that a human stays in the loop. But research on automation bias keeps delivering the same inconvenient finding: a human in the loop is not automatically an effective human in the loop. If the reviewer defers to the system, lacks the expertise to challenge it, or is buried under the volume of things to check, the oversight is nominal. A checkpoint staffed by someone who waves everything through is indistinguishable from no checkpoint, except that it produces an audit trail implying otherwise.

The debt nobody books

So what happens when organizations generate faster than they verify? They borrow. The term of art is epistemic debt, coined in manufacturing research back in 2019 and now applied, with increasing urgency, to AI-assisted work. It describes the accumulated cost of relying on claims whose foundations were never properly established.

The mechanism is mundane, which is what makes it dangerous. A generated claim enters a report. The report feeds a strategy deck. The deck informs a budget. The budget shapes a product decision. At each step the claim grows more embedded and its original evidentiary status grows harder to see. Nobody along the chain did anything wrong, exactly. Each person inherited a statement that looked settled. Coherence compounds even when evidence does not.

Article content

Software teams are running the same experiment at higher speed. AI can generate code faster than a team develops a shared mental model of what the code does and why. Tests confirm behavior without confirming intent. Documentation explains the system fluently while reproducing the same mistaken assumption as the implementation. The codebase grows. The understanding doesn’t. Anyone who has inherited a legacy system knows how that loan gets repaid, and at what interest rate.

Epistemic debt is not “the AI made a mistake.” Mistakes are cheap and correctable. Epistemic debt is “we built six months of downstream work on a claim nobody ever checked,” and it is booked nowhere until it comes due all at once.

The objections worth taking seriously

These three pushbacks deserve better than a dismissive footnote.

  1. Humans are wrong too. Constantly. Humans hallucinate, misremember, and radiate unearned confidence in meetings every day. But the argument was never that AI must clear a bar of infallibility humans don’t clear. The question is system-level: which human-AI workflow produces trustworthy decisions at acceptable cost, and where in that workflow should the checking live? The old throttle at least forced a human to touch the evidence on the way to the assertion. Remove the throttle and the touching has to be designed back in somewhere.
  2. Verification will be automated too. Yes, and this is the right long-term bet. But notice what it implies: verification stops being a final step performed by a tired reviewer and becomes an architecture, layered and risk-tiered, with failure modes of its own. That is not a rebuttal of the argument. It is the argument.
  3. Better models will hallucinate less. Almost certainly. But per-answer error rates and total exposure move independently. When generation gets cheaper, volume explodes. An organization producing a thousand times more machine-written claims at a fifth of the error rate is carrying two hundred times the unverified surface area. Falling error rates make verification more tractable, not less necessary. Safer cars did not eliminate the need for brakes.

Engineering humility

The answer is emphatically not to have humans re-inspect every token. That model burns the entire productivity gain and replaces it with the world’s most boring job. The answer is to make verification a property of the system rather than a heroic act of individual diligence.

Article content

Some of it is deterministic and already sitting on the shelf: compilers, test suites, schemas, static analysis, database constraints, calculators. These tools have always been on the shelf; what’s still needed is the engineer review to use the right ones to create those verifications.  These are instruments that don’t find prose persuasive. Some of it is provenance: retrieval from authoritative sources with the citation trail preserved, so the evidence travels with the claim instead of being amputated at the first copy-paste. Some of it is newer science: researchers at Oxford published a method in Nature that uses semantic entropy, a measure of how much a model’s meanings vary across generations, to detect likely confabulations, so the system can abstain or escalate instead of bluffing. NIST’s generative AI risk framework now lists confabulation as a distinct risk with recommended controls, which tells you this has left the realm of conference-talk speculation and entered the realm of compliance.

But the deepest requirement is a design posture. We should want systems that distinguish between statements the way a careful colleague does: I found this in an authoritative source. I inferred this from the evidence available. This is an estimate. There are competing interpretations. I cannot verify this. Call it epistemic humility. A system that can say “I don’t know” is more valuable than one that compresses sourced facts, inferences, and guesses into the same confident paragraph. The confident paragraph forces every downstream reader to do the sorting the machine declined to do.

Call it epistemic humility. I argued for this last year in Epistemic AI, and what the verification economics add is the business case. A system that can say “I don’t know” is more valuable than one that compresses sourced facts, inferences, and guesses into the same confident paragraph. 

Time-to-trust

Which brings us to the metric, and to what leaders should actually do with all this.

Today, AI productivity is celebrated almost entirely as time-to-first-answer: how fast the draft appeared, how many lines were generated, how many documents were summarized. But if the draft then consumes hours of checking, or worse, consumes none and ships, time-to-first-answer is measuring the wrong end of the pipeline. It’s the metric equivalent of judging a restaurant by how quickly the food leaves the kitchen.

Article content

The better measure is time-to-trust: how long it takes to move from a plausible output to an output a person or system can responsibly act on. That number is a property of your workflow, not of your model, and it responds to design. Can evidence travel with the claim? Is uncertainty surfaced early, or exported downstream to become someone else’s incident? Do low-stakes outputs move fast while consequential ones trigger stronger validation, or does everything get the same theatrical once-over? A leadership team that starts asking for time-to-trust will make different investments than one still counting generated words: fewer demos, more instrumentation.

This also dissolves the tired question “should we trust AI?”

Trust is not a binary property of a model, to be granted or withheld like a security clearance. It is an outcome a workflow produces, claim by claim, at a cost you can measure and reduce.

Which is why I no longer read that little cartoon as a joke. “I think, therefore I might be true” is not a confession of weakness. It is the correct epistemic starting position, the same one Descartes adopted before rebuilding everything on the one claim he could actually defend. The danger was never that the machine says might. The danger is when the machine, or the organization deploying it, quietly promotes might to therefore because the prose was smooth and the quarter was short.

Article content

Generative AI has made answers abundant. The next competitive advantage belongs to whoever makes justified confidence abundant too. When everyone can generate an answer, the valuable question becomes: why should anyone believe it?

Article content


Further Reading

  1. Systematic review and meta-analysis of the evidence for an illusory truth effect and its determinants — Ye et al., Nature Communications, February 2026. The largest synthesis of the fluency-and-truth literature to date, underpinning the “fluency is not evidence” argument.
  2. Why Language Models Hallucinate — Kalai, Nachum, Vempala & Zhang (OpenAI), September 2025. Argues that hallucinations persist because evaluations reward confident guessing over calibrated abstention, the incentive problem the “engineering humility” section addresses.
  3. The verification economy is redefining productivity — Evan Reiss (Foxit), TechRadar Pro, May 2026. Industry research finding that time saved generating content is being absorbed by the time required to trust it, with executives netting only minutes per week.
  4. Mitigating “Epistemic Debt” in Generative AI-Scaffolded Novice Programming using Metacognitive Scripts — arXiv, February 2026. Recent research applying epistemic debt to AI-assisted programming and testing interventions that force understanding to keep pace with generation.

Disclaimer: The perspectives shared in this article are our own and do not represent those of our employer or any affiliated organizations. All company names, product names, logos, and brands mentioned are the property of their respective owners and are used for identification and illustrative purposes only. No endorsement, sponsorship, or affiliation is intended or implied. References to specific companies or case studies are based on publicly available information and are used solely for educational and discussion purposes.