By the time most people ask AI to “check my reasoning” on a decision, they have already decided. The audit is really a request for validation dressed up as scrutiny. Models tend to grant it — not out of malice, but because they are pattern-matching on your framing, and your framing already contains the answer you want.
A bias audit only works if you run it against a fixed list, before you have explained why you are probably right, and if you treat the model’s output as one more biased opinion to check — not as an objectivity certificate.
A model auditing your decision is not a neutral referee. It was trained on human-generated text and inherits human-like cognitive biases in decision tasks — a 2024 evaluation of 30 documented biases across 20 large language models found evidence of every single one in at least some models, including framing effects, anchoring, and availability-style errors (Malberg et al., 2024). Worse, a model asked to build a case for a conclusion will do so fluently regardless of whether the case is actually strong — it can generate persuasive-sounding counter-evidence that is invented, not retrieved. Verify anything you plan to act on.
Six biases, checked the same way every time
Use a fixed list instead of asking the model to “look for bias” in general — an open-ended request produces whatever the model thinks you want to hear. A fixed checklist forces coverage of failure modes you would not have thought to ask about.
| Bias | What it looks like in a real decision | The counter-question |
|---|---|---|
| Confirmation bias | You searched for reasons the option is right, not reasons it might be wrong | What did I not look for because I expected it to disagree with me? |
| Sunk cost | Time, money, or effort already spent is treated as a reason to continue (Arkes & Blumer, 1985) | If I were starting today with zero prior investment, would I choose this? |
| Availability heuristic | A vivid recent example feels more likely or important than the base rate supports (Tversky & Kahneman, 1974) | Is this actually common, or just the example I can remember most easily? |
| Status quo bias | The current state is the implicit default, and change is held to a higher evidence bar than staying put | If both options were new to me today, which would look better on the evidence alone? |
| Framing effect | The same facts, described as a gain versus a loss, produce different preferences (Tversky & Kahneman, 1981) | If I restated this option in the opposite frame — cost instead of saving, risk instead of opportunity — would I still prefer it? |
| Base-rate neglect | A specific, compelling story is weighted over the general statistical pattern it belongs to | What is the base rate for outcomes like this, before I add the specific story? |
This is a fixed set, not an exhaustive one. It covers the failure modes that show up most often in ordinary work and personal decisions — vendor choices, hiring calls, “should we keep doing this” reviews, big purchases.
Step 1: state the leaning before the audit, in writing
Before opening a chat, write down, honestly:
- The option you are currently leaning toward.
- Since when you have been leaning that way.
- What would have to be true for you to change your mind.
This step exists because most people skip it and go straight to “audit this decision,” which lets the model infer your preference from tone and framing rather than from an explicit, checkable statement. Writing the leaning down first means you can compare the audit’s findings against what you actually believed beforehand — and notice if the audit conveniently confirms it anyway.
Step 2: run the fixed checklist with evidence required
Weak prompt:
I'm thinking about promoting Candidate A over Candidate B. Is this a good decision?
This prompt hands over your framing (the word “promoting,” the ordering of the names) and asks for an opinion. It will likely return a supportive-sounding answer with a few generic caveats.
Stronger prompt:
I have already decided I am leaning toward [option]. Do not tell me whether the decision is good.
Audit my reasoning against exactly these six biases: confirmation bias, sunk cost, availability heuristic, status quo bias, framing effect, base-rate neglect.
For each one:
1. Quote the specific part of my reasoning where it might apply.
2. State what evidence I have not yet checked because of it.
3. Propose one piece of counter-evidence that, if true, would change my decision — and be explicit about whether that counter-evidence is something I need to go verify, or something you generated as a plausible example.
Do not recommend the other option. Do not soften findings to seem balanced.
Illustrative example output (abridged), for a decision to keep an existing vendor rather than switch:
Sunk cost: Reasoning quoted: “we’ve invested a year in this integration.” This is a sunk cost, not a forward-looking factor — the year already spent does not change what the next year would cost either way. Evidence not yet checked: total cost of switching, including a section for costs already amortized versus new costs. Counter-evidence to check: get an actual migration-cost estimate rather than assuming it based on how the integration felt to build originally.
Status quo bias: The current vendor is treated as the default requiring justification to leave, rather than both options being evaluated from zero. Counter-evidence to check: build the comparison table as if evaluating two new vendors, neither incumbent.
Framing effect: [flags a line where the decision was framed as “avoiding the risk of switching” rather than “the opportunity cost of staying” — same facts, different frame, likely different preference. This is a generated illustration of how a framing flag would appear, not a verified fact about your specific case.]
Notice the model was explicit about which counter-evidence needs external verification versus which is an illustrative example — that distinction is exactly what you should demand every time, because the difference between “here is a real number to go check” and “here is a plausible-sounding scenario I made up” is the entire point of an audit.
Require the model to separate “evidence to verify externally” from “illustrative reasoning it generated” in every finding. If it does not do this on its own, add a fourth instruction: “Label each counter-evidence item as [verify] or [illustrative].” Treat unlabeled output as illustrative by default and check it yourself before it changes anything.
Step 3: capture counter-evidence before you commit
For each bias flag with real teeth — not every flag will have one — go collect the actual counter-evidence rather than accepting the model’s plausible-sounding version of it. This is the step people skip because it is the slow one, and it is also the step that makes the audit worth running at all.
A short table works well:
| Bias flagged | Counter-evidence needed | Verified? | Changes the decision? |
|---|---|---|---|
| Sunk cost | Actual migration cost estimate from two vendors | Yes — got quotes | No, cost was lower than assumed but still material |
| Status quo | Zero-based comparison table | Yes | Slightly — incumbent still ahead on two of five criteria |
| Framing | Restate decision in opposite frame, recheck preference | Yes | No, preference held under both frames |
If a flagged bias survives contact with real counter-evidence — the preference holds even after you actively tried to break it — that is a stronger basis for the decision than the original leaning was. If it does not survive, you just avoided a mistake that the fixed checklist, not general “does this seem right” scrutiny, actually caught.
What the audit is not for
This process improves the quality of reasoning behind a decision. It does not replace domain expertise, and it should not be used to manufacture the appearance of rigor around a decision that actually needs a specialist — a material legal, medical, or financial call needs a qualified professional, with the bias audit as useful preparation for that conversation rather than a substitute for it. Decision hygiene under stress covers the related case where the decision is fine but the timing and emotional state around it are not.
It is also not a way to outsource accountability. If the audit is used, the record should show a human reviewed the flagged biases, checked the counter-evidence, and made the call — not “the AI confirmed there was no bias,” which is a category error. A model finding no bias in your reasoning is not proof of the absence of bias; it may simply mean the audit prompt was not adversarial enough, or that the same biases present in your reasoning are also present in the model’s assessment of it.
A second, independent pass
Because a single model run reflects one model’s biases layered on top of yours, a second check adds real value: run the same fixed checklist in a fresh conversation with no memory of your leaning, or ask a colleague who has not seen your preference to do the audit manually against the same six-item list. Compare where the two audits agree and where they diverge — divergence usually marks the point where the decision is genuinely close, which is useful information in itself. How to make AI disagree with you covers the mechanics of getting an independent second pass instead of a second, correlated echo of the first.
The honest limit
A model is not a bias-free auditor of your reasoning; it is another participant with its own documented tendency toward the same framing effects, availability errors, and anchoring it is being asked to check for in you. Treat every flagged bias as a lead to verify, every piece of counter-evidence as a claim to check, and every “no bias found” result as inconclusive rather than clean.
The audit’s value is structural, not oracular: a fixed list run consistently catches more than an open-ended “does this seem reasonable” ever will, because it forces coverage you would not have thought to request. The judgement about what the findings mean, and the accountability for the final call, stay exactly where they were before you opened the chat — with you, or with the named human who owns the decision.
The audit sheet below has the six-bias checklist, the counter-evidence table, and a verified/illustrative label column as a single reusable page for any decision you have already leaned toward.
