Asking a model to “check my reasoning” after stating a preferred option can anchor the response to the user’s framing. The output may repeat assumptions or produce plausible objections without retrieving evidence; it is not an independent psychological assessment.
A fixed checklist, an explicit initial leaning, and external verification can make the exercise more auditable than an open-ended request for validation. They do not guarantee that the audit detects bias or improves the decision.
A model auditing your decision is not a neutral referee. A 2024 preprint evaluating 30 documented biases across 20 large language models reported evidence of each bias in at least some tested models, including framing and anchoring effects (Malberg et al., 2024); that study is evidence about its tested models and prompts, not proof that every model behaves identically. A model can also generate plausible counter-evidence that was never retrieved. Verify anything you plan to act on.
Six biases, checked the same way every time
Use a fixed teaching list instead of relying only on an open-ended request. The checklist makes requested coverage visible; it does not force correct analysis or cover every relevant cognitive process.
| Bias | What it looks like in a real decision | The counter-question |
|---|---|---|
| Confirmation bias | You searched for reasons the option is right, not reasons it might be wrong | What did I not look for because I expected it to disagree with me? |
| Sunk cost | Time, money, or effort already spent is treated as a reason to continue (Arkes & Blumer, 1985) | If I were starting today with zero prior investment, would I choose this? |
| Availability heuristic | A vivid recent example feels more likely or important than the base rate supports (Tversky & Kahneman, 1974) | Is this actually common, or just the example I can remember most easily? |
| Status quo bias | The current state is the implicit default, and change is held to a higher evidence bar than staying put | If both options were new to me today, which would look better on the evidence alone? |
| Framing effect | The same facts, described as a gain versus a loss, produce different preferences (Tversky & Kahneman, 1981) | If I restated this option in the opposite frame — cost instead of saving, risk instead of opportunity — would I still prefer it? |
| Base-rate neglect | A specific, compelling story is weighted over the general statistical pattern it belongs to | What is the base rate for outcomes like this, before I add the specific story? |
This is a teaching set, not an exhaustive or clinically validated diagnostic instrument. It is suitable only as structured reflection on ordinary low-consequence decisions—not for labelling a person as biased, assessing capacity, or assessing mental state.
Step 1: state the leaning before the audit, in writing
Before opening a chat, write down:
- The option you are currently leaning toward.
- Since when you have been leaning that way.
- What would have to be true for you to change your mind.
This step makes the initial preference explicit instead of leaving the model to infer it from tone and framing. That creates a record you can compare with the audit’s findings, but the record does not prevent anchoring or confirmation bias.
Step 2: run the fixed checklist with evidence required
Weak prompt:
I'm thinking about promoting Candidate A over Candidate B. Is this a good decision?
This prompt hands over your framing (the word “promoting,” the ordering of the names) and asks for an opinion. Depending on the model and context, the answer may echo that framing, oppose it, or add generic caveats; none of those outcomes is an evidence audit.
Stronger prompt:
I have already decided I am leaning toward [option]. Do not tell me whether the decision is good.
Audit my reasoning against exactly these six biases: confirmation bias, sunk cost, availability heuristic, status quo bias, framing effect, base-rate neglect.
For each one:
1. Quote the specific part of my reasoning where it might apply.
2. State what evidence I have not yet checked because of it.
3. Propose one piece of counter-evidence that, if true, would change my decision — and be explicit about whether that counter-evidence is something I need to go verify, or something you generated as a plausible example.
Do not recommend the other option. Do not soften findings to seem balanced.
Illustrative example output (abridged), for a decision to keep an existing vendor rather than switch:
Sunk cost: Reasoning quoted: “we’ve invested a year in this integration.” The expenditure already incurred is sunk, but the integration or capability it created can still change forward switching, maintenance, retraining, and opportunity costs. Evidence not yet checked: a forward-looking comparison that separates past expenditure from future stay/switch cash flows, risks, and benefits. Counter-evidence to check: get an actual migration and operating-cost estimate rather than using the effort already spent as the deciding reason.
Status quo bias: The current vendor is treated as the default requiring justification to leave, rather than both options being evaluated from zero. Counter-evidence to check: build the comparison table as if evaluating two new vendors, neither incumbent.
Framing effect: [flags a line where the decision was framed as “avoiding the risk of switching” rather than “the opportunity cost of staying” — same facts, different frame, likely different preference. This is a generated illustration of how a framing flag would appear, not a verified fact about your specific case.]
Notice the model was explicit about which counter-evidence needs external verification versus which is an illustrative example. Require that distinction so generated scenarios are not mistaken for retrieved facts.
Require the model to separate “evidence to verify externally” from “illustrative reasoning it generated” in every finding. If it does not do this on its own, add a fourth instruction: “Label each counter-evidence item as [verify] or [illustrative].” Treat unlabeled output as illustrative by default and check it yourself before it changes anything.
Step 3: capture counter-evidence before you commit
For each material bias flag, collect the relevant counter-evidence rather than accepting the model’s plausible version. External verification is what distinguishes this workflow from a purely generated critique, but it does not guarantee that the evidence set is complete or correctly interpreted.
A short table works well:
| Bias flagged | Counter-evidence needed | Verified? | Changes the decision? |
|---|---|---|---|
| Sunk cost | Actual migration cost estimate from two vendors | Yes — got quotes | No, cost was lower than assumed but still material |
| Status quo | Zero-based comparison table | Yes | Slightly — incumbent still ahead on two of five criteria |
| Framing | Restate decision in opposite frame, recheck preference | Yes | No, preference held under both frames |
If the preference remains after relevant counter-evidence is checked, the record is more explicit than the original leaning, not necessarily correct or complete. If it changes, record which evidence changed it; do not attribute the improvement to the checklist alone without a comparison.
What the audit is not for
This process is designed to make assumptions and missing evidence more visible; this article does not establish that it improves a particular decision. It does not replace domain expertise, and it should not be used to manufacture the appearance of rigor around a decision that needs a specialist — a material legal, medical, or financial call needs a qualified professional, with the bias audit only as possible preparation rather than a substitute. Decision hygiene under stress covers the related case where timing or emotional state may require a different boundary.
Do not use the exercise as a substitute for mental-health assessment, crisis support, safeguarding, or decisions made under coercion or abuse. If fear, mania, severe depression, suicidality, intoxication, or another acute condition may be impairing judgment, pause the AI exercise and contact an appropriate licensed professional or emergency service. An argument from a model is not a capacity assessment.
It is also not a way to outsource accountability. If the audit is used, the record should show a human reviewed the flagged biases, checked the counter-evidence, and made the call — not “the AI confirmed there was no bias,” which is a category error. A model finding no bias in your reasoning is not proof of the absence of bias; it may simply mean the audit prompt was not adversarial enough, or that the same biases present in your reasoning are also present in the model’s assessment of it.
A second, independent pass
Because a single model run reflects one model’s errors layered on top of yours, compare the output with a colleague who has not seen your preference or with a qualified domain expert when stakes require it. A fresh model conversation may reduce conversational anchoring, but it is not an independent expert and can reproduce correlated errors. Investigate agreement and disagreement against evidence; divergence does not by itself mean the decision is close. How to make AI disagree with you covers one way to elicit counterarguments, not certify independence.
What the audit cannot certify
A model is not a bias-free auditor of your reasoning; it is another participant with its own documented tendency toward the same framing effects, availability errors, and anchoring it is being asked to check for in you. Treat every flagged bias as a lead to verify, every piece of counter-evidence as a claim to check, and every “no bias found” result as inconclusive rather than clean.
The audit’s intended value is structural, not oracular: a fixed list makes requested coverage visible and repeatable. Whether it catches more than an open-ended review is something to evaluate, not assume. The judgement about what the findings mean, and the accountability for the final call, stay with the named human or qualified professional who owns the decision.
The audit sheet below has the six-bias checklist, the counter-evidence table, and a verified/illustrative label column as a single reusable page for any decision you have already leaned toward.
