What AI image tools get wrong, and why detectors don't fix it

AI-generated and AI-edited images can contain useful visual clues, but the clues change and detector results depend on the models, edits, and test data involved. Use source and provenance checks instead of treating a score as certification.

What you should be able to do

AI image generation has real, learnable failure patterns - and a real, unsolved detection problem. Learn the patterns so you notice them. Do not rely on a detector score to settle a specific case, because no detector can guarantee it.

Saved only in this browser.
In this article

Somebody shares an image and asks, “is this AI?” The instinct is to look for the classic tells - too many fingers, garbled text, waxy skin - or to run it through one of the “AI detector” tools that have sprung up. Both instincts are reasonable. Both also fail in ways that matter: visible clues vary across models and images, and the detector tools carry a reliability problem that does not go away just because the interface looks confident.

This article covers what is actually true: the specific patterns where AI image tools still struggle, and why no detector - human or automated - can give you a guaranteed verdict on a single image.

The failure patterns that still hold up

The following categories are heuristics, not measured guarantees. They may justify a closer source check, but tool capabilities change and the absence of a clue says nothing conclusive about origin:

Hands and fine anatomy. Look for impossible joints, inconsistent fingers or overlapping limbs that do not fit the scene. These are observations to investigate, not a tested ranking of which body parts current tools handle best.

Embedded text. Signage, labels, and text within an image remain uneven across tools and versions - some generations look clean; others still show near-words, inconsistent lettering, or text that is crisp in one area and smeared in another. Treat “looks readable” as weak evidence either way.

Precise counting and exact repetition. Count the objects yourself when an exact count matters. A generated image or a model’s description can miss the requested number; verify against the actual image.

Consistency across multiple images or frames. Compare distinguishing details across the sequence. A changing object or character may warrant investigation, but edits, compression and ordinary scene changes also need considering.

Reflections, shadows, and physically consistent lighting. Check whether the reflections and shadows fit the depicted scene. An apparent inconsistency can have other explanations and does not establish synthesis.

None of these are reliable enough to build a firm verdict on their own - a skilled or lucky generation can avoid all of them, and a genuine photo can contain an odd shadow or an awkward hand. Treat them as things worth a second look, not proof. They also age quickly: verify against current tools rather than treating this list as permanent.

Why “AI detector” tools cannot give you certainty

NIST’s synthetic-content transparency report reviews detection, watermarking, provenance, and labelling as distinct approaches with research gaps and implementation limitations. NIST’s ongoing GenAI evaluation programme and image challenge evaluate generators and discriminators under defined test conditions. That is the correct frame for a detector result: performance for a specified system, dataset, transformation, and date, not certification of every image from every generator.

Do not use a detector tool’s score as the deciding factor in a consequential decision - a workplace dispute, a relationship conflict, a public accusation. A false positive can wrongly brand a genuine photo as fake; a false negative can wave through a convincing fabrication. Use detector output, if at all, as one weak input alongside everything else in this article - never as the verdict.

What actually helps, in order of usefulness

Provenance information, where it exists. A Content Credential under the C2PA standard can show who signed a file and what creation or editing history they asserted - genuinely stronger evidence than a detector’s guess, though it depends on the tool and platform supporting it and surviving the file’s journey to you. See Content Credentials and watermark basics for what that badge does and does not prove.

Finding the original source. For many images, tracing where the file first appeared, from whom, and when is more useful than inspecting pixels alone. A reverse-image search or direct search for the claimed event may surface original context, but a failed search is not proof of fabrication.

Consistency with what else is known. Does the claimed event show up anywhere else, from an independent source? Does the setting, clothing, or timestamp line up with what is otherwise verifiable? Record which independent evidence supports the claim and which parts remain unresolved.

The failure patterns above, as a supporting signal only. Worth a look, never worth a verdict on their own.

Where this applies beyond a single suspicious photo

This is the same reason synthetic media provenance and consent treats a Content Credential as useful-but-not-sufficient, and it is the direct answer to a common question in the adult deepfake first-response plan: do not delay reporting while trying to “prove” something is synthetic first.

Separate origin from the claim

Invented example: a photograph is shared as today’s flood. A reverse-image search finds the same image in an older original report. That mismatch is enough to challenge the date claim even if the image’s production method remains unknown. Record the earlier source and the disputed assertion; do not call the file AI-generated merely because its current caption is false.

If no earlier source is found, record “origin unresolved.” Absence of a credible original is not affirmative evidence of synthesis. A detector score or unusual reflection may prompt more checking, but cannot close that gap by itself.

An older original report challenges a fictional flood image’s 'today' caption without proving AI origin; a failed source search leaves origin unresolved.
In the article’s invented flood example, evidence about the caption’s date is separate from evidence about how the image was made. Open the full-size diagram.

Read next

Continue through the same learning path with the next practical articles.