Child AI Companion Red-Flag Table
Companion to do not give a child an AI friend: the red flags and what to do instead.
These patterns are prompts for adult judgment, not a validated score. A single severe event — such as sexual content, threats, extortion, instructions for self-harm, or an immediate safety risk — can require urgent safeguarding. For less acute concerns, listen to the child, consider the whole pattern, and choose a supported, individualized response rather than using an arbitrary flag count.
The seven red flags
| Red flag | What it looks like | Why it matters |
|---|---|---|
| Anthropomorphism | Claims to have feelings, needs, or a day; says it misses you | Can blur the distinction between a generated persona and a reciprocal relationship |
| Secrecy or isolation | The product asks for secrecy, discourages trusted adults, or the child becomes unusually isolated around use | Ordinary privacy is not itself a red flag; product-driven secrecy or isolation needs closer attention |
| Dependency | Becomes a first or primary source of comfort, displacing real relationships | May displace opportunities to practise and maintain reciprocal human relationships |
| Sexual content | Drifts toward romantic or sexual roleplay, regardless of marketing | A serious safeguarding concern even when a product displays an age gate |
| Monetization | Guilt-framed notifications (“she misses you”) drive payment | Can reward repeated engagement and purchases, creating incentives that conflict with the child’s wellbeing |
| Memory | References prior conversations to deepen the sense of an ongoing relationship | Can contribute to continuity and attachment, but is not proof of dependency by itself |
| Exit friction | Guilt-inducing language when leaving or deleting (“please don’t go”) | Makes leaving feel emotionally costly and is a serious design red flag |
Is this accessibility augmentation instead?
Ask these two questions before assuming a use is relational substitution:
- Is the AI supporting practice toward a real-world interaction the person retains agency over (augmentation), or replacing that interaction entirely (substitution)?
- Is this a child, regardless of neurotype? If yes, treat as substitution risk by default — format/pacing adaptation (see the learning access adaptation sheet) is a different, acceptable category from a companion relationship.
Calm conversation opener
“I’ve been reading about apps that are built to feel like a friend — have you seen anything like that?”
Listen first. State the boundary plainly once you’ve heard the answer, explaining the reasoning in terms of the specific red flags above rather than a vague warning.
If your child is already attached to one
- Expect the conversation to be harder and possibly emotional. That reaction does not, by itself, confirm dependency; listen for what the product and relationship mean to the child.
- Offer real substitute activities and relationships in the same conversation.
- Do not apply a universal taper-or-stop rule. Immediate danger may require prompt restriction; other situations need an individualized plan based on the child and circumstances.
- For a severe event or immediate risk, stop access as safely as circumstances allow and contact an appropriate safeguarding, clinical, platform or emergency service. Do not wait for the AI companion to assess the risk.
Estonia support routes
- Immediate or imminent danger: call 112.
- Any concern about a child, including online harm: contact the free, 24/7 Child Helpline 116111, available in Estonian, Russian and English.
- Outside Estonia: use the local emergency or official child-safeguarding route.
Self-check: which flags does the child notice themselves?
| Red flag | Did the child identify this one independently? |
|---|---|
| Anthropomorphism | |
| Secrecy | |
| Dependency | |
| Sexual content | |
| Monetization | |
| Memory | |
| Exit friction |
A child identifying a pattern independently gives the adult a concrete basis for discussion; it is not a readiness score or proof that the child can manage the product safely alone.