Asking AI to rank a list of people feels like a small, practical convenience: which resume is strongest, which team member contributed most, even which friend to prioritize this month. It is also, structurally, the exact pattern that regulators around the world have identified as one of the most consequential ways AI can harm people, precisely because it looks so mundane on the way in.
Why this specific pattern gets special regulatory attention
The EU AI Act flatly prohibits a defined category of “social scoring”: AI systems that evaluate or classify people based on social behavior or inferred personal characteristics, where the resulting score leads to unfavorable treatment in a context unrelated to how the data was collected, or treatment disproportionate to the underlying behavior (EU AI Act, Article 5(1)(c), Regulation (EU) 2024/1689). That is a public-and-private-sector prohibition, not a rule limited to governments - the accompanying recital states plainly that such scoring “may violate the right to dignity and non-discrimination” regardless of who deploys it.
Employment scoring gets its own dedicated regulation in some jurisdictions precisely because the pattern recurs so often there. New York City’s Local Law 144 requires employers and employment agencies using an “automated employment decision tool” to screen or rank candidates or employees for a job located in New York City to ensure an independent bias audit was completed within the prior year and to publish a summary of the results; separately, candidates and employees who are New York City residents must be notified at least ten business days before the tool is used. Note which trigger is which: the audit and publication duties follow the job’s location, not where the applicant lives (NYC Department of Consumer and Worker Protection, Local Law 144 FAQ). The law exists because ranking people at scale, without independent verification, reliably produces bias that is invisible until someone goes looking for it.
That is not a hypothetical. In 2018, Reuters reported that Amazon had spent years building an internal AI recruiting tool that scored resumes, only to discover by 2015 that the system had taught itself that male candidates were preferable - penalizing resumes containing the word “women’s” and downgrading graduates of two women’s colleges - because it had been trained on a decade of resumes submitted to a male-dominated industry. Amazon’s engineers tried to fix the specific patterns they found, could not be confident the fixes would not surface new, hidden ones, and ultimately scrapped the project; people familiar with the effort told Reuters that recruiters had looked at the tool’s recommendations, while Amazon said the tool was never used by recruiters to evaluate candidates (Reuters, “Amazon scraps secret AI recruiting tool that showed bias against women,” 2018). This was a company with substantial machine-learning expertise, working on the problem deliberately, and the bias still emerged during years of development before the project was abandoned.
Why “just for my own use” does not make it safer
The regulatory examples above involve institutions, but the underlying mechanism - a model trained on unexamined historical patterns producing a confident-looking score that quietly encodes those patterns - does not require a company-scale system to cause harm. Asking a general-purpose AI tool to rank which of your reports is “most promising,” which candidate “seems like the best culture fit,” or which family member is “more reasonable” in a dispute runs the same risk at a smaller scale: a fluent-sounding number or ranking that reflects patterns in the model’s training data and the specific way you phrased the prompt, presented with a confidence that has no actual verification behind it. The absence of a formal audit process does not remove the bias; it just removes the only mechanism that might have caught it.
Never use an AI-generated score, rank, or grade of a person as the stated or unstated reason for a consequential decision about them - a hire, a promotion, a grade, a custody or care decision, or even an informal judgment you act on - without an accountable human process that can be examined, questioned, and overturned. If you cannot explain and defend the criteria behind a ranking to the person it affects, do not use it.
The refusal checklist
Before asking AI to rank, score, or grade people - in hiring, performance review, grading, matchmaking, or any everyday context - run through these checks:
- Would I be comfortable explaining the exact criteria to the person being ranked? If the honest answer is that the criteria are vague, opaque, or were never actually specified before the model produced a number, stop.
- Is there an accountable human who owns this decision and can be questioned about it? A ranking with no named, answerable human behind it is an unaccountable scoring system, regardless of scale.
- Am I using AI to organize evidence, or to produce the verdict itself? Organizing facts, deadlines, and documented criteria is a reasonable use. Asking the model to output “who is best” from that material is not - the verdict has to come from the accountable human, using the organized evidence.
- Would this ranking hold up to an audit? You do not need a formal audit process for everyday decisions, but asking whether one could survive scrutiny is a useful proxy for whether it should exist at all.
The full refuse-people-ranking checklist walks through this for hiring, performance reviews, school grading, and informal everyday comparisons separately, since the stakes and appropriate safeguards differ across them.
What AI can still help with
None of this means AI has no role near decisions that involve comparing people. It can help you:
- Draft consistent, specific evaluation criteria before you look at any candidate or person, reducing the chance that criteria shift to match whoever you already favor.
- Organize documented evidence (attendance, specific deliverables, dated feedback) into a clean comparison format for a human to review.
- Check whether your own written evaluation of several people uses consistent language and standards across all of them, flagging where your own wording seems to shift for one person versus another.
What it should never do is take that organized material and output a ranked list, a score, or a “winner” that a human then treats as the actual decision, rather than as one more input a human reviewed and owns.
This connects to work and dignity more broadly
Refusing to let AI rank people is a specific application of a broader principle covered in the dignity of work when machines can do more of it: worker and candidate dignity requires that consequential decisions about people retain a human who can be questioned and held to account, not just a process that is technically faster. It also connects to what not to delegate to AI - ranking or scoring a person fails the “consequence” and “reversibility” tests in that framework almost by definition, since a person who is filtered out by a hidden, unaudited score rarely gets a chance to contest the reasoning behind it.
A common misconception
The misconception is that ranking becomes safer once it is informal - a private note about which team member seems “most reliable,” a personal mental score of which friend to prioritize, a quick AI-generated comparison you never write down anywhere official. Informality removes the paper trail, not the mechanism. The Amazon case above involved a formal system with engineers actively looking for bias, and the pattern still took years to surface; an informal, unrecorded ranking has no equivalent chance of being caught, because nobody is looking for it at all. The absence of a formal process is not a safeguard - if anything, it removes the only thing that might have caught the problem.
Try it today
Look at any recent or upcoming decision where you were tempted to ask AI to rank, score, or grade people - candidates, team members, students, even a personal comparison. Run it through the four-question refusal checklist. If it fails even one, restructure the task: use AI to organize criteria and evidence, and make the actual ranking or decision yourself, in a form you could explain and defend to the person it affects.
