Almost every mid-market ATS now ships some form of AI screening: a match score, a shortlist, a summary of the candidate against the role. Used well, it removes hours of mechanical reading. Used carelessly, it produces a defensible-looking number that quietly encodes last year's hiring pattern.
The difference is not the model. It is what the HR team decides the model is allowed to do.
What automation is genuinely good at
Extraction. Pulling structured fields out of unstructured resumes — titles, dates, tools, certifications — is a solved problem and a real time saver.
Consistency prompts. Asking every hiring manager the same structured questions, and summarising interview notes into the same template, reduces the variance that makes hiring feel arbitrary.
Surfacing the overlooked. A well-configured search can pull the internal applicant from eighteen months ago whom nobody remembered. That is inclusion work, done by software.
Drafting. Job descriptions, outreach notes and rejection messages that a human then edits.
Where it goes wrong
Proxy variables. A model told to find candidates resembling current top performers will latch onto whatever correlates: a university cluster, a former employer, a pin code, a gap in employment, even phrasing that varies by gender or first language. None of those is a skill, and none is asked for explicitly.
Gaps read as risk. Career breaks for caregiving, illness or migration are penalised by pattern-matching that has learned continuous employment as a signal of quality.
Confidence without explanation. A "78% match" carries an authority the underlying evidence rarely deserves. When a recruiter cannot say which requirement produced that number, nobody can contest it.
Feedback loops. If the model learns from who got hired, and who got hired was shaped by the model, the funnel narrows every quarter while the dashboard looks stable.
The governance checklist we implement with clients
- Write down the decision boundary. Automation may rank, summarise, extract and flag. A human makes every advance-or-reject decision. Put this in the hiring SOP, with a named owner.
- Screen on requirements, not resemblance. Configure criteria from the job description — must-have skills, certifications, licences, language, location, work authorisation. Never "similar to our best performers".
- Keep protected and proxy attributes out. Name, age, gender, marital status, photo, caste or community indicators, religion, mother tongue, and pin code should not be inputs. Institution tier and employer prestige deserve the same suspicion.
- Test for adverse impact before rollout, then quarterly. Compare pass-through rates at each stage across the groups you can lawfully measure. A large, persistent gap is a finding, not noise.
- Demand reason codes. Every score shown to a recruiter should list the evidence behind it. If your vendor cannot provide that, it cannot be used for screening.
- Give candidates a route to a human. Disclose that automated assistance is used, and publish a channel for review. Emerging regulation is converging on notice, explanation and human review — building it now is cheaper than retrofitting.
- Log decisions and versions. Keep the criteria set, the model version, the scores and the human decision for the retention period your counsel specifies. "The tool did it" is not a defence you can evidence without logs.
- Ask vendors five questions in writing. What data trained this? What has been tested for bias, how, and when? What are the inputs? Where is our data stored and for how long? Is our data used to train shared models?
What good looks like in practice
A recruiter opens a requisition of 300 applicants. Automation extracts fields, removes the 60 profiles missing a mandatory licence, and presents the remaining 240 in three tiers with reason codes against each must-have. The recruiter reads tier one and a random sample of tier three — the sample is how you catch a mis-scoring rule. Rejections at every stage are made by the recruiter and logged with a reason.
Time saved: most of a day. Auditability: intact. Candidate experience: unchanged, except faster.
The ethical position is not "avoid AI in hiring". It is that accountability cannot be automated. Whoever signs the shortlist owns the shortlist — and should be able to explain it in plain language to the person who did not make the cut.