Audit an automated hiring system

---
name: audit-an-automated-hiring-system
category: data
description: Audit automated hiring tools for job relevance, validity, subgroup harm, governance, and recourse. Use when screening, ranking, matching, assessment, interview, or monitoring automation influences employment opportunity.
---

# audit-an-automated-hiring-system

## When to use

- Use before procurement, deployment, material change, expansion, renewal, or continued use of consequential hiring automation.
- Use an ordinary interview or scorecard review when no automated output affects access or evaluation.

## Preconditions

- Confirm accountable employer, jurisdictions, worker and candidate populations, employment and privacy counsel, equal-opportunity and accessibility owners, data authority, system access, vendor cooperation, process owner, appeal capacity, and power to pause use.

## Procedure

1. Build a decision inventory across sourcing, advertising, matching, application, screening, assessment, interview, ranking, recommendation, offer, monitoring, and retention.
2. Create a system and decision register with owner, vendor, version, input, feature, output, threshold, user, population, decision consequence, integration, data flow, and fallback.
3. Establish the actual job outcome, role analysis, essential functions, baseline, alternatives, and prohibited or irrelevant factors.
4. Test job-related validity for the exact population, role, use, version, threshold, and decision rather than accepting vendor-wide accuracy claims.
5. Audit data provenance, labels, missingness, proxies, accessibility, accommodations, language, device effects, drift, leakage, and feedback loops.
6. Perform disparate impact analysis across legally and ethically relevant groups, intersections, stages, thresholds, false outcomes, missing data, and practical access.
7. Examine automation bias, reviewer overrides, consistency, workload, notice, explanation, documentation, and whether human review changes outcomes meaningfully.
8. Provide human appeal with accessible notice, timely review, evidence correction, alternative assessment, qualified authority, and protection from retaliation.
9. Compare removal, redesign, narrower use, threshold change, added evidence, accommodation, human process, and nonautomated alternatives.
10. Revalidate after change and monitor selection, errors, complaints, appeals, drift, subgroup outcomes, and work performance with bounded retention.

## Failure plan

- Pause or bypass use when validity, population fit, accessibility, data authority, material subgroup harm, decision ownership, or appeal cannot be established.
- Do not infer sensitive traits, fill missing demographic data, or treat a lack of complaints as proof of fairness.
- Escalate employment, discrimination, disability, labor, privacy, notice, record, and vendor-liability conclusions to qualified owners.

## Done

- A system and decision register reconciles every automated component, input, output, threshold, owner, population, consequence, and fallback
- A validity and subgroup report documents job-related validity, disparate impact analysis, accessibility, uncertainty, error, and reviewer behavior
- Human appeal is tested for notice, access, correction, alternative assessment, authority, timing, and outcome traceability
- A remediation and monitoring record proves approved use, paused scope, corrective action, revalidation, drift checks, complaints, appeals, and decision ownership