For twenty years the answer to “who is responsible for a hiring decision” was
boring and settled. The employer was. You could buy software to help you sort
applicants, and if the sorting turned out to be discriminatory, that was still
your problem, because you were the one who did the hiring.

That answer is coming apart, and it is coming apart in a way that creates jobs.

The question a federal court is now working through

A collective action is moving through the Northern District of California
against a large applicant tracking system. The claim is that its algorithmic
screening rejected applicants on the basis of age, race and disability. In
February the court authorised collective notice, which opened the case to a very
large group: applicants over forty rejected through that platform going back
several years. In June the court let significant claims survive a motion to
dismiss, including state civil rights claims and a disability claim built on a
theory of proxy discrimination.

We are not naming the vendor here. The identity of the defendant is the least
interesting part of this, and it will be a different name in eighteen months.

The interesting part is the theory. The argument is that a screening tool
acting on an employer’s behalf can be treated as the employer’s agent.
If that holds, the chain has no insulated link in it. The employer cannot point
at the software. The vendor cannot point at the employer for having made the
final call. Both are in it.

Why this is a hiring story and not a legal story

Here is the pattern, and it has now run three times without variation.

Sarbanes-Oxley, 2002. Internal controls had existed as an idea for
decades. They became a department, with a budget and a head count, the year
executives started facing personal criminal exposure for signing off on the
numbers. The modern compliance officer is a creature of that statute.

GDPR, 2018. The Data Protection Officer went from a job almost nobody
held to a role with a statutory definition, a certification market and a salary
band, in roughly eighteen months. Not because data protection got more
interesting. Because non-compliance acquired a number attached to global
revenue.

Algorithmic hiring, now. Same shape. Companies have been running
automated screening for years with nobody in the building who could answer for
it. That was survivable while liability was theoretical. It stopped being
theoretical when claims of this kind started surviving motions to dismiss.

Nobody staffs a governance function because it is prudent. They staff it after
the first case gets past the pleadings.

The next defendant will not be an ATS

Here is a prediction, and we will put a date on it so it can be held against us.

The current case involves an applicant tracking system, because that is where
the screening happens today. It will not stop there. The same agency theory,
applied one step earlier in the funnel, lands on job boards.

Think about what a large board actually does now. It decides which postings a
given candidate is shown. It scores and ranks applicants before a recruiter ever
opens the list. It recommends candidates to employers and jobs to candidates,
using models nobody outside the company has inspected. If an algorithm deciding
who gets screened out can be an employer’s agent, it is a short walk to
arguing that an algorithm deciding who ever sees the posting is doing the
same work, earlier and more invisibly.

A candidate rejected by a screening tool at least knows they applied. A
candidate who was never shown the job has nothing to point at.

That is not a coy answer, it is an instruction. The policy statements are
public. Read what a board says it does with your profile, what it says it does
with an employer’s posting, and which of those two it optimises the match for.
Read the section on recommendations and ask who the recommendation is for. You do
not need inside information to work out where this lands. You need to read the
documents the companies wrote about themselves.

We run job boards. We are aware of what we are saying. It is also why every
posting on this site is reviewed by a human being before it appears, and why we
do not rank, score or algorithmically sort the people who apply through it. That
is not a virtue, it is a design decision, and it was made partly because this was
always going to arrive.

What a general counsel actually needs on Monday morning

If you are the person who read about this over the weekend and is now trying to
work out what to do, the honest answer is that you need someone whose actual job
is to answer four questions in writing:

• What tools touch our hiring decisions, and at what point in the funnel?
  Most organisations cannot answer this. Screening is often bought by talent
  acquisition, configured by an integrator, and never inventoried anywhere legal
  can see it.
• What did the vendor tell us about bias testing, and can we produce it?
  A contractual representation you cannot locate is not a defence.
• What does our own outcome data show? Adverse impact analysis of your
  actual funnel, by protected class, on your real applicant flow. Not the
  vendor’s published audit of their model.
• Who signed off, and what were they shown? This is the board-level
  version, and it is the one that hurts, because in most companies the answer is
  that nobody was ever formally asked.

Those four questions are a job description. They are not a project, they are not
a quarter of somebody’s time, and they do not belong to the vendor.

The roles this creates

We index these postings, so we watch the titles change in something close to real
time. What is appearing now, and what will appear more of:

• AI Governance Lead and Responsible AI Manager, sitting inside
  legal or risk rather than inside the data science function, which is the
  structural signal that matters.
• Algorithmic Auditor and Model Risk roles moving out of financial
  services, where model risk management has been a mature discipline since the
  banking regulators made it one, and into technology, retail and healthcare.
• AI Vendor Risk specialists. Third party risk management has existed for
  years. What is new is that the third party’s model is now inside your
  employment decisions.
  See what an AI vendor assessment involves.
• Employment counsel with technical literacy. Rare, expensive, and about to
  get more so.

If you are a compliance, privacy or audit professional reading this and wondering
whether you are qualified: you probably are, and you are closer than the machine
learning engineers are. The scarce skill here is not building models. It is
knowing how to evidence a control, write a finding somebody will act on, and sit
in front of a regulator. That is a GRC skill set, and it does not retrain in a
weekend.

What to do if you run the function

Three things, in this order.

Inventory first. You cannot govern what you have not listed. Every tool
that scores, ranks, filters or recommends a human being, and where it sits in the
process. Start there and start this week, because the inventory is also the
discovery response.
See how an AI risk register is built.

Then name an owner. Not a committee, not a working group. A person with a
title, who is accountable, and whose name appears on the sign-off. The reason
this matters is narrow and practical: an organisation with a named owner produces
documentation. One without a named owner produces meeting invitations.
See how an AI governance committee is structured.

Then test your own outcomes. Your applicant flow, not the vendor’s model
card. If there is an adverse impact in your funnel, you would rather find it
yourself than have it found during discovery, and the difference between those
two is roughly the difference between a remediation plan and a settlement.

The honest limit of all this

The case is not decided. Surviving a motion to dismiss means a court found the
claims plausible enough to proceed, not that they are correct. The agency theory
could be narrowed on appeal, and it may look different in two years.

That does not change the hiring picture, and this is the part people get wrong.
Companies do not wait for a final judgment before staffing a risk. They staff it
when their outside counsel tells them the theory is live. That already happened.
The reqs follow the memo, not the verdict.

Which is the short version of why this field exists at all. Not because
artificial intelligence is interesting. Because somebody is going to be sued, and
somebody has to be hired to prevent it.

Frequently asked questions

Can a software vendor be held liable for hiring discrimination, or only the employer?

That is the question being litigated. The theory being tested is that a screening tool acting on an employer's behalf can be treated as the employer's agent, which would make the vendor directly liable under anti-discrimination law rather than merely a supplier. Claims on that theory have survived a motion to dismiss in federal court. It is not settled law yet, but employers should not assume the vendor absorbs the risk, and vendors should not assume the employer does.

Does using an AI screening tool protect an employer from a discrimination claim?

No, and it never did. The employer remains responsible for the outcome of its own hiring process regardless of what software produced the ranking. Automation is not a defence. What is changing is that the vendor may now be exposed as well, which means there are two defendants rather than one, not that the employer has been let off.

What is proxy discrimination in the context of hiring algorithms?

It is discrimination that happens through a variable that correlates with a protected characteristic without naming it. A model that never sees age but weights graduation year, or continuous employment history, or familiarity with a superseded technology, can produce an age-based outcome without an age field anywhere in the data. It is difficult to detect by inspecting the inputs, which is why outcome testing matters more than model documentation.

What should a company do first if it uses automated hiring tools?

Build the inventory. List every tool that scores, ranks, filters or recommends a candidate, and record where each one sits in the funnel. Most organisations cannot produce that list today, because screening tools are frequently bought by talent acquisition, configured by an integrator, and never registered anywhere legal can see them. The inventory is also, conveniently, the first thing you would be asked to produce in discovery.

Is a vendor's published bias audit enough?

It is useful and it is not sufficient. A vendor audit describes how the model behaved on the vendor's test population. It says nothing about how it behaved on your applicant flow, in your configuration, with your thresholds, for your roles. Adverse impact is a property of your actual outcomes. You have to measure your own.

Which job titles are being created by this?

AI Governance Lead, Responsible AI Manager, Algorithmic Auditor, AI Vendor Risk specialist, and model risk roles moving out of financial services into other sectors. The structural signal worth watching is where the role reports. When it sits under legal or risk rather than under data science, the organisation has understood that this is an accountability function rather than a technical one.

I work in compliance or privacy. Am I qualified for these roles?

Very likely, and you are closer to qualified than most machine learning engineers are. The scarce skill is not building models. It is knowing how to evidence a control, write a finding that somebody acts on, run an audit trail, and sit across from a regulator. That is a GRC skill set and it takes years to build. The AI-specific knowledge on top of it is a matter of months, not years.

Do we need a committee or a named owner?

A named owner, first. Committees are useful for review and escalation, and they are not accountable, because a committee cannot be held to anything. An organisation with a named owner produces documentation. An organisation without one produces meeting invitations and, eventually, a gap in the record that somebody reads aloud in a deposition.

The case is not decided. Why should we act now?

Because hiring follows legal advice, not verdicts. Companies staff a risk when outside counsel tells them the theory is live, and that has already happened. Waiting for a final judgment means beginning your search in a market where every comparable employer started two years earlier, competing for the same small population of people who can do this work.

How is algorithmic hiring risk different from ordinary third party risk?

Traditional third party risk asks whether a supplier will fail, leak your data, or breach a contract. This asks something different: the third party's model is making a decision about a person, inside a process you are legally answerable for, in a domain with its own body of anti-discrimination law. The vendor is not adjacent to the employment decision. For the duration of the screen, it is making it.

Could job boards face the same liability as applicant tracking systems?

We think so, and we run job boards. A large board decides which postings a candidate is shown, scores applicants before a recruiter sees them, and recommends people to employers using models nobody outside the company has inspected. If an algorithm that screens someone out can be an employer's agent, an algorithm that decides whether they ever saw the posting is doing similar work one step earlier. It is also harder to detect, because a candidate who was never shown a job has nothing to point at.

How would I work out which platforms are most exposed?

Read their own published policies. Look at what a platform says it does with a candidate profile, what it says it does with an employer's posting, and which of the two the matching is optimised for. Read the section on recommendations and ask who the recommendation is actually serving. None of this requires inside information. The companies have described their own systems in public documents, and those descriptions are the evidence.