HiringBranch, creators of the first AI-based assessments to measure soft skills, wanted to audit their algorithms under New York City Local Law 144. This local AI legislation regulates automated employment decision tools that have the potential to perpetuate biases, exclude qualified candidates, and increase legal risks.

HiringBranch selected BABL AI Inc., a leading AI systems analysis firm, to audit its algorithms. The BABL AI team is trained to evaluate these tools for proper governance, ethical risks, alignment, disparate impacts, and compliance.

Understanding and mitigating potential biases in candidate assessments allows employers using AI hiring technology to ensure equitable opportunities. Biases can undermine both the fairness of AI hiring assessments and the validity of the results.

The study sampled 4,718 candidates who completed one of two HiringBranch assessments between May and September 2024. Following the assessment, these candidates participated in a voluntary survey with three drop-down questions regarding gender, ethnicity, and first language.

Alongside demographic data, each candidate's scores were recorded, including their overall assessment score and specific category scores, such as reading, writing, listening, and speaking. Additional categories were included for the two assessments, like Sales & Loyalty with a negotiation skills focus, or Care & Tech with a customer and quality skills focus. The candidate's geographic location was also inferred from the location of the customer administering the assessment.