Unbiased Hiring Decisions with HiringBranch Soft Skills AI
Unbiased Hiring Decisions with HiringBranch Soft Skills AI
Background
HiringBranch does not collect data about the candidates' gender, race, sexual orientation, and nativity in the target language. Fair and unbiased assessments are of utmost importance to us. We adhere to a zero-discrimination policy when assessing candidates. “Our goal is to discover what a candidate can do, regardless of who they are,” says HiringBranch Chief Research and Development Officer Assaf Bar-Moshe, PhD
Challenge:
HiringBranch, creators of the first AI-based assessments to measure soft skills, wanted to audit their algorithms under New York City Local Law 144. This local AI legislation regulates automated employment decision tools that have the potential to perpetuate biases, exclude qualified candidates, and increase legal risks.
HiringBranch selected BABL AI Inc., a leading AI systems analysis firm, to audit its algorithms. The BABL AI team is trained to evaluate these tools for proper governance, ethical risks, alignment, disparate impacts, and compliance.
Understanding and mitigating potential biases in candidate assessments allows employers using AI hiring technology to ensure equitable opportunities. Biases can undermine both the fairness of AI hiring assessments and the validity of the results.
The study sampled 4,718 candidates who completed one of two HiringBranch assessments between May and September 2024. Following the assessment, these candidates participated in a voluntary survey with three drop-down questions regarding gender, ethnicity, and first language.
Alongside demographic data, each candidate's scores were recorded, including their overall assessment score and specific category scores, such as reading, writing, listening, and speaking. Additional categories were included for the two assessments, like Sales & Loyalty with a negotiation skills focus, or Care & Tech with a customer and quality skills focus. The candidate's geographic location was also inferred from the location of the customer administering the assessment.
Result:
In this latest analysis and audit, HiringBranch attempted to identify any potential disparities in performance across different demographic groups, such as gender and ethnicity. The results are as follows.
BABL AI Audit Results
Following the audit process, BABL AI concluded that HiringBranch satisfies New York City Law 144’s requirement for bias. HiringBranch passed all sections, including those evaluating disparate impact quantification, governance, and risk assessment.
A breakdown of the impact ratio for gender from BABL AI’s report is provided below:
As shown, male candidates have a slightly higher scoring rate than female candidates: 50.3% of males scored above the sample median compared to 49.6% of females. This small difference does not indicate any disparate impact, as reflected in the closely aligned impact ratios.
A similar pattern was observed across ethnic groups, with all demographic impact ratios exceeding the 0.8 threshold, indicating no evidence of adverse impact.
To learn more, visit our AI page.
.svg)
.svg)
.svg)
.png)
.png){kind=avatar}
.webp)
