“”

Proven to give you confidence in your hires.

We could tell you all the reasons why to choose HiringBranch, but we'll show you instead

AI Governance in HiringBranch

HiringBranch is committed to the trustworthiness of its products. We take our clients’ need to be reassured about the impact, ethics, and accountability of our technology very seriously. The HiringBranch AI Governance Framework has been developed based on guidance from the Government of Canada’s Algorithmic Impact Assessment (AIA), the Institute of Corporate Directors (ICD) and NuEnergy.ai.

Our team is educated about the principles of AI Governance and is trained to identify the risks, ask the right questions, and seek solutions.

We keep close track of the AI-related legislation around the world and comply with standards like the GDPR,PIPEDA or PIPA. We actively think ahead and address AI Governance as an integral part of our product development pipeline.

The Five pillars of HiringBranch AI 
Governance are:

Quality - we pride ourselves on producing reliable communication assessments that predict the ability of candidates to perform in their future work environment. To that end, we constantly monitor and validate our assessments performance. Contact us for information on our assessment methodology and validation procedures.

Security - the security of our assessments is a top priority for us. Several security layers and different security measures are integrated into our system to ensure the reliability of our results. Please contact us for detailed documents on security procedures.

Privacy - all candidate data is private and confidential. Candidate data can only be accessed by authorized users using the latest security access policies. HiringBranch complies with privacy and confidentiality policies including GDPR and PIPEDA. The Privacy Policy is here.

Transparency & Accountability - HiringBranch AI is based on rules-based systems. We produce easy to understand reports and keep an audit trail of the decisions it makes. The system’s decisions are constantly inspected by human evaluators, and in the rare cases where it is necessary, changes can be made. Contact us for more information.

Fairness - our assessments treat all people fairly and do not discriminate against candidates based on their gender, race, sexual orientation, age, geographical region, nativity in the target languages, etc.

As HiringBranch does not collect data about the candidates gender, race, sexual orientation, and nativity in the target language, we could only conduct fairness analyses based on obvious features that are either provided to us by our corporate clients (like the geographical region of the candidates) or that can be assigned to voice samples of random candidates by human annotators.

Fair and unbiased assessments are of utmost importance to us. We adhere to a zero-discrimination policy when assessing candidates. Our goal is to discover what a candidate can do, regardless of who they are. As previously mentioned, gender, race, sexual orientation, age, location and native language data are not collected in assessments.

In collaboration with an undisclosed customer, we set out to prove that biases don't belong in the workplace. Using sample population data provided by the customer, we cross referenced gender and native tongue data with hired HiringBranch candidates' four-in-one assessment scores.

Nativity in the target language

No significant difference was noted for the final score of native vs. non-native speakers of English

Gender

No significant difference was noted for the final score of male vs. female candidates

Results

We are delighted (but not surprised) to see that there is no significant difference in the assessment of scores of native versus non-native speakers of English or in male versus female candidates. We don't stop there, but continue to check for bias in our assessments on an ongoing basis.

Communication Skills

Standardized Language Tests (ST) focus on language proficiency only whereas HiringBranch (HB) adds another layer on top of that in which communication skills are assessed. Thus, HB does not only check how the candidate says things but also what they say. This is done based on real scenarios that imitate the candidate’s anticipated work, which enables HB to assess the candidates that are fit for the intended environment.

HB’s test not only checks language proficiency but also assesses the candidate’s real-world communication skills, like paying attention to details, acknowledging the customer’s concern, their ability to perform under pressure, using positive language, etc. To see how the incorporation of communication skills into a test affects its validity, let’s look at the following data:

Correlation Coefficient between Communication Skills and Test Scores:

Table 1 - The Pearson correlation coefficient between communication skills and test scores

Table 1 shows that there is a high correlation between the candidate’s communication skills and their test results, not only in comparison to HB but also in comparison to ST. Thus, communication skills are highly efficient in predicting both the candidate’s linguistic ability and their chances to succeed in their future work.

To demonstrate this, let’s look at a few specific outliers: For the following question, the candidate needs to show that they understood the customer’s issue as well as to assure the customer that they are going to help them.

The following response, of a candidate named Sam, is quite good. Sam rephrases the customer’s issue in a concise way that shows that Sam understood it. Then, Sam describes how the feature should work. Moreover, the sentence “It would help us resolve your issue better” conveys Sam’s commitment to helping the customer.

Now, let’s consider the following response to the same question written by candidate Mary:

Mary repeats the word “issue” twice but does not state what the issue is. Thus, we cannot know whether Mary understood the issue or not. The response includes a general statement that shows Mary’s willingness to help, but when Mary started writing the actual solution, Mary ran out of time and did not complete the task.

In this case, then, not only did Mary not fulfill the requirements of the task, but Mary was also unable to complete it within the four-minute deadline. Thus, from the perspective of skills like working under pressure and attention to detail, two crucial skills in Mary’s future work, Mary was unable to deliver.

Sam’s communication skills are better than Mary’s. Nonetheless, Mary was ranked quite high in the Standard’s test while she failed HB’s test. The opposite is the case with Sam.

Table 2 - Mary vs. Sam’s rank in percentile (10th percentile means only10% of candidates have lower scores.)

Similar differences in the ranking are found with other scenarios, both in writing questions and in speaking questions. In many cases, the standardized language test produces a different ranking than the HB assessment.

Spoken fluency

The second issue that accounts for the imperfect correlation between HB and ST speaking tests (and by extension between the final scores of both tests) is the reliability of the spoken fluency measurements in each of the tests.

To enable a fair comparison, the results were manually scored for spoken fluency and pronunciation separately by two people. Their manual scores were then averaged to better represent the candidate’s spoken fluency.

Correlation Coefficient between Fluency and Test Scores

Table 4 - The standard test’s correlation coefficient between the tests’ speaking scores and manually scored spoken fluency

As can be seen from Table 5, HB speaking test results correlate better with the candidates’ spoken fluency.

Fluency is an important component of ‘intelligibility’, or the ability for a listener to easily understand a speaker without having to ask for clarifications. It is separate from accent or pronunciation and is a key factor in how smoothly a spoken conversation can be conducted.

The HB assessment uses a variety of metrics that together score on intelligibility. The ST assessments take a more traditional approach. This leads to differences in how the candidates are ranked in the speaking results.

Listening and reading comprehension

As was noted above, HB dedicates part of its test for reading and listening comprehension while standardized language tests do not. Let’s examine the correlations between reading/listening and the final test scores:

Table 5 - The Pearson correlation coefficient between HB’s reading and listening score and final tests results

The reading and listening comprehension scores correlate with HB’s final test scores but due to the limited weight in the total scoring formula (to reflect the more important writing and speaking skills), their effect is limited as well. As for ST’s final score, HB’s reading and listening comprehension scores correlate negatively with it.

This means that not only does ST’s test NOT account for a candidate’s reading and listening abilities but that ignoring them leads to wrong predictions.

Thus, listening and reading comprehension is an important component in a test. Naturally, this is true regardless of the empirical results, as the ability to understand written and spoken content is essential to a candidate’s ability to succeed as a customer service agent.

The absence of a reading and listening comprehension component from ST’s test results in false positives. One of them is Mary, who was ranked last in this component in HB’s test. This was reflected also in Mary’s final score on HB’s test. ST’s test, in comparison, ranked Mary quite high. In another example (Kristine), the highest-scoring candidate in reading and listening ranked high in HB’s test but ranked in the 30th percentile in ST’s test. Finally, another candidate, (Oliver) did quite poorly in reading and listening, which affected Oliver’s final ranking in HB’s test. ST’s test, on the other hand, was unable to detect Oliver’s weakness in this respect and ranked Oliver first among all candidates.

Let’s look at all the example candidates and how they we reranked.

Differences in Candidate Rankings 
90th percentile is high, 10th is low

Table 6 - Mary, Kristine, Oliver and Sam rankings in percentile

As can be seen from Table 5, HB speaking test results correlate better with the candidates’ spoken fluency.

Fluency is an important component of ‘intelligibility’, or the ability for a listener to easily understand a speaker without having to ask for clarifications. It is separate from accent or pronunciation and is a key factor in how smoothly a spoken conversation can be conducted.

The HB assessment uses a variety of metrics that together score on intelligibility. The ST assessments take a more traditional approach. This leads to differences in how the candidates are ranked in the speaking results.

Further discussion

Why would the addition of communication skills and comprehension matter?

We know from research and surveys that specific communication skills correlate directly with agent performance, such as customer satisfaction, average contact handle time, reopen rate and customer satisfaction.

Industry surveys show that the top 3 skills contact center trainers look for (in addition to spoken and written fluency)are the ability to listen carefully and acknowledge, pay attention to detail and to reassure a customer that their needs will be looked after.2

Recent consumer behaviour research conducted on ‘concrete language’ or attention to detail, as well as listening or ‘attending to’ a customer, shows that even small changes in words used by agents have positive outcomes.

The field data suggest that increasing linguistic concreteness by one standard deviation improves customer satisfaction by 9% and actual spending by at least 13%. 3

This data reinforces the appropriateness of the HB assessment in targeting the real-world skill requirements of frontline customer service employees.

Conclusions

Despite the high correlation between candidates’ scores in HB and ST tests, we have shown three major differences between the two tests:

1. While both HB and ST evaluate linguistic proficiency, HB’s test adds a layer of assessment that accounts for the candidate’s communication skills. This layer contributes dramatically to the prediction of the candidate’s chances to succeed in their prospective work.
2. HB achieves more reliable results in scoring candidates’ spoken fluency.
3. HB is the only test that analyses the candidates’ reading and listening abilities, two crucial components in the candidates’ future work.

The multiple layers of assessment and test components enable HB to easily spot false positives, thus addressing one of the biggest challenges of the recruiting industry by saving time and money.

The HB assessment also addresses false negatives and ensures that the right candidates are not passed over.

Contact centers looking to hire for customer service excellence can consider a wider range of testing than what is available from standard language tests. Specifically, assessments that evaluate communication, intelligibility and comprehension skills will produce different results thana standard language test and the results will be more accurate for performance in the contact center work environment.

About HiringBranch

HiringBranch is an AI-powered hiring assessment driving down costs and improving performance for hiring teams around the world. HiringBranch evaluates not only what candidates say, but how they say it. Quick, accurate and authentic, it’s a soft skills and language assessment in-one-go. Serving hiring teams in retail, sales, health and IT of sizes 100 to 10000+, you too can hire confidently and effortlessly.

Learn how you can start your proof of concept.

Book a demo

Ready to finally hire
with confidence?

HiringBranch is the only tool that replaces hours of screening and interviewing, intelligently.
Learn how you can start a pilot.

Book a demo
Some of our happy customers
some of our happy customers

Download our exclusive 2023 performance report

Forget moving one pre-hire metric, this is never-seen-before data on what the HiringBranch assessment is capable of AFTER THE HIRE.