A study: HiringBranch vs standardized language tests

Problem

Almost every contact center agent working today has passed a language test of one kind or another. But how well do those tests work? How often have language test results correlated with the candidates’ actual performance? Over one billion USD is spent by corporations on language testing every year. This represents a significant cost for Talent Acquisition departments. Are they getting value for the dollar?

Research shows that in offshore centers (outside North America and Europe), up to 20% of agents do not have the required language and communication skills to do their job effectively. ¹

And across the industry, customer satisfaction levels are reported to have declined in 2020. According to data compiled by the CFI group, this year’s reading is the lowest score since the report was first issued and it represents the uphill climb many companies face in satisfying increasingly demanding consumers.

"In the world of self-service, customers now reserve only the toughest problems for the contact center agents," says Sheri Petras, CEO of CFI Group. "Many simple questions are now handled by the customer on the company website, saving the complicated, frustrating questions for contact center agents. Organizations must implement the systems and processes needed to provide effective online self serve tools to customers, while also ensuring that customers easily can reach an effective and empowered agent when needed.

"One of the biggest challenges facing the recruiting industry is to both fill seats AND provide an accurate screen. Even one poor hire who lacks language skills (a ‘false positive’) and has to be either trained, shifted into another position, or let go, represents a significant cost for an organization.

Similarly, in a competitive job market, no employer wants to pass over quality candidates who may fail a standard language test for the wrong reasons.

So why are the standard industry language tests selecting the wrong candidates?

Communication Skills

Standardized Language Tests (ST) focus on language proficiency only whereas HiringBranch (HB) adds another layer on top of that in which communication skills are assessed. Thus, HB does not only check how the candidate says things but also what they say. This is done based on real scenarios that imitate the candidate’s anticipated work, which enables HB to assess the candidates that are fit for the intended environment.

HB’s test not only checks language proficiency but also assesses the candidate’s real-world communication skills, like paying attention to details, acknowledging the customer’s concern, their ability to perform under pressure, using positive language, etc. To see how the incorporation of communication skills into a test affects its validity, let’s look at the following data:

Correlation Coefficient between Communication Skills and Test Scores:

^{Table 1 - The Pearson correlation coefficient between communication skills and test scores}

Table 1 shows that there is a high correlation between the candidate’s communication skills and their test results, not only in comparison to HB but also in comparison to ST. Thus, communication skills are highly efficient in predicting both the candidate’s linguistic ability and their chances to succeed in their future work.

In this case, then, not only did Mary not fulfill the requirements of the task, but Mary was also unable to complete it within the four-minute deadline. Thus, from the perspective of skills like working under pressure and attention to detail, two crucial skills in Mary’s future work, Mary was unable to deliver.

Sam’s communication skills are better than Mary’s. Nonetheless, Mary was ranked quite high in the Standard’s test while she failed HB’s test. The opposite is the case with Sam.

^{Table 2 - Mary vs. Sam’s rank in percentile (10th percentile means only10% of candidates have lower scores.)}

Similar differences in the ranking are found with other scenarios, both in writing questions and in speaking questions. In many cases, the standardized language test produces a different ranking than the HB assessment.

Listening and reading comprehension

As was noted above, HB dedicates part of its test for reading and listening comprehension while standardized language tests do not. Let’s examine the correlations between reading/listening and the final test scores:

^{Table 5 - The Pearson correlation coefficient between HB’s reading and listening score and final tests results}

The reading and listening comprehension scores correlate with HB’s final test scores but due to the limited weight in the total scoring formula (to reflect the more important writing and speaking skills), their effect is limited as well. As for ST’s final score, HB’s reading and listening comprehension scores correlate negatively with it.

This means that not only does ST’s test NOT account for a candidate’s reading and listening abilities but that ignoring them leads to wrong predictions.

Thus, listening and reading comprehension is an important component in a test. Naturally, this is true regardless of the empirical results, as the ability to understand written and spoken content is essential to a candidate’s ability to succeed as a customer service agent.

The absence of a reading and listening comprehension component from ST’s test results in false positives. One of them is Mary, who was ranked last in this component in HB’s test. This was reflected also in Mary’s final score on HB’s test. ST’s test, in comparison, ranked Mary quite high. In another example (Kristine), the highest-scoring candidate in reading and listening ranked high in HB’s test but ranked in the 30th percentile in ST’s test. Finally, another candidate, (Oliver) did quite poorly in reading and listening, which affected Oliver’s final ranking in HB’s test. ST’s test, on the other hand, was unable to detect Oliver’s weakness in this respect and ranked Oliver first among all candidates.

Further discussion

Why would the addition of communication skills and comprehension matter?

We know from research and surveys that specific communication skills correlate directly with agent performance, such as customer satisfaction, average contact handle time, reopen rate and customer satisfaction.

Industry surveys show that the top 3 skills contact center trainers look for (in addition to spoken and written fluency)are the ability to listen carefully and acknowledge, pay attention to detail and to reassure a customer that their needs will be looked after.<sup>2

‍</sup>Recent consumer behaviour research conducted on ‘concrete language’ or attention to detail, as well as listening or ‘attending to’ a customer, shows that even small changes in words used by agents have positive outcomes.

This data reinforces the appropriateness of the HB assessment in targeting the real-world skill requirements of frontline customer service employees.