The role of psychometric assessments in the hiring process has increased significantly in the past decade, coinciding with the data-driven movement that has overtaken human resources within organizations. In last month’s article, I discussed the role of psychometric assessments within current talent practices. In this article, I’ll move into the specific details about choosing the right assessment for your organization.
When it comes to hiring, the main role of assessments is to provide objective information about a candidate that wouldn’t otherwise be uncovered in a typical interview process. Therefore, organizations should look to utilizing an assessment with the highest level of validity and consistency in order to obtain results with the lowest margin for error. Psychometric assessments have the power to obtain accurate data on a candidate’s personality, however, the structure and itemization of the assessment is what determines how susceptible it will be to inaccuracy. Some of the factors that render this susceptibility are as follows:
- Length of assessment
- Structure of scale
- Item relevance and complexity
No psychometric assessment is 100% free of error; however, the control for error can be maximized depending on the factors stated above.
Length of Assessment: This factor is simply rationalized through the amount of data collected. If an assessment takes 10 minutes to complete and consists of 20 questions, chances are, the candidate’s mood and interpretation of each item has an effect on how they answered each question. Therefore, the results of the assessment are based on one or two data points per criterion that could have been susceptible to external factors, and aren’t capturing enough data that is valid and true. Organizations that want the most specific, valid data on their people should use length as a factor when choosing an assessment to use, since asking variations of the same question over and over again will contribute to correcting for any errors.
Structure & Design of Scale: There are a few properties related to the design of the scale that should be considered when evaluating what assessment to use. The most common design used to measure items is a Likert scale, in which candidates rate their individual level of similarity or relevance to a particular trait or verbal anchor. For example, the verbal anchor may be “Rate how detail-oriented you are”, in which the supplementary scale consists of 7 intervals from 1-7, 1 being “not at all” and 7 being “very”. The main problem with these types of scales is that, depending on the role in question, a candidate can easily skew their answers based on how desirable the trait is to the employer. For example, a candidate will likely state that they are extremely detail-oriented if they are applying for a Director of Finance role.
That being said, the scale design that has become increasingly more popular within pre-employment psychometric testing particularly is a forced-choice scale. Where Likert scales utilize a verbal anchor specific to a certain criterion, the anchor used in a forced-choice scale is simply “Choose the statement that is the most like you and the statement that is the least like you”. Following the anchor, four distinct statements related to four different criterion (with no obvious positive or negative valence) are presented, in which the candidate is essentially forced to choose which statements are most like them vs. least like them. This scale design is scientifically the least susceptible to candidate faking because it is unclear which statement will be the most desirable to the prospective employer. Here is an example of a forced choice question:
“Choose the statement that is most like you and the statement that is least like you:
- I am a social person
- I always thoroughly check my work
- I am focused on achieving my goals
- I listen to other people speak intently”
Now, the candidate taking this assessment might look at this question and think “All of these are me. How can I choose which one is least like me?!”, which, ideally, forces the candidate to choose the one that truly represents who they are. When an assessment imposes this type of question over and over again, the likelihood of correcting for error and obtaining valid results increases.
Item Relevance & Complexity: The questions used in the assessment should be written clearly and should each be specific to one criterion that is being measured. Ambiguity, specialist jargon, and leading questions should be avoided in order to obtain results that are not skewed by a participant’s misunderstanding of the question. Negatively worded questions can also lead to confusion, however, this take is controversial. Many researchers like to throw in negatively worded questions to prevent participants from falling into an absent-minded response pattern (i.e., not reading the question and choosing “C” every single time). However, these questions tend to create even more confusion and misunderstanding, so the costs end up outweighing the benefits.
It is a common perception that there are certain assessments best suited for certain roles. While there are particular assessments designed and geared towards evaluating a candidate’s potential in a particular type of job (i.e., Sales, Operations, etc) which can be very helpful, the employer is left with data on a candidate that only revolves around the scientific factors related to that role. Consequently, this can render the employer with tunnel vision regarding that candidate’s potential. I recommend implementing an assessment that measures a wider variety of traits in order to get a more holistic picture of that candidate’s personality.
When choosing an assessment for pre-hiring, obtaining the most valid and true data should be a hiring manager’s highest priority. Many employers often shy away from utilizing lengthy assessments in the hiring process for fear of deterring candidates from applying. My take on that is, if you are a company that cares about bringing in talent that aligns with your culture, then you should be striving towards collecting the most valid data in order to avoid the costly mistake of bringing on someone who doesn’t fit. Throwing in an assessment that is susceptible to error just for the sake of collecting some data and checking a box is not useful, and may cause you to be making decisions about a person based on inaccurate information. Finally, if a 60-minute psychometric assessment is enough to deter a candidate from applying to a role, then the organization should either feel confident that they are weeding out the talent that isn’t really interested in the role in the first place, or they should seek to re-vamp their employee value proposition.
Some examples of assessments that meet all of the criteria stated above are the Caliper Assessment (calipercorp.com) and the Hogan Personality Inventory (hoganassessments.com).
About the author: Erica Naccarato is an Executive Search Consultant and Industrial Organizational Psychologist at Fulcrum Search Science Inc., a Toronto-based executive search firm solving Mission-Critical hiring challenges throughout North America. She specializes in the use of psychometric data and assessments to assist with building and developing teams. She can be reached at (416)-847-4990 x 107 or (M) (416)-389-0583 or by email at erica.naccarato@fulcrumsearchscience.com.
