Hacking the Interview

7 Recommendations from Industrial Psychology

This is a guest post by Daniel Maurath, a former Associate Data Scientist at Bright.com and recent graduate of San Francisco State University, earning a Masters of Science degree in Industrial Organizational Psychology.

In this article Daniel summarises some of the key insights from the study of structured interviews, and proposes a much more data driven approach to interviewing candidates than is normally found in recruitment processes.

Most organizations fail their job applicants. 

They want to hire bright and exceptional individuals, but they have virtually no idea how to identify them. Instead, applicants are put through an irrational and terrible gauntlet of untrained individuals asking unproductive questions. 

As a data collection and assessment method, these unstructured interviews are unreliable, invalid, and susceptible to a myriad of human biases. 

There’s a better, objective and data-driven way: the structured interview. 

Think of a structured interview as an oral standardized test. As with academic tests, all applicants are asked the same questions, and their answers are summarized using statistics to produce a score. 

The principal difference is that a structured interview is testing for performance in a specific job rather than a general aptitude for success in college or graduate school. 

This standardization is important because it translates into several advantages. A structured interview is better than an unstructured interview for three key reasons.

  • It links the candidate’s interview performance directly to their future job performance 
  • It limits extraneous and distracting information  
  • It allows for direct and objective comparison of candidates. 

Structured interviews were first developed about 20 years ago. The most popular framework was published in a seminal 1997 paper by Michael Campion, David Palmer, and James Campion entitled ‘A Review of Structure in the Selection Interview’. 

Campion et al. defined 15 total components of a structured interview, which can be grouped into two categories: content and evaluation

Content components focus on the questions and procedures that comprise the interview, while evaluation focuses on the methods used to judge and select candidates. 

Now, you may be thinking 15 components are overwhelming! That’s OK. An update to the original paper found that of the 15, there were only six that were most often used. I included a seventh: statistical prediction.

Below I’ve outlined each component and the best practice identified by Campion et al. With the most high value techniques I’ve suggested some ways to hack the process and get value faster - I’d love to get your feedback on these ideas, you can reach me on Twitter - I’m @dmaurath

#1. Base the interview on a job analysis 

By the book technique

Job analysis is a rigorously developed matching of all the Knowledge, Skills, Abilities, and Other Characteristics (KSAOs) to all the tasks that comprise a job. This is usually conducted with interviews of Subject Matter Experts, such as current incumbents in the position or the hiring manager for a new position. When you base an interview on a job analysis, you ask questions that directly relate to the needed KSAOs for the job.

The hack! 

Base the interview questions on the job requirements and the group norms for the hiring team. Take some time to think about what good performance in the job looks like in terms of deliverables and goals achieved. Reverse engineer what KSAOs are required to achieve successful performance. Consider also what company values are important to you and how important it is for a candidate to be aligned to them, and how the candidate could compliment or supplement their future team. 

#2. Ask better questions 

By the book: 

Questions that are situational, about past behavior, about work experience or about specific job knowledge have been demonstrated by the research as the best ones to assess future on-the-job behavior. Puzzle or brain teaser questions, for example, have been found to be a waste of time.

The hack!

Describe the candidate’s first project they will have to do if they are hired, and have them walk you through their approach including the tools they would use. Next, ask the candidate how they would handle situations that could realistically arise, such as a difficult team member, a deadline that was suddenly pushed up, or feedback from their supervisor. 

#3. Ask all candidates the same set of questions 

By the book: 

Think of this as following and not deviating from a script for each candidate. Even minor changes in wording can influence a candidate’s answers, and when you are trying to predict human behavior, you need to control as many influences as you can in order to reduce as much error in your prediction as possible.

The hack! 

See “By the book” for this one! It’s more efficient because it requires less questions, and is the only way to do it. Without standardized questions, it’s impossible to effectively compare candidates.

#4. Rate each answer on a quantifiable scale or multiple scales 

By the book: 

You can use a simple 5-point scale with Strongly Disagree on one end and Strongly Agree on the other to rate an applicant’s answer to a question, which enable the interview to be scored and quantified. 

The hack! 

Set up a simple spreadsheet or Google form for interviewers to rate each candidates’ responses to each question on a 5-point scale. 

#5. Use behavioral anchors to aid raters. 

By the book: 

A behavioral anchor is a generalizable example of behaviors that demonstrate the proficiency at that level. For example, a question on team building skills rated on a 5-point scale could have the following anchor for 5, “Demonstrates that the candidate shares management responsibilities with a team, works hard to achieve consensus on all major decisions, calls attention to team achievements, etc.” These behavioral anchors are developed from the job analysis process discussed in the first point.

The hack! 

Expand the spreadsheet by adding a column containing general examples of answers for each of the five points on the scale. Or use a survey tool to tie the anchors directly to the numbers.  

#6. Train interviewers

By the book: 

Interviewers need to be trained to follow and maintain the structure of the interview. When interviewers are standardized as much as possible, it reduces the error associated with your interview process. 

The hack! 

Record an ideal short mock structured interview and show it to the interviewer. Then ask them to record their own, and discuss areas for improvement. The initial upfront effort in creating the video training will pay off because the training can be given on demand to interviewers and reused as necessary. 

#7. Use statistical prediction 

By the book: 

You have numeric ratings, now use them. Don’t use your gut. Humans are biased to only remember when gut feelings work, curiously forgetting the many times they did not. The most common method is to weight all ratings the same and then take a sum or mean of the question ratings for each candidate, which can be done with ease in Excel. No advanced technical skill required. 

The hack! 

No hack needed here. Just sum or take the average of the scores.  

Conclusion: Structuring your interviews will help you build a better team

Structured interviews save time, money, and headaches because they enable you to better identify productive hires. By using data and objective methods instead of biased subjective judgment, judgment errors are reduced and accuracy for predicting future performance and turnover is increased. 

After all, an interview is a small sample of an individuals behavior, so you must take all necessary steps to collect the most accurate sample you can. 

So why doesn’t everyone use structured interviews if they’re so great? It’s simple. People are lazy! Structured interviews require more work to develop and conduct. 

Human Resources and the hiring team need to collaborate to conduct a job analysis, choose a set of questions, conduct the interviews and then choose the candidates. This takes time. 

However, it’s time well spent. Once a job analysis is complete, it only needs to be updated every few years for the life of the job. Do it once and benefit for years to come. 

Another reason that structured interviews are not as widely used as they should be is that candidates dislike structured interviews, and have been found to rate an organization, job and interviewer lower due to a high-amount of structure. 

It’s therefore important then to train interviewers well, to explain the purpose of the structured process clearly to candidates and allow candidates to ask questions and speak freely at the end after the formal interview has been completed. 

Ultimately, interviewing is a difficult job. And like any difficult job it takes training and practice to get good at it! But by being prepared, using data and thinking objectively you will make better decisions. 

You can always grab a coffee with the candidate afterwards, just as you do with your colleagues - but while you’re on the job you should take it seriously, and that means using structured techniques.

Give Daniel your feedback on this post on twitter - he’s @dmurath

If you’re looking for smart people to invite to your structured interviews, head over to Hire my Friend - there’s thousands of developers, designers and digital marketers waiting to hear from you