Can we really measure attitude?
What about potential?
How might we evaluate authenticity?
When organizations talk about “fit”, they’re usually referring to more than a checklist of skills. While most skills can be learned, what makes or breaks success in a role goes far beyond what can be imparted in a classroom.
“Fit” is the elusive combination of having the right attitude, willingness, potential, personality, experience, and mindset that will enable a candidate to succeed in the organizational context.
Several strategies have emerged to evaluate this mix of special ingredients:
Some organizations rely on rounds of interviews to see whether a candidate “jives” with the team(s) they will be joining.
Others opt for “validated” personality, critical thinking, and other tests
Many organizations attempt to incorporate elements of their culture into a structured interview guide
And a few eschew skills altogether, putting stock in candidates’ affirmations of having desirable core beliefs and attitudes
While each method offers value, the trouble with most is that they rely a proxy measure (e.g., personality test), and/or depend heavily on candidates’ self-report.
That’s right: appraisals of the precious “secret sauce” are often a matter of a candidate’s vow to possess it, at a time when…
Candidates want to be selected, which introduces a series of conscious and unconscious biases (e.g., self-serving bias, social desirability bias)
Candidates are reporting on qualities they are not familiar with, that are not universally defined; which invites inaccuracies, usually in the direction of overestimation (Dunning-Kruger effect)
Oh dear.
Like many assessment challenges, the problem is not really in what we are doing, but in how we are doing it. In other words, the risk comes from how we are using the data gathered, not the data themselves.
A reconfiguration of how we are using our assessment methods (including placing the highest priority on identifying direct measures) can, in fact, transform the necessity of self-reporting from an obstacle to offering valuable insights.
There are three guiding principles that must drive measuring the Secret Sauce:
#1 Get Clear on the ingredients
Get really specific about what makes the organization work and why. These answers go beyond the list of attributes found on a resume to include a critical qualitative component. To get there, be prepared to push yourself beyond what you think you’re seeing, when you first observe the team in action.
For example: Are your team members generally good at collaboration? How does that show up?
Are they all proactive about collaborating? Or maybe they intuitively know when to collaborate and when not to.
Maybe they are comfortable sharing the lead; or perhaps they approach conflict as constructive.
Alternatively, maybe they all have a strong drive for innovation and collaboration is a necessary activity to get there.
#2 Prioritize Direct Measures
Once you’ve identified what great looks like, it must be translated into observable behaviours that can be measured directly.
Here’s the trick to measuring hidden attitudes, beliefs, values, and testing behaviours:
Put them in the situation.
NOT by asking them what they have done, or what they would do.
NOT by asking them what they think or believe.
Whenever possible, make them do it.
Make coaches coach, sales people sell, team members collaborate. Put candidates directly into the situations they will encounter, focusing on the necessary ingredients you identified for that Secret Sauce.
Attitudes will come out, hidden biases will be revealed, actual skills will be tested.
#3 Use Multiple Measures
The use of multiple measures is a best practice across all forms of assessment. After all, you want to make sure you’re really seeing what you think you’re seeing; that a candidate’s affirmations are legitimate.
Use a few different measures that look for the same things, in different ways:
Direct measures should confirm
Indirect measures should provide evidence to support
Outcome measures should show the results you’d expect, if the right ingredients are there
Countermeasures should rule out competing qualities
Using these three principles, the methods currently in use are still viable:
Interviews to see whether a candidate “jives” with the team(s) they will be joining can be assessed against pre-identified qualities.
“Validated” personality, critical thinking, and other tests can be selected to measure pre-identified qualities directly
Elements of their culture can be behaviouralized, and used in scenario-based and other structured interviews (with underlying scoring rubrics)
Even candidates’ affirmations of having desirable core beliefs and attitudes are meaningful when those attitudes and beliefs have been tested in real scenarios (self-assessments give insights into self-awareness!)