From the Ask Tom mailbag –

Question:

How can I test to see if a person has Stratum II or Stratum III capability?

Response:

If you are looking for a paper and pencil test, there is none.  There is no test with a set of answers that you shove into a computer that divines a person’s capability.  Elliott chuckled when this question was posed.  Most psychometric instruments, he observed, have, at best, a .66 correlation with reality.  Most are based on personality, or behavior, or behavior connected to temperament.  While those tests, or profiles have statistical significance for repeatability and in most cases, a stunningly accurate description of a person’s tendencies or behaviors, their evidence of predictability, a specific profile for a specific role has significance barely above flipping a coin (.5 correlation).

Elliott conjectured, if there were a paper and pencil test for capability, its likelihood to stand the same test would likely yield no more than the same .66 correlation with reality.

But your question is still valid and there is a method to satisfy the high curiosity we have about a person’s capability related to the level of work.  There is no trick, no special technique, no psychological requirement that we climb inside the head of our candidate and play amateur psychologist.

Moreover, the validity of this method reveals between .89 and .97 inter-rater reliability.

It’s all about the work.  Focus on the work.  As you define the role, its task and activities, goals and objectives, what is the level of work?  Does the role contain Stratum II level of work or Stratum III level of work?  Examine the decisions that have to be made and the problems that have to be solved.  Examine the time-span of the goals and objectives in the role.  What is the longest time-span task in the role?

The biggest mistake most companies make is underestimating the level of work required in the role.  A defect in the definition of the level of work in the role will most assuredly result in hiring the wrong person.

Examine your role description.  What are the tasks and activities?  What are the decisions that have to be made?  What are the problems that have to be solved?  What is the time-span of the longest task assignment in the role?

Based on that definition of the role, does the candidate provide evidence of effective task completion?  It’s all about the work.

When we spend the time to accurately define the work, and accurately calibrate the level of work in the role, the questions become very simple.  Does this person work as effectively as someone in the top half of the role or the bottom half of the role?  And, in that half, does this person operate as effectively as someone in the top, middle or bottom.

When you ask the team member to do a self-assessment, ask the manager and ask the manager-once-removed (MOR) about effectiveness, the inter-rater agreement approaches .97 (.89-.97).  With this practical evaluation system, why would you want to resort to other methods that might only have a .66 correlation with reality?

It’s all about the work.