Is this quote accurate?
In Nine Lies About Work: A Freethinking Leader’s Guide to the Real World authors, Marcus Buckingham and Ashley Goodall, share all the performance evaluations, 360-degree feedbacks and surveys you’ve participated in, are based on the belief that people can reliably rate other people.
And they can’t.
“Over the last forty years, we have tested and retested people’s ability to rate others, and the inescapable conclusion—reported in research papers such as “The Control of Bias in Ratings: A Theory of Rating” and “Trait, Rater and Level Effects in 360-Degree Performance Ratings” and “Rater Source Effects Are Alive and Well After All”—is that human beings cannot reliably rate other human beings, on anything at all.
We could confirm this by watching the ice-skating scoring at any recent Winter Olympics—how can the Chinese and the Canadian judges disagree so dramatically on the scoring of that triple toe loop?—but instead, let’s take a look at the most revealing real-world study of our rating prowess, or lack thereof.
From the data it was apparent that each rater—regardless of whether he or she was a boss, a peer, or a direct report—displayed his or her own rating pattern. Some were very lenient raters, skewing far to the right of the rating scale, while others were tough graders, skewing left. Some had natural range, using the entire scale from one to five, while others seemed to be more comfortable arranging their ratings in a tight cluster. Each person, whether he or she realized it or not, had an idiosyncratic pattern of ratings, so this powerful effect came to be called the Idiosyncratic Rater Effect."
Idiosyncratic Rater Effect
What is the Idiosyncratic Rater Effect? “When you judge others, you don’t define them, you define yourself.” Watch Marcus Buckingham’s 2:40 video:
The word “Business Acumen” is something you may be rated on in a 360-degree-survey. Each person who rates you in a 360-degree-survey will have his own idiosyncratic definition of business acumen.
As the authors note, “The same applies to other characteristics such as influencing, decision making, or even performance. Each of these is an abstract vessel into which we pour our own unique meaning: we are not well-informed, and we are, as raters, about as effective as farmers would be estimating numbers of atoms.”
The Second Hurdle – Data Insufficiency
Reliable, variable, and valid—are the signs of good data.
The problem with almost all data relating to people—including you—is that it isn’t reliable.
Goals data reports your “percent complete”; competency data compares you to abstractions; ratings data measure your performance and your potential through the eyes of unreliable witnesses: it wobbles by itself and fails to measure what it says it’s measuring.
No one knows, and anyone who claims to know simply doesn’t know good data from bad.
Reliable doesn’t mean accurate. Reliable means something doesn’t fluctuate randomly.
Reliable Data – Don’t Ask Others, Ask About Ourselves
Rather than asking whether another person has a given quality, we need to ask how we would react to that other person, if he or she did. Stop asking about others, and instead ask about ourselves.
If we ask you to rate one of your team on “growth potential,” your rating is unreliable. What is growth potential, and how can you be the judge of it?
On the other hand, if we ask if you plan to promote her today, your answer is reliable.
Rating of a team member on something called “performance” is unreliable, because your definition of performance is unique to you. Contrast this with, your response to the question, “Do you turn to this team member when you want extraordinary results?” is entirely reliable.
The authors note, “you may not be able to project into her psyche and accurately perceive her growth potential, you are able to ask yourself if you plan to promote her today, and the answer you get back will be a reliable one. (When we report on our own experiences, we have all the data we need—we have perfect data sufficiency—because we’re with ourselves a lot!)”
If you’re thoroughly confused at this point, good. You should be. The point is simple, we, nor our peers, can’t reliably rate others. We can share our experience of others, but we can not rate them because when we do, we don’t define them, we define ourselves.
Growth demands Strategic Discipline.
You need disciplined people, engaged in disciplined thought, to take disciplined action, to produce superior results, making a distinctive impact in the world.
Discipline sustains momentum, over a long period of time, to lay the foundations for lasting endurance. It’s the framework for Good to Great:
Positioning Systems is obsessively driven to elevate your teams Discipline. A winning habit starts with 3 Strategic Disciplines: Priority, Metrics and Meeting Rhythms.
Creating Execution Excellence demands creating/defining, understanding, with creativity and DISCIPLINE your Flywheel.
Meeting Rhythms achieve a disciplined focus on performance metrics to drive growth.
Positioning Systems helps your business achieve these outcomes on the Four most Important Decisions your business faces:
DECISION |
RESULT/OUTCOME |
PEOPLE |
|
STRATEGY |
|
EXECUTION |
|
CASH |
We help your business Achieve Execution Excellence.
Positioning Systems helps mid-sized ($5M - $250M) business Scale-UP. We align your business to focus on Your One Thing! Contact dwick@positioningsystems.com to Scale Up your business! Take our Four Decisions Needs Assessment to discover how your business measures against other Scaled Up companies. We’ll contact you.
High potential is the corporate equivalent of Willy Wonka’s Golden Ticket: you take it with you wherever you go, and it grants you powers, and access denied to the rest of us. The first problem we encounter is how to measure it? You can’t. People have Potential, Lie #7 next blog.