This is the fourth installment in our consideration of why performance appraisals fail to correct poor performance.  We’ve covered a lot of ground, though there is still much more to think about. Maybe you’ve had enough…

Maybe you’re convinced that anything that makes everyone feel miserable, destroys productivity, and costs your organisation bags of irreplaceable and valuable time makes the entire activity untenable.

And you’d be right. It does all those things. But it gets worse.

The next thing we have to talk about is the effectiveness of the appraisal itself; that is whether or not they’re reliable or valid as a means of measurement.

This, too, is far from straightforward.

It may seem like easy-peasy - that anyone can construct an appraisal. After all, you shouldn’t be doing anything more than evaluating performance that has already occurred. How hard could it be?

You may remember from an earlier article that reliability has to do with get the same results over time, and the validity is about measuring what the instrument claims to measure.

And no doubt you’re surprised that we’re devoting any time to this at all because there can’t possibly be anything more straightforward than comparing what someone said they would do against what they actually did; or evaluating the difference between a predetermined standard and the extent to which an individual met that standard, right?

If that's what you’re thinking, then read on. You have no idea of what the problems really are…

The problem with rating forms is the language that's used on them. That may sound bizarre.  What you have to understand is that an appraisal is a kind of questionnaire.

Merely by using the wrong words or using the right ones in the wrong way, you can create bias before anyone writes down anything. For example, let’s say that you have five categories.

They are: Outstanding, Excellent, Good, Below Average, and Poor.

Now think about the words that you use to describe anything of the highest worth. Are words such as outstanding or excellent a regular part of your vocabulary? Do you normally use them in your speech? Are they the words that you think other managers would use in day-to-day speech?

You see, the problem is that if these words aren’t part of your normal vocabulary, then you won’t know what you’re looking for when you come to use them in an appraisal.

To look at this another way, if Good is the word that you use to describe something of the highest worth to you, then even the most outstanding employee is likely to be rated as Good by you.

Or think of it this way: If Good is as good as it gets for you, then what does Excellent or Outstanding look like? It’s a bit like asking how full a glass is. Is it full, really full, brimming, about to drip down the side? You see, it’s impossible to make a distinction between the shades of meaning.

Or suppose that you have that bell-shaped curve in the back of your mind. In that case, you’ll make sure that about 70% of your employees are Good or Satisfactory, if that’s one of the choices. And everyone will tell you that only those who walk on water get anything better than that. When it comes to poor performance, that, too, may be a difficult term for some to use.

You see, if you’re an optimist, then you’ll probably look for the good in others. Unless someone is obviously taking the mickey out of you, you’re more likely to rate a poor performer as Below Average. That’s because Poor sounds too much like hopeless. And you don’t want to see anyone who’s trying to get the sack.

So, all appraisals that use words to describe the categories of performance have a reliability problem because no two evaluators are likely to share the same interpretation of what those words actually mean.  In an organisation that has more than one department or division, that could also pose a problem.

The one where the supervisor uses words like Outstanding and Excellent will tend to rate employees higher than one who doesn’t.  And so the same form could yield entirely different results.

The next problem is validity.

Does the performance review actually measure what it claims to measure? And in all likelihood, the answer is, “no”. It doesn’t.

Why doesn’t it? It’s because behaviour is very difficult to measure. If you don’t believe it, then talk to any Chartered Psychologist. Among the difficulties is isolating what contributed to the outstanding or the poor performance. Who was involved? It’s not like a sports team where one player gets a goal, and maybe some other one gets the credit for assisting in it. Lots of things contribute to lots of other things.

Behaviours are intertwined. And we’ve known about this for 200 years. It’s not as if someone just discovered it. And here’s another thing. It goes back to what we learned in the first article. When you use an appraisal to record poor performance, then you are making what might have been a valid evaluation into an invalid one.

Why? Because the word appraise means that you are praising someone for work that was done well. When you use it to record what was done poorly, what you’re really doing is identifying the fact that good or outstanding performance was left out. That’s not the purpose of an appraisal.

If they are used, and we don’t think that they should be, then they are only to be used to record what people did right; because that’s what the word means. In the problem of reliability, we saw that using words to describe categories causes problems.

Using numbers instead, doesn’t solve that issue.

Perhaps you thought it would. Here’s why. Let’s say that your form has a choice of seven numbers. One is the lowest and seven is the highest. Four is obviously in the middle.

There are a couple of problems with using them. One is that some managers don’t believe that anyone ever deserves the top number. Maybe it’s because they never got one or they think you have to be perfect. (Ask a university lecturer sometime about the grading scale he or she uses. Then ask what work would look like to get 90% or 95%.)

That pesky bell-shaped curve is going to be in the back of the minds of some people. They’ll rate most people at a four because few people above them will question it. No one is perfect, so the top number won’t get used, and so that leaves fives and sixes.

A few more of the former than the latter, and Bob’s your uncle. No questions from on high. And that points to the other problem with numbers.

Many people will be aware that this is how others are doing it, and so rather than penalising their people by using the full range of points, they, too, will only use four, five, and six. It makes you wonder what value there is in having numbers one to three and seven.

Of course, it’s about context. Four, five, and six look okay in a field of seven numbers, but if you used only them, then four would be the bottom and six the top. Even with only three numbers, you’d run into the same kind of bias.

So, rating inflation or deflation isn’t the answer.

The biggest problem comes with the bonuses that are often tied to the ratings. That in itself is a minefield, and we’ll save that for the last article.

In the next article, we’re going to think about how performance appraisals are conducted.

Stay tuned so that you can learn about your favorite.

If you want to become more effective at managing performance, then contact me here.