Strategic 360s

Making feedback matter

What is a “5” ?

leave a comment »

[tweetmeme source=”anotherangle360″]

The Sunday New York Times Business section typically has a feature called “Corner Office” where a CEO is interviewed. These CEO’s seem to often be from small businesses. The one today (October 16, 2011), for example, is the CEO of a 25-person, $6 million consulting firm. The questions are often the same, having to do with the process of becoming a leader, lessons learned, hiring and promotion strategies. I have referenced these in a couple earlier blogs since they touch on topics relative to behavior change, leadership development and culture change that are relevant to aspects of 360 processes.

In today’s column, the CEO was asked about how he hires. He notes that part of the interview often includes asking the applicant to rate him/herself on a five point scale regarding various areas of knowledge and experience from their resume. If the applicant rates themselves as a “5,” he then asks if there is nothing else that they could learn about whatever it is. Of course, they say, “Oh, no, no. There is.” To which the CEO asks, “then why did you rate yourself a five?”  And he goes on to say he has never hired someone who rates themself as a five.

While this CEO’s decision not to hire these high self raters may seem arbitrary, think of the culture he is trying to create by the people he selects and the message this communicates to new hires during the process. He says that he appreciates humbleness and their understanding that they don’t know everything.  (In an earlier column, the CEO asks applicants if they are “nice,” and then explains what “nice” means in their culture, i.e., a good team player.)

(Someone told me of a senior executive who greeted applicants in the lobby and asked them whether they should take the stairs or elevator. If they said elevator, he didn’t hire them. That seems less job related, but is our “5” CEO doing a similar thing? Food for thought.)

We don’t use 360’s to hire people (though I have heard of multisource reference checks  from  past coworkers that are being positioned as 360’s), but we do have an opportunity with 360’s to create or support a culture when they involve many people. But we also know that 360’s are notorious for having severe leniency, i.e., mostly 4’s and 5’s on a five point scale.

Do all these 5’s that we collect mean that our leaders can’t get any better at what they do? Of course not. But that seems to be the message that we allow and even reward (even if not tangibly).

The vast majority of 360 processes use an Agree/Disagree (Likert) scale where “Strongly Agree” is scored as a “5” (scales that score it as a “1” seem confusing and counterintuitive to me).  The VAST majority of processes also do not include rater training that could be used to help raters (and ratees for that matter) to stop attaching any meaning to “Strongly Agree” that they wish. Which they currently do.

I have used a rating scale where “5” is defined as “role model, in the top 5-10%” that attempts to create a frame of reference for raters (and ratees) that does help reduce leniency effects.

What if we defined “5” as “can’t get any better” or something equivalent to that. I think “role model” implies that this person can be a teacher as well as example to others, and perhaps doesn’t need to get better (i.e., focus on other areas of improvement).  Some raters will undoubtedly ignore those directions, but rater training can help drill in the need for everyone to reconfigure their conceptualization of what optimal behavior is and, by the way, foster the learning and development culture that our CEO seems to be nurturing.

A recalibration of rating scales is badly needed in this field. We need to stop raters from giving all “5’s” and from ratees giving self ratings of all “5’s”.  With our current mentality on rating scales, there is really nothing to stop rating inflation. It should be no surprise that senior leaders find it to be difficult to use and support our 360 programs.

©2011 David W. Bracken


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: