Strategic 360s

Making feedback matter

Archive for September 2013

Frequency: Too Often

with 4 comments

[tweetmeme source=”anotherangle360″]

I delivered a webinar last week on using 360 Feedback in Performance Management Processes (PMP), partially built upon a recent article that Allan Church and I published in HRPS’s People & Strategy journal on that topic (let me know if you want a copy).  In the webinar, I spent a little time talking about the challenges of creating reliable/valid measurement when we are relying on input not from the target person but from observers of his/her behavior. 

One of the many elements that come into play when asking employees to rate something (a person, an organization) is the rating scale that is being used.  Note also that the rating scale’s effectiveness is likely to be directly affected by the quality of rater training, which is often neglected beyond the most basic of written instructions. 

In the webinar, I shared a list of a dozen or so various rating scales that I have encountered over the years, all in a 5 point format.  We also see in The 3D Groups recent benchmark study of over 200 organizations that use 360 feedback that, by far, the 5 point scale and the Likert Agree/Disagree format are used more often than any other scale type.  I’m not going too far out on a limb to propose that the use of the 5 point Likert scale is a carry over from employee surveys.  While there is something to be said for familiarity, I also propose that this practice is a form of laziness in 360 designers who haven’t reflected long or hard enough to consider scales that work better when the target is a specific person and not some nebulous entity like an organization that is the focus of the engagement survey.

I have advocated for the need to have the scale to match the purpose in an earlier blog (https://dwbracken.wordpress.com/2010/09/02/put-your-scale-where-your-money-is-or-isnt/) so I will move on to another pet peeve.

In the last few weeks, I pulled together a group of colleagues to submit a proposal for a SIOP symposium on helping managers to be better coaches. This process is always fun when you see research others are conducting in an area where you have special interests (kind of like buying a box set of CDs by a favorite artist and discovering some less well known gems).  One of the research papers demonstrates once again the inadequacy of frequency scales (typically 5 point scales that ask how often the person does something, ranging from Never to Always). 

Frequency scales continue to be widely used.  The aforementioned 3D Group study indicates that 23% of the reporting organizations use this scale, third most often behind Agreement (49%) and Effectiveness (31%) (which adds up to more than 100%; it may be that companies were allowed to report on more than one 360 process in their organization).  Frankly, the 23% is shockingly high.  Very recently (https://dwbracken.wordpress.com/2013/08/11/what-is-a-coach-redux/) I cited a study that presents a newly developed questionnaire about manager behaviors in the context of performance management that uses a frequency scale, to my chagrin.

For starters, a frequency scale is conceptually flawed. People can’t do everything “Always” (or even Almost Always, as some scales use).  And because they do something “always” doesn’t mean they do it well, and, conversely, because they do it Rarely or Never doesn’t mean they are bad at it. 

As importantly, every time I have seen them scrutinized in research, frequency scales come out poorly in comparison to other formats in terms of reliability and validity.  This is the 20th anniversary of a paper Karen Paul (now at 3M) and I presented at SIOP that indicated that frequency scales severely penalize supervisors who do some things infrequently but are otherwise perceived to be effective.

In a (frankly) more rigorous piece of research by Kaiser and Kaplan (2006) (that you can access here: http://kaplandevries.com/thought-leadership/list/C44), they also demonstrate that frequency scales are, by far, less satisfactory when compared to Evaluative and “Do More/Do Less” scales.

Frequency scales are used far too frequently.  They should be used Never.

 

Kaiser, R.B., & Kaplan, R.E. (2006, April). Are all scales created equal? Response format and the validity of managerial ratings. Paper in B.C. Hayes (Chair), The Four “Rs” of 360º Feedback: Second Generation Research on Determinants of Its Effectiveness, symposium presented at the 21st Annual Conference of the Society for Industrial and Organizational Psychology, Dallas, TX.

 

©2013 David W. Bracken

Written by David Bracken

September 25, 2013 at 1:09 pm

Webinar: 360 and Performance Management (9/17/13)

leave a comment »

Please join me and Stan Cooper for a complimentary webinar.

360 and Performance Management

Tuesday, September 17th, 2013  at 12:30 PM EDT, 9:30 AM PST

Your performance management process (PMP) is probably broken. An ever-increasing number of organizations are creating a competitive advantage by integrating 360 Feedback into their PMPs, as well as other HR processes such succession planning, talent management, leadership development , coaching, and high potential programs. Learn how a well-designed and implemented 360 process can add Alignment, Consistency, Validity, Credibility, and even Legality to your PMP and other HR systems. Attendees will receive both the slides as well as a copy of my (along with Allan Church of PepsiCo) most recent publication, The “New” Performance Management Paradigm: Capitalizing on the Unrealized Potential of 360 Degree Feedback” (People & Strategy Journal, 2013)

You can register for free by using the following link: 

https://attendee.gotowebinar.com/register/2918545117149989888 

After registering, you will receive a confirmation email containing information about joining the webinar. We look forward to “seeing” you there!

Hope you can make it!

Dave

Dr. David W. Bracken – Consultant

Dr. David Bracken leads OrgVitality’s  Leadership Development and Assessment practice. In additional to employee engagement surveys, David’s work focuses on supporting Multisource, 360 Feedback and Coaching work for clients. David assists clients to make design and implementation decisions that result in sustainable change utilizing a sustainable 360 process. Internationally, David is well known as a leader in and for advancing the science of multisource (360 degree) feedback, particularly in its use to create large scale change and to improve talent management decisions.  Prolific and widely published, David is the senior editor and a contributor to “The Handbook of MultiSource Feedback” (Jossey-Bass, 2000), “Should 360 Degree Feedback Be Used Only for Developmental Purposes?” (CCL, 1997).  David also contributed a chapter on 360 Feedback to the handbook on “Organizational Surveys: Tools for Assessment and Change” (Jossey-Bass, 1996). David received his BA degree from Dartmouth College, and MS and PhD degrees in Industrial/Organizational Psychology from Georgia Tech.

Stan Cooper

Stan has over 30 years of experience as an internal and external consultant focusing on leadership development, talent management and organizational effectiveness.  His experience includes coaching executives to develop critical leadership skills; conducting individual leadership assessments as well as participating in formal group assessment centers; assisting new executives who are integrating into new organizations or positions; developing senior leadership teams; developing leadership competencies; performing organizational effectiveness assessments; and facilitating change management initiatives; Stan has also had extensive experience working with international leadership teams at both the regional and country levels, particularly in Latin America. Stan earned a B.A. degree from Queens College in Economics and has an M.B.A. from Long Island University in Organizational Management.

Written by David Bracken

September 10, 2013 at 4:50 pm

Posted in Uncategorized