Strategic 360s

360s for more than just development

I Have a Dream

with 3 comments

In the next couple weeks, I have a workshop to do on “Creating a Coaching Climate” for the Greater Atlanta Chapter of ASTD, and then a conversation hour at SIOP on “Strategic 360 Feedback” that I wrote about last week (http://dwbracken.wordpress.com/2014/04/18/holes-in-the-wall-a-siop-preview/).

Clearly I am still trying to influence people about some things that I feel strongly about. So I was thankful that my wife brought to my attention a TED talk by Simon Sinek that has over 16 million views (http://www.ted.com/talks/simon_sinek_how_great_leaders_inspire_action­­) that she thought I would find interesting because it was positioned to be about leadership. And it is. But, as importantly, it is about influencing others (which is part of leadership). It is also about sales, and he uses the word “buy” often, which can be taken both literally (sales) and as a euphemism (“buy into”).

In this TED talk, Mr. Sinek proposes that the best way to influence others is not to talk about “what you do”, or “how you do it”, but to express “why” you do it, i.e., the passion behind the subject. He reminded us that Martin Luther King didn’t say, “I have a plan,” (though he undoubtedly did). Instead, he said “I have a dream,” and went on to describe what that dream looked like. There are many other examples, such as John F. Kennedy’s dream of putting a man on the moon that was not only realized but created countless scientific innovations that have become part of our daily lives.

So part of my dream is captured in the tagline from The Handbook of Multisource Feedback that I also referenced in my last blog: Large scale change occurs when a lot of people change just a little. One of the great things about being an I/O psychologist is we have the opportunity and challenge to touch “a lot of people” with our work. One way we do that is the ways we help organizations make better decisions about people, such as in the decisions about who to hire, fire, promote and develop, and by constantly striving to improve the accuracy of those decisions for the benefit of the organization and the individual. And you may (or may not) know that I am a proponent of use 360 Assessments to help improve the quality (i.e., reliability and validity) of decisions we have to make about many employees (e.g., development plans, training, promotions, staffing, compensation, succession plans, high potential identification).

We can also touch “a lot of people” with processes that affect employees once they are on board. The versions of 360 processes that The Handbook primarily focuses on are those that do touch “a lot of people” to create change one person at a time (but all at once). What is missing in that phrase is the critical notion of creating sustainable change. My criticism of many 360 processes is that they do not burden themselves with worrying about what it takes to create sustainable behavior change, seemingly feeling that the simple act of creating awareness of a need to change (a gap between observed and desired behaviors) will somehow make people magically change. Some do, but not often enough nor are they the people who need it most.

Sustained behavior change can also be thought of as a habit. Part of my dream is to have behavior change (which is a choice) become a reflex, a natural reaction.

My son-in-law, who has two daughters (with my daughter, of course), put a post on Facebook last week that asked, “Am I the only one who puts the toilet seat down in my hotel room?” I, and a few others, responded “No, I do it too”, and I (also having two daughters) have been known to use this very behavior as an example of a voluntarily adopted behavior that becomes a habit, even if the behavior has no obvious benefit to the actor. The “benefit” to the actor is that he/she (“he” in this case) is part of an organization (the household, family) and by being considerate of others, can expect to in turn maintain the cohesiveness of the organization.

Last year, right after Nelson Mandela’s death, I listened in on an interview of a BBC journalist who had made a career out of following the life of Mandela. He shared that he was so moved by this man that he gave his son the middle name “Nelson,” and the interviewer asked what he hoped to affect his son’s life by doing so (which is an interesting question). The journalist, though, had an immediate answer: He hoped that his son would show kindness to others as a reflex (i.e., ingrained habit, my words).

The notion of “kindness” is one I am hearing more often in organizations, sometimes in the context of the desire to be empathetic without sacrificing the need to make tough decisions about people. Then I saw this article (http://goo.gl/iz5Qdj) about “compassion” that seems to capture the idea of kindness and shared values. Defined as “when colleagues who are together day in and day out, ask and care about each other’s work and even non-work issues,” some cited research indicates that to the extent that there’s a greater culture of companionate love, that culture is associated with greater satisfaction, commitment and accountability.”

This piece on compassion then goes on to say, “Management can do something about this, They should be thinking about the emotional culture. It starts with how they are treating their own employees when they see them. Are they showing these kinds of emotions? And it informs what kind of policies they put into place. This is something that can definitely be very purposeful — not just something that rises organically.”

You can create a culture by the behaviors that leaders exhibit, whether it’s a culture of compassion, kindness, quality, customer service, fear, anger, fun, feedback, and so on. The point is that these cultures can be defined by behaviors. And a behavior is a choice, i.e., whether to do it or not. And the behavior can become a habit or reflex. We shouldn’t buy the excuse, “Well, that isn’t who I am.” I/we don’t care. The type of person/leader you are is determined by what you do, not what you think or think you think.

And when employees (at all levels) report that they want to be respected, valued, developed, and have trust in their leaders (see this report from APA: (http://www.apaexcellence.org/resources/goodcompany/newsletter/article/530), organizations should listen and act, i.e., define the desired behaviors and hold leaders accountable. Someday those behaviors will become habits/reflexes.

 

So, what is my dream? In this context, it includes things like this:

  • That more organizations will acknowledge the intuitive and research-based advantages to treating their employees with respect and kindness, and engendering trust along the way, and then do something to create sustainable change.
  • Focus on the potential benefits of processes like 360’s that can potentially improve our decisions, not focus on the challenges in doing so
  • Speaking of decisions, that we can use tools like 360’s to identify leaders early in their career who are poised to do damage via inappropriate behaviors, and get rid of them (or at least not promote them)
  • Admit that human nature is such that behavior change requires not only awareness but accountability for sustainable change to occur
  • Acknowledge that sustainable culture change requires integration into HR processes to create ongoing alignment, accountability, and measurement
  • That kindness, compassion and respect become habits for all of us.

 

That’s enough dreaming for now.

©2014 David W. Bracken

Written by David Bracken

April 23, 2014 at 6:16 pm

Holes in the Wall: A SIOP Preview

with one comment

I will be leading a Conversation Hour at the upcoming SIOP Annual Conference, surprisingly titled, “Strategic 360 Feedback.” I would love to hear from any of you as to what you would like to talk about in your use of 360’s for more than “just” leadership development, whether you are going to be there or just wish you were.

One topic I do want to address is the use of 360’s in creating large scale change in organizations (climate change??), harkening back to the tagline at the beginning of The Handbook of Multisource Feedback: “Large scale change occurs when a lot of people change just a little.”

I am thinking about using a metaphor building off the observation (criticism?) of “when you have a hammer, everything looks like a nail,” here applied to 360’s. Of course, I look at things a little differently, as in missed opportunities. To extend the metaphor, I see many (most) organizations frustrated with the inability to sustain processes such as performance management systems or other culture change initiatives. So let’s say the “initiative” is like a picture we are trying to hang on the wall. So we have to get a hook nailed into the wall. I believe they are trying to push in nails with just their thumbs, and, of course, the picture might hang on the wall for minutes or a few hours, but then crashes with a large thump and lots of broken glass. And leaves a hole in the wall, maybe adding to all the holes already there from other unsuccessful attempts to hang that picture or other pictures.

To wrap up the metaphor, let’s survey the scene (so to speak). A broken picture with lots of accompanying noise that everyone can see and refer to, including the cost of repair if they are going to try to hang it again. And of course the holes in the wall everyone will point at as evidence of all the failed attempts to hang pictures in the past. So where is the hammer (i.e., 360 feedback processes)?

Well, let’s see. We had a hammer but lost it. And someone hit their thumb with the last one. The last time we used it, it was too small (or big, take your pick). A new hammer is expensive. The person who had the hammer left the company and took it with them (and we really didn’t like that hammer anyways). The last time we used it, we used the wrong end (must have been a manager). Maybe a shoe would work next time?

Like any tool, a hammer (aka 360) can be misused and even dangerous. Allan Church and I produced an article that tries to demonstrate how the 360 “hammer” can be used to improve performance management in the right hands. http://www.orgvitality.com/articles/HRPSBrackenChurch OV.pdf

And maybe hang on the wall for a long time.

Please let me know if you have any observations about how your “hammer” hasn’t worked and/or how this metaphor works or doesn’t work for you.

See you in Hawaii??

P.S.  The 3rd meeting of the Strategic 360 Forum will convene in Chicago on September 16.  Let me know if you have an interest.

©2014 David W. Bracken

A Matter of Trust

with one comment

“Apologizing does not always mean that you are wrong and the other person is right. It just means you value the relationship more than your ego.”

I saw that anonymous quote on LinkedIn recently and it drew me back to a small note in Traning & Development magazine dated February 24 on this topic. (http://goo.gl/8X6yRe) The text follows:

A recent survey of 954 global professionals by the Forum Corporation found that although 87 percent of managers say that they either always or often apologize for their mistakes at work, only 19 percent of employees say that their managers apologize most or all of the time.

Naturally, managers not owning up to their errors has a direct impact on employee trust levels. Another interesting insight from the survey is that while 91 percent of employees say it’s “extremely important” to have a manager they can trust, only 48 percent of managers agree that it’s extremely important for employees to trust their managers.

So we can only assume that it’s those managers who do not place a premium on trust who are committing the following worst management sins, as identified by survey participants:

  • lying
  • taking credit for others’ ideas or blaming
    employees unfairly
  • gossiping
  • poor communication
  • lack of clarity.

Managers may condone their mistakes because they are afraid of tarnishing their image. According to the survey, 51 percent of managers believe apologizing makes them appear incompetent, 18 percent believe it makes them look weak, and 18 percent shrug it off, saying that apologizing is unnecessary.

Unfortunately, the study also shows that a low regard for employees’ trust may result in low engagement levels.

This note caught my attention for a few reasons. First, this concept of trust is one that is central to the “manager as coach” work we have been doing in defining the foundation of a productive relationship that is required (in our opinion) if a manager is to be a successful coach for his/her team members.

Trust is also manifested in the perceptions of senior management, whether that group is perceived as individuals or in their aggregate actions. Either way, time after time we see that employee surveys indicate that “trust in senior leadership” is usually the primary driver of employee engagement, confirming the last sentence of the article.

Secondly, the basis for trust (or lack thereof), as listed in the bullets, is determined by behaviors. Behaviors are a choice; a person can choose to do them or not. That choice can be influenced by consequences. Evidently, a majority of managers see more value in behaving badly. We can change that behavior by making them aware that they are behaving badly, and then having negative consequences for doing so. From top to bottom.

Thirdly was the discrepancy between the importance of trust to employees versus their managers. It is hard to believe that organizations do not preach honesty, integrity and so on, whether through Values statements that hang on the walls, or by lip service. It does suggest that there is inadequate accountability.

This T+D blurb is another in a series of articles and blogs I have seen recently that bemoan bad leader behavior and the effect on an organization’s climate (see my recent blog http://dwbracken.wordpress.com/2014/02/05/nimble-and-sustainable/), but with no specific recommendation as to a solution.

I really hate whining without a proposed solution. I have suggested that a 360 process with accountability (i.e., consequences, good or bad) is a viable solution.   I recently heard of a major organization that has introduced a new leadership behavior (competency) model, and, when I asked how leaders are to be measured against the model, the response what to fall back on single-source supervisor evaluation because “360’s haven’t worked here.” I felt like I was in a backward time warp to 20 years when we started talking seriously about the shortcomings of single-source (manager) performance evaluations (see Edwards and Ewen’s first 360 degree feedback book).

Behaviors can be shaped, starting with creating awareness that change is needed, aligning to the desired behavior, and usually requiring consequences (i.e., accountability). A few leaders will change without the carrot & stick, but those are usually the ones who are not the ones who need fixing.

If you have leaders who are undermining trust, you have a problem. I think there is a solution.

 

Nimble and Sustainable

with one comment

Adam Bryant, who writes the NY Times Corner Office feature that I have referenced on multiple occasions, is finally publishing an overview of observations from his interviews of senior leaders in the form of a book (“Quick and Nimble”) and a synopsis in the January 4 edition of the Times’ Business section (http://www.nytimes.com/2014/01/05/business/management-be-nimble.html?pagewanted=1&_r=0&hp), called “Management Be Nimble.”

In this article, offers 6 drivers of innovation, and I’m going to highlight 3 to make a point.  So here they are, each with a descriptive quote from the article:

Rules of the Road

“…if employees start seeing a disconnect between the stated values and how people are allowed to behave, the entire exercise of developing explicit values will damage the organization. People will shut down, roll their eyes and wonder why on earth they hoped that this time might be different.”

A Little Respect

“When we have problems with somebody gossiping, or someone being disrespectful to a superior or a subordinate, or a peer, it is swarmed on and dealt with”

It’s About the Team

“To foster such a culture, many C.E.O.’s establish a simple rule for their employees: They have to do what they say they are going to do.”

OK, I think we get it.  I, and many/most of you, understand these things, and I, for one, have been building these principles into talks about culture change for a long time. They basically come in the form of:

  • Define the values/culture/climate of the organization in behavioral terms, and then walk the talk
  • Call out bad behavior and address it
  • Hold people accountable when they violate promises, either to the company or each other

The problem, of course, is that creating and sustaining a culture requires that it applies to everyone in the organization so that employees know what to expect from each other (and their leaders), positive behavior can be rewarded, and misbehavior addressed. 

About this same time, Booz & Company released a report, Culture’s Role in Enabling Organizational Change, that has received quite a bit of attention and points out the significant potential barrier to change that culture can present:

A change plan may be especially hard to implement if employees see the transformation as being contrary to the company’s culture—to the many things, such as feedback and peer and manager behavior, that determine (as people often put it) “how we do things around here.”

The question that Adam’s article raises is how organizations can maintain their “nimbleness” while at the same time maintaining the kind of culture they desire.  I maintain that “nimbleness” and “sustainable culture” don’t have to be oxymorons.  But as organizations grow and evolve, things happen that challenge the maintenance of their culture, such as:

  • More people, more supervisors, more variability in styles
  • Larger spans of control, less ability to monitor
  • Bring in leaders from outside, not “home grown”
  • Remote locations

In my last blog (http://dwbracken.wordpress.com/2013/12/19/get-in-touch/), we considered Liz’s opinion that 360 feedback processes are all vile and that no organization needs that level of formality and rigor.  It is undoubtedly true that small organizations do not need a traditional 360 feedback process to know how their employees are behaving or misbehaving. But with challenges such as those listed above confronting growing, thriving organizations, it becomes impractical to expect that a culture can be monitored and maintained by walking around and hanging out at the virtual or real water cooler.

So I ask Mr. Bryant, just how are these drivers going to be operationalized?  The article I published with Allan Church (http://www.orgvitality.com/articles/HRPSBrackenChurch OV.pdf ) enumerates the benefits of 360 feedback processes in bringing about sustainable behavior change and resulting culture change, which, by the way, requires integration into performance management and other human resource systems (which is also endorsed in the Booz report). 

Part of the challenge is in putting in place the feedback process that will define and then monitor behavior that is consistent with the desired culture without it becoming too cumbersome.  One approach we see surfacing is the “nudge,” a kind of pulse feedback process using an abbreviated list of key behaviors administered on a regular (quarterly?) basis with some sort of accountability attached.  We see Google, for instance, implementing such a process with significant success (http://www.hreonline.com/HRE/view/story.jhtml?id=534355733&ss=The+People+Scientist). 

If someone else knows a better way to satisfy the requirements for system-wide behavioral definition, measurement, and accountability that doesn’t use multisource feedback, I’m all ears.

©2014 David W. Bracken

Written by David Bracken

February 5, 2014 at 4:15 pm

Get In Touch

with 3 comments

My friend and co-manager of the Linked In SIOP Practitioner Network, Paul Thoresen, passed along this link (http://lnkd.in/bP4ea69) that took me to a blog titled, “The Truth About 360-Degree Feedback”, noting that it was getting a lot of traction on LI.  At last, someone has seen the “truth”!!  This person is Liz Ryan, and she does indeed have quite a following.

Well, her “truth” is that 360 is “vile” and “garbage” and all sorts of nasty, inhuman things that keep us from talking to one another.  She says that employees shouldn’t give “performance reviews” on their peers but evidently it’s OK to give feedback, so I’m not sure what the distinction is.  We have all hoped that someday employees would be able to give honest, constructive feedback to each other. We also hope that all supervisors act consistently with organizational goals and values.  I also hope to win the lottery.

She says, “If your managers want to know how their Team Leaders are doing, they can get out of their offices and observe.”  Really?  There are so many realities working against that, they are impossible to count. The realities of work life are larger spans of control, empowered leaders performing in teams and out of the line of sight of their management, work happening in virtual teams across geographical boundaries on nonobservable technologies, matrix organizations and so on.

She says that leaders want to know, “Can you tell me about a specific situation where I did something you would have liked me to do differently, so that I can learn from it?” That’s what the 360 instrument can’t give you.”  Every 360 I do asks exactly that in the write in comments, and we expect our 360 participants to have a discussion with their team members and peers regarding their feedback. In other words, it is a discussion starter, a conversation enabler.  I wonder what kind of 360 she has experienced because there are indeed some bad ones in design and use.

Her solution:  Create an “assignment”.  I want you to sit down with each person on your team, individually and in a private place, and ask him or her “What is one thing I can do to be a better team leader?” Just ask for one suggestion. Write down the suggestions you get and then let’s sit and talk about them. When we meet for that conversation, I want to hear your suggestions for how I can be a better manager for you.

According to Liz, that’s going to build trust even where none exists. According to Liz, that’s going to create a more human environment. What she is describing is the kind of “one time” event she rails against in describing the vile 360 feedback system she has experienced.  Somehow writing down suggestions is going to create behavior change where none has existed in the past, where there is no follow up nor accountability.  Welcome to La La Land, Inc.

I suggest Liz and you all take a look at this recent Doonesbury cartoon (http://www.gocomics.com/doonesbury/2013/12/08#.UrC21OLWs24) that gets at one aspect of the need for employees to have a vehicle for identifying inappropriate supervisory  (or even peer) behavior that is inconsistent with organizational expectations, whether they be values or even policies and laws.  What Liz is proposing is basically following the “chain of command” in the organization, and expecting the employee to have faith that they can all of a sudden what they have not been perhaps allowed to do for their whole career, i.e., be honest with their boss, and hope that their input will be accepted and that there will be no retribution. That is a huge leap of faith unfortunately for all too many employees. And I’m afraid Liz is out of touch with that reality as well. Installing a “hot line” seems to be her solution for handling the “end runs” she also rails against. The Doonesbury example is an extreme one, but the point can be taken down a few notches to less egregious examples of behavior inconsistent with company values, even as simple as refusing to acknowledge the viewpoints of others.

But using 360’s to manage the negative performing end of the distribution is a minor part of the story. They are also used to identify and develop the leaders of the future for placement in our high potential programs and succession planning systems. They help guide development planning and coaching experiences and give us data that can measure progress over time so we know if our training and development systems are working.

The fact is that a well-designed and implemented 360 feedback system creates a level playing field for employees to see what is expected of their managers and peers. If every leader is required to participate, it creates real and perceived fairness.  It creates an opportunity to receive feedback which is necessary to for behavior change to occur.  Some people don’t like to receive feedback and they probably don’t have mirrors in their houses either.

As for stack rankings and performance reviews and even 360’s that are used to help make decisions about employees, the fact is that in organizations some people get more and some get less, whether it’s pay, promotions, even development experiences.  I’m  not sure how those decisions get made in a fair way in Liz’s La La Land where there is no performance review or 360’s, but 360’s done fairly and consistently can help inform decisions by collecting reliable information that is superior than that collected by a single source, whether that be a single supervisor, HR manager, or water cooler.

At the end Liz proposes 3 questions that a manager can use at his/her staff meeting:

“How are we doing?”

“How are you doing?” and

“How am I doing, managing you guys?”

These are great suggestions and are often part of the action plans coming out of 360 feedback coaching.  The problems are that:

1)      Many managers don’t do it, and they are the ones who need it most

2)      When managers do it, the responses are nonexistent or not honest

360 Feedback will catch up with the managers in Group 1, and create a forum for employees in Group 2.

No two 360 Feedback processes are alike and therefore certainly vary in their quality and effectiveness. To lump them all together and then label them as “vile” or whatever is certainly not responsible, Nor are they the answer to all organizational woes; far from it. But they can make some processes incrementally better if done well.

Get in touch, Liz

Written by David Bracken

December 19, 2013 at 2:33 pm

Strategic 360 Forum II

leave a comment »

Last July, 12 companies convened in New York city for the first meeting of the Strategic 360 Forum, a full day of presentations and discussions by companies that use 360 Feedback for more than just leadership development.  The event was a great success and the group agreed that a second meeting would be useful.  Almost all of the same organizations will be returning, and we already have 3 new member organizations signed up!  We have room for a few more attendees, so here is the information:

Strategic 360 Forum II

February 25, 2014

Description

One day meeting, coordinated by David Bracken (OrgVitality), of organizations using 360 Assessments for strategic purposes, including support of human resource processes (e.g., talent management, staffing, performance management, succession planning, high potential programs).   Attendees will be senior leaders with responsibilities for both process implementation as well as strategic applications. Larger organizations (5000+ employees) will be given priority consideration for inclusion..

Location and Date:  February 25, 2014 at KPMG, 345 Park Avenue, New York, NY 10154-0102

Tentative Participant Organizations:  GlaxoSmithKline, KPMG, Starwood, PepsiCo, Federal Reserve NY, JP Morgan Chase, Cargill, Estee Lauder, WalMart, Schott NA, Thomson Reuters

Benefits for Participants

  • Learn of best practices in the use of 360 Assessments in progressive organizations
  • Discover ways that 360 Assessments support human resource initiatives, including problems and solutions
  • Create personal networks for future situations
  • Create opportunities for future professional contributions, including 2014 SIOP Conference

NOTE: The specific process and agenda will evolve as the organizers interact with the participants and discover their expectations and ways that they can best contribute to the event.

Cost

There is no cost for participants beyond their active contribution. Lunch is provided.

Content

The core content will consist of brief presentations by select attendees.  Presentations will be followed by a group discussion where questions can be asked of the presenter and alternative viewpoints shared.

Depending on the programs and interests of the participating organizations, we will explore select  theme topics of high relevance relating to use of 360 Feedback. These topics may include:

  • Performance Management
  • Succession Planning
  • High Potential Identification and Development
  • Staffing/Promotions
  • Coaching Programs
  • Sustainability

Contact Information

Interested organizations should email me with a brief description of the 360 process(es) you wish to highlight/share (purpose, size, longevity, innovations), and your personal role/responsibilities.

David W. Bracken, Ph.D.

Vice President, Leadership Development and Assessment

OrgVitality, LLC

402-617-5152 (cell)

david.bracken@orgvitality.com

Frequency: Too Often

with 4 comments

I delivered a webinar last week on using 360 Feedback in Performance Management Processes (PMP), partially built upon a recent article that Allan Church and I published in HRPS’s People & Strategy journal on that topic (let me know if you want a copy).  In the webinar, I spent a little time talking about the challenges of creating reliable/valid measurement when we are relying on input not from the target person but from observers of his/her behavior. 

One of the many elements that come into play when asking employees to rate something (a person, an organization) is the rating scale that is being used.  Note also that the rating scale’s effectiveness is likely to be directly affected by the quality of rater training, which is often neglected beyond the most basic of written instructions. 

In the webinar, I shared a list of a dozen or so various rating scales that I have encountered over the years, all in a 5 point format.  We also see in The 3D Groups recent benchmark study of over 200 organizations that use 360 feedback that, by far, the 5 point scale and the Likert Agree/Disagree format are used more often than any other scale type.  I’m not going too far out on a limb to propose that the use of the 5 point Likert scale is a carry over from employee surveys.  While there is something to be said for familiarity, I also propose that this practice is a form of laziness in 360 designers who haven’t reflected long or hard enough to consider scales that work better when the target is a specific person and not some nebulous entity like an organization that is the focus of the engagement survey.

I have advocated for the need to have the scale to match the purpose in an earlier blog (http://dwbracken.wordpress.com/2010/09/02/put-your-scale-where-your-money-is-or-isnt/) so I will move on to another pet peeve.

In the last few weeks, I pulled together a group of colleagues to submit a proposal for a SIOP symposium on helping managers to be better coaches. This process is always fun when you see research others are conducting in an area where you have special interests (kind of like buying a box set of CDs by a favorite artist and discovering some less well known gems).  One of the research papers demonstrates once again the inadequacy of frequency scales (typically 5 point scales that ask how often the person does something, ranging from Never to Always). 

Frequency scales continue to be widely used.  The aforementioned 3D Group study indicates that 23% of the reporting organizations use this scale, third most often behind Agreement (49%) and Effectiveness (31%) (which adds up to more than 100%; it may be that companies were allowed to report on more than one 360 process in their organization).  Frankly, the 23% is shockingly high.  Very recently (http://dwbracken.wordpress.com/2013/08/11/what-is-a-coach-redux/) I cited a study that presents a newly developed questionnaire about manager behaviors in the context of performance management that uses a frequency scale, to my chagrin.

For starters, a frequency scale is conceptually flawed. People can’t do everything “Always” (or even Almost Always, as some scales use).  And because they do something “always” doesn’t mean they do it well, and, conversely, because they do it Rarely or Never doesn’t mean they are bad at it. 

As importantly, every time I have seen them scrutinized in research, frequency scales come out poorly in comparison to other formats in terms of reliability and validity.  This is the 20th anniversary of a paper Karen Paul (now at 3M) and I presented at SIOP that indicated that frequency scales severely penalize supervisors who do some things infrequently but are otherwise perceived to be effective.

In a (frankly) more rigorous piece of research by Kaiser and Kaplan (2006) (that you can access here: http://kaplandevries.com/thought-leadership/list/C44), they also demonstrate that frequency scales are, by far, less satisfactory when compared to Evaluative and “Do More/Do Less” scales.

Frequency scales are used far too frequently.  They should be used Never.

 

Kaiser, R.B., & Kaplan, R.E. (2006, April). Are all scales created equal? Response format and the validity of managerial ratings. Paper in B.C. Hayes (Chair), The Four “Rs” of 360º Feedback: Second Generation Research on Determinants of Its Effectiveness, symposium presented at the 21st Annual Conference of the Society for Industrial and Organizational Psychology, Dallas, TX.

 

©2013 David W. Bracken

Written by David Bracken

September 25, 2013 at 1:09 pm

Follow

Get every new post delivered to your Inbox.

Join 36 other followers