Built to Fail/Don’t Let Me Fail

This is a “two sided” blog entry, like those old 45 rpm records that had hit songs on both sides (think  “We Can Work It Out”/”Daytripper” by the Beatles),though my popularity may not be quite at their level.  This is precipitated by a recent blog (and LinkedIn discussion entry) coming from the Envisia people. The blog entry is called, “Does 360-degree feedback even work?” by Sandra Mashihi and can be found at http://results.envisialearning.com/.   It would be helpful if you read it first, but not necessary.

Sandra begins by citing some useful research regarding the effectiveness of 360 processes. And she concludes that sometimes 360’s “work” and sometimes not.  Her quote is, “Obviously, the research demonstrates varied results in terms of its effectiveness.”

What is frustrating for some of us are the blanket statement about failures (and using terms like “obvious”) without acknowledging that many 360’s are “built to fail.” This is the main thesis of the article Dale Rose and I just published in the Journal of Business and Psychology. http://www.springerlink.com/content/85tp6nt57ru7x522/

http://www.ioatwork.com/ioatwork/2011/06/practical-advice-for-designing-a-360-degree-feedback-process-.html

Dale and I propose four features needed in a 360 process if it is likely to create sustainable behavior change:

1)      Reliable measurement: Professionally developed, custom designed instruments

2)      Credible data: Collecting input from trained, motivated raters with knowledge of ratees

3)      Accountability: Methods to motivate raters and ratees to fulfill their obligations

4)      Census participation: Requiring all leaders in an organizational unit to get feedback

We go on to cite research that demonstrates how the failure to build these features into 360 can, in some cases, almost guarantee failure and/or the ability to detect behavior change when it does occur. One such feature, for example, is whether the ratee follows up with raters (which I have mentioned in multiple prior blogs). If/when a 360 (or a collection of 360’s, such as in a meta analysis) is deemed a “failure”, I always want to know things such as whether raters were trained and whether follow up was required, for starters.

We are leaning more and more about the facets that increase the probability that behavior change will occur as a result of 360 feedback. Yet all too often these features are not built into many processes, and practitioners are surprised (“shocked, I’m shocked”) when it doesn’t produce desired results.

Sandra then goes on to state: “I have found 360-degree feedback worked best when the person being rated was open to the process, when the company communicated its purpose clearly, and used it for development purposes.” I assume that she means “development only” since all 360’s are developmental.  I definitely disagree with that feature. 360’s for “development (only) purposes” usually violate one or more of the 4 features Dale and I propose, particularly the accountability one. They often do not generate credible data because too few raters are used, even the best practice of including all direct reports.

The part about “being open to the process” is where I get the flip side of my record, i.e., don’t hurt my feelings.  In one (and only one) way, this makes sense. If the ratee doesn’t want to be in a development-only process, then by all means don’t force them. It is a waste of time and money. On the other hand, all development only processes are a waste of money in my opinion for most people. (And, by the way, development only is very rare if that means that no decisions are being made as a result.)

But if we ARE expecting to get some ROI (such as sustained behavior change) from our 360’s, then letting some people to opt out so their feelings aren’t hurt is totally contrary to helping the organization manage its leadership cadre. Intuitively, we should expect that those who opt out are the leaders that need it the most, who know that they are not effective and/or are afraid to be “discovered” as the bullies, jerks, and downright psychopaths that we know exist out there.

I have some fear that this fear of telling leaders that they are less than perfect is stemming from this troubling trend in our culture where everyone  has to succeed. I think that the whole “strengths” movement is a sign of that.

Over the last couple of weeks, I have seen a few things that further sensitized me to this phenomenon. One big one is this article in The Atlantic: http://www.theatlantic.com/magazine/archive/2011/07/how-to-land-your-kid-in-therapy/8555/1/.  Protecting our children from failure is not working. Protecting our leaders from failure is also dooming your organization.

I swear I never watch America’s Funniest Videos, but during a rain delay of a baseball game recently, I did stumble upon it and succumbed. AFV is all about failure, and I’m not so sure that people always learn from these failures. But one video I enjoyed showing a 2 year old boy trying to pour apple juice from a BIG bottle into a cup. He put the cup on the floor and totally missed the first two times (with the corresponding huge mess). As a parent and grandparent, I was quite amazed that the person behind the camera just let it happen. But on the third try, the task was accomplished successfully, followed by applause and smiles! There was a huge amount of learning that occurred in just a minute or two because the adults allowed it to happen, with a bit of a mess to clean up.

How many of us would have just poured the juice for him? His learning isn’t over; he will make more mistakes and miss the cup occasionally. But don’t we all.

As a parting note, Dale and I support census participation for a number of reasons, one of which is the point I have already made about otherwise missing the leaders that need it most. We also see 360’s as a powerful tool for organizational change, and changing some leaders and not others does not support that objective. Having all leaders participate is tangible evidence that the process has organization support and is valued. Finally, it creates a level playing field for all leaders for both evaluation and development, communicating to ALL employees what the organization expects from its leaders.

©2011 David W. Bracken

I Don’t Care

Last week I led a workshop for the Personnel Testing Council of Metropolitan Washington that was a modified reprise of the workshop Carol Jenkins and I did at the Society for Industrial and Organizational Psychology in April. I really enjoy these workshops and the opportunity to interact face-to-face with practitioners in the field of 360 degree feedback.

I do wish that participants in these workshops would engage me in a little more debate, and, to that end, I sometimes throw out comments in the hope of raising some hackles. For example, at the PTCMW session, I twice said “I don’t care” regarding two topics that I will explain below. Unfortunately, no one took the bait in the workshop, but maybe I can lure some of you into the discussion using this blog as a vehicle.

So here are the two areas where a ton of research is being done but where, as a practitioner, I don’t care:

1)      The personality of the participant. I don’t care. Everyone seems to want to know how the personality of the participant is going to affect his/her reaction to the feedback.  In past blogs, I have fessed up to being a behaviorist, and in that respect all I really “care” about is getting the person to accept the feedback and to change, whether they want to or not. In my last blog, I used the examples of people’s apparent reluctance to do simple things like apologize for mistake and/or to say “thank you.”  Behaviorally, those are pretty easy things to do, but evidently some internal force (e.g., personality) makes it difficult.  In fact, those internal forces vary greatly across people, and I find chasing them down to not be a very fruitful use of time for the participant or for myself. If the organization and feedback tells you that you need to modify your behavior, just do it!

Sometimes what is going on inside the person’s head is more an issue of awareness than of personality, and awareness is something we can change through 360’s. Occasionally the journey from awareness to acceptance is difficult due to personality factors. It is our job to design the 360 process to make it difficult to not accept the feedback, including ensuring that raters are knowledgeable, reliable, motivated and in sufficient quantity.

On a practical level, when many 360 processes involve dozens or hundreds of participants, it becomes very challenging to integrate personality assessment, for example, into the mix. Not to say it can’t be done. Carol Jenkins does some of that in her practice with groups of feedback recipients. But part of my “I don’t care” mentality has come from a need to get large numbers of people to use the feedback productively without being able to “get inside their head.”

2)      The gap between self-ratings and “other” ratings. I don’t care. As a psychologist, I do find it interesting to see how ratees approach self-ratings, especially the first time around. And they usually change their self-ratings once they see how they are perceived by others. But I am increasingly convinced that self-ratings are more a reflection of the ratee’s agenda than any real self-assessment. (All raters are susceptible to using their ratings to this kind of error.) One memorable instance for me was in working with a Chief Legal Officer who gave himself all 5’s and stated, “do you think I would be crazy enough to actually document less than optimal performance?”

I DO think that participants should complete the rating process, but for other reasons. One is to ensure that they are familiar with the content and how he/she is expected to behave as defined by the organization. Secondly, it is some evidence of at least minimal commitment to the process.

In general, I am not very interested in why a ratee behaves in a certain way if it needs to change. It is highly unlikely that we can change the “why” part of behavior (i.e., personality) other than to affect their awareness of how they are perceived and the importance of accepting that feedback on the way to behaving differently.What is going on in the person’s head is fun for psychologists to research, but doesn’t necessarily help achieve sustainable behavior change.

©2011 David W. Bracken

What You See Is What You Get

Every month or so I get an invitation/newsletter from Marshall Goldsmith and Patricia Wheeler. This month’s had a couple gems in it, and I have provided the link at the end of this article.  Marshall’s entry on life lessons is very much worth reading. But Patricia’s offering particularly struck me since I have been thinking a lot about leader behavior. As you will see it also relates directly to the hazards of misdiagnosis, another human flaw that is especially salient for those of us in consulting and coaching where we are prone to jumping to conclusions too quickly.

Several years ago my mother experienced stomach pains.  Her physician, one of the best specialists in the city, ordered the usual tests and treated her with medication.  The pains continued; she returned to his office and surgery was recommended, which she had.  After discharge the pains recurred, stronger than ever; she was rushed to the emergency room, where it was determined that her physician had initially misdiagnosed her. She had further surgery; unfortunately she was unable to withstand the stress of two surgeries, fell into a coma and died several days later.  Several days after her second surgery, her physician approached me, almost tearfully, with an apology.

“I apologize,” he said, “this is my responsibility.”  He should have done one additional test, he said, requiring sedation and an invasive procedure, but he did not want to impose the pain of that procedure on her, feeling at the time that his diagnosis was correct.  “I am truly sorry and I will never make that mistake again.”  What struck me at the time and continues to stay with me is that this doctor was willing to take the risk of telling the whole difficult truth, and that taking responsibility for the situation was more important to him than the very real possibility of a malpractice suit.  I forgave him, and I believe my mother would have as well.

Real apologies have positive impact that, in most if not all cases, outweigh the risk factors.  Ask yourself, when does an apology feel heartfelt to you? When does it seem empty?  Think of a time when you heard a public or corporate figure apologize and it rang true and think of a time when it didn’t.  What made the difference? Here are a few guidelines:

Is it from the heart or the risk management office?  If your apology reads like corporate legalese, it won’t be effective.

Is it unequivocal?  Too many apologies begin with “I’m sorry, but you were at fault in this too.”  An attempt to provoke the other party into apologizing or accepting fault will fail.

Is it timely?  If you delay your apology, perhaps wishing that the issue would just go away (trust me, it won’t), its effect will diminish proportionately.

Does it acknowledge the injury and address the future?  In other words, now that you know your words or actions have caused injury, what will you do going forward?

While we can’t avoid all errors, missteps and blind spots, we can at least avoid compounding them with empty words, blaming and justification.

Patricia is focusing on a particular behavior, i.e., apologizing. This behavior, like all other behaviors, is modifiable if we are aware of the need to change and motivated to do so.  It may not be easy and you may not be comfortable doing it, but that is no excuse. And, by the way, people really don’t care what is going on inside your head to justify not changing (e.g., “they know that I’m sorry without me saying it”). Making an apology is often difficult, as Patricia points out, and maybe that’s why it can be so striking and memorable when someone does it well.

In his book, “What Got You Here Won’t Get You There,” Marshall makes a similar point about the simple behavior of saying “thank you,” which is a common shortcoming in even the most successful leaders.  Leaders find all sorts of excuses for avoiding even that seemingly easy behavior, including “that’s just not me.” The point is that what you do and what people see (i.e., behaviors) IS who you are.

The good news for us practitioners of 360 Feedback is that observing behaviors is what it is (or should be) all about. In a 360 process, the organization defines the behaviors it expects from its leaders, gives them feedback on how successful they are in doing so, and then (ideally) holds them accountable for changing.

This also means that we go to great lengths to ensure that the content of 360 instruments uses items that describe behaviors, hopefully in clear terms.  We need to ensure that we are asking raters to be observers and reporters of behavior, not mind readers or psychologists.  We need to especially wary of items that include adjectives that ask the rater to peer inside the ratee’s head, including asking what the ratee “knows” or “is aware of” or “believes” or even what the leader is “willing” to do.

As a behaviorist, in the end I only care what a leader does and not why (or if) he/she wants to do it. That’s the main reason why I have found personality assessments to be of little interest, with the exception of possibly providing insights into how the coaching relationship might be affected by things like openness to feedback or their preferred style for guidance and learning.

Another piece of good news for us behaviorists came out in a recent article in Personnel Psychology titled, “Trait and Behavioral Theories of Leadership: An Integration and Meta-Analytic Test of Their Relative Validity” (Derue, Nahrgang, Wellman and Humphrey, 2011).  To quote from the abstract, they report:

Leader behaviors tend to explain more variance in leadership effectiveness than leader traits, but results indicate that an integrative model where leader behaviors mediate the relationship between leader traits and effectiveness is warranted.

The last part about mediation suggests that, even when traits do a decent job (statistically) of predicting leader effectiveness, they are “filtered” through leader behaviors. For example, all the intelligence in the world doesn’t do much good if you are still a jerk (or bully, or psychopath, etc.)

All of this reinforces the importance of reliably measuring leader behaviors, especially if we believe that the “how” of performance is at least as important as the “what.”

Link:  http://email.e-mailnetworks.com/hostedemail/email.htm?h=bdd4c78f38fd64341d6760533238799c&CID=4826566929&ch=487D1DD320A1A801E8ACD8949CEAC445

©2011 David W. Bracken

On the Road… and Web and Print

I have a few events coming up in the next 3 weeks or so that I would like to bring to your collective attention in case you have some interest.  One is free, two are not (though I receive no remuneration). I also have an article out that I co-authored on 360 feedback.

In chronological order, on May 25 Allan Church, VP Global Talent Development at PepsiCo, and I will lead a seminar titled, “Integrating 360 & Upward Feedback into Performance and Rewards Systems” at the 2011 World at Work Conference in San Diego (www.worldatwork.org/sandiego2011).  I will be offering some general observations on the appropriateness, challenges, and potential benefits of using 360 Feedback for decision making, such as performance management. The audience will be very interested in Allan’s descriptions of his experiences with past and current processes that have used 360 and Upward Feedback for both developmental and decision making purposes.

On June 8, I am looking forward to conducting a half day workshop for the Personnel Testing Council of Metropolitan Washington (PTCMW) in Arlington, VA, titled “360-Degree Assessments: Make the Right Decisions and Create Sustainable Change” (contact Training.PTCMW@GMAIL.COM or go to WWW.PTCMW.ORG). This workshop is open to the public and costs $50.  I will be building from the workshop Carol Jenkins and I conducted at The Society for Industrial and Organizational Psychology. That said, the word “assessments” in the title is a foreshadowing of a greater emphasis on the use of 360 Feedback in a decision making context and an audience that is expected to have great interest in the questions of validity and measurement.

On the following day, June 9 (at 3:30 PM EDT), I will be part of an online virtual conference organized by the Institute of Human Resources and hr.com on performance management. My webinar is titled, “Using 360 Feedback in Performance Management: The Debate and Decisions,” where the “decisions” part has multiple meanings. Given the earlier two sessions I described, it should be clear that I am a proponent of using 360/Upward Feedback for decision making under the right conditions. The other take on “decisions” is the multitude of decisions that are required to create those “right conditions” in the design and implementation of a multisource process.

On that note, I am proud to say that Dale Rose and I have a new article in the Journal of Business and Psychology (June) titled, “When does 360-degree feedback create behavior change? And how would we know it when it does?” Our effort is largely an attempt to identify the critical design factors in creating 360 processes and the associated research needs.

This article is part of a special research issue (http://springerlink.com/content/w44772764751/) of JBP and you will have to pay for a copy unless you have a subscription. As a tease, here is the abstract:

360-degree feedback has great promise as a method for creating both behavior change and organization change, yet research demonstrating results to this effect has been mixed. The mixed results are, at least in part, because of the high degree of variation in design features across 360 processes. We identify four characteristics of a 360 process that are required to successfully create organization change, (1) relevant content, (2) credible data, (3) accountability, and (4) census participation, and cite the important research issues in each of those areas relative to design decisions. In addition, when behavior change is created, the data must be sufficiently reliable to detect it, and we highlight current and needed research in the measurement domain, using response scale research as a prime example.

Hope something here catches your eye/ear!

©2011 David W. Bracken

Who’s in charge here?

In my last blog, I touched on an issue that I would like to give a little more attention, namely the question of who our leaders (i.e., the recipients of 360 Feedback) should be listening to when prioritizing their leadership skills and behaviors: the organization, their coworkers, and/or some combination?  This is a topic that has cultural implications that I alluded to in that earlier blog, and I will post this out on the Society for Industrial/Organizational Psychology (SIOP) Going Global group on LinkedIn to see if I can get a response from someone there. But it also has broad implications for how we approach leadership selection, assessment and development in general.

One end of the continuum is to state that an organization has the right (and need) to define leadership competencies/behaviors, ideally derived from strategy, to support its initiatives and create a unique competitive advantage (see the treatment of leadership as an intangible asset in the book, The Invisible Advantage).  Organizations regularly create leadership models that are used to align HR systems in creating the type of leader needed to succeed at the individual and organization level.  This includes values statements that often are operationalized through behavioral items in 360’s that hopefully apply to leaders and line employees alike.

In the context of 360’s, I have long maintained that 360’s can draw their relevance (read “validity”) from a direct line of sight from strategy to leadership models to 360 content, and that it is not necessary to conduct predictive studies to demonstrate the validity of the 360 process. The content of a 360 instrument can define success, and that leaders that behave in ways consistent with the model are successful by definition.  Those who do not conform to those expectations should be give the choice of changing or to find alternative employment.

That is the “top down” version of “who’s in charge here?”  I was influenced in my thinking many years ago by some monographs by Bill Byham at DDI on this topic that were often in the context of assessment center processes but certainly no less applicable. Dr. Byham has written a number of papers on aligning HR systems with competency models, and you can find much of that on their web site.

The other end of the continuum of “who’s in charge here?” is the “bottoms up” view of leadership effectiveness that suggests that the leader’s behavior should be directed by coworkers, particularly direct reports. This view came into greater clarity for me at SIOP during a symposium that I briefly described in my last blog on implicit leadership models.

For decades, this “bottoms up” view of leadership has been implicit in the use of importance ratings. I have been opposed to importance ratings for as long as I can remember, partially because of the extra burden to raters, but, more importantly, because raters are not in a good position to understand the needs of the leader and the organization. I still believe that importance is best determined by the ratee and his/her boss in conjunction.  Asking raters for importance ratings feels like a customer survey. If your 360 treats raters as “customers” of leader behavior, then use a satisfaction scale and design the whole system accordingly.

The SIOP symposium had a number of interesting research studies that explicitly state that an effective leader should understand and react to the needs of coworkers based on the coworkers’ expectations of how an “effective” leader should behave which is, in turn, derived from their cultural backgrounds (i.e., nationality). This is very interesting and cultural awareness is an important issue in our global community.  As some of these papers pointed out, leaders now (especially with virtual teams) can have coworkers and direct reports from multiple nations and cultures, which creates a requirement that somehow the leader has to understand and adapt to the needs of each of these people. My head began to spin!

By the way, I first learned about the concept of Situational Leadership back in the 80’s and I still believe that philosophically it makes a lot of sense to not treat every subordinate the same way. But if you know Situational Leadership, it focuses exclusively on the person’s “maturity” (ability to perform a task) and the need to adapt leadership style depending on an assessment of that maturity level. It is very task oriented, and has little (if anything) to do with the needs and expectations of the follower.

There was some sentiment on the SIOP panel for asking the leader to negotiate or compromise between the “bottoms up” and “top down” views of leadership. I’m not sure how that would work, but it probably compounds one of the main problems I cited as discussant, namely the overload we are creating for leaders by inundating them with all this information in the form of job expectations. I have to believe that leaders are asking, or will ask, this question of “who is in charge?”, the organization or their coworkers?

I will take a stance. I believe that the organization is “in charge.” I did some consulting for a company of about 3500 people in Dubai that had employees with 80 different passports. They were run by South Africans, and ran the company that way. In a nutshell, they expected employees to conform to a common set of values and expectations, effectively leaving their cultural backgrounds at the door. Or at least the company “culture” should take precedence in defining effective leadership. I believe that this aligned focus on organization needs is a necessity, and that we need to make it clear to our leaders “who is in charge” when it comes to deciding how the company will leverage one of its most powerful intangible assets.

©2011 David W. Bracken

What I Learned at SIOP

The annual conference of the Society for Industrial/Organizational Psychology (SIOP) was held in Chicago April 14-16 with record attendance. I had something of a “360 Feedback-intensive” experience by running two half-day continuing education workshops (with Carol Jenkins) on 360 feedback, participating on a panel discussion of the evolution of 360 in the last 10 years (with other contributors to The Handbook of Multisource Feedback), and being the discussant for a symposium regarding Implicit Leadership Theories that largely focused on cultural factors in 360 processes. Each forum gave me an opportunity to gauge some current perspectives on this field, and here are a few that I will share.

The “debate” continues but seems to be softening. The “debate” is, of course, how 360 feedback should be used: development only and/or for decision making. In our CE Workshop, we actually had participants stand up and stand in corners of the room to indicate their stance on this issue, and, judging from that exercise, there are still many strong proponents of each side of that stance. That said, one of the conclusions the panel seemed to agree upon is that there is some blurring of the distinction between uses and some acknowledgement that 360’s are successfully being used for decision making, and that 360’s are far less likely to create sustainable behavior change without accountability that comes with integration with HR systems.

We need to be sensitive to the demands we place on our leaders/participants. During our panel discussion, Janine Waclawski (who is currently an HR generalist at Pepsi) reminded us of how we typically inundate 360 participants with many data points, beginning with the number of items multiplied by the number of rater groups. (I don’t believe the solution to this problem is reducing the number of items, especially below some arbitrary number like 20 items.)  Later, I had the opportunity to offer commentary on four terrific research papers that had a major theme of how supervisors need to be aware of the perspectives of their raters that may well be caused by their cultural backgrounds.

As someone who is more on the practitioner end of the practitioner-scientist continuum, I tried to once again put myself in the seat of the feedback recipient (where I have been many times) and consider how this research might be put into practice. On one hand, organizations are using leadership competency models and values statements to create a unified message (and culture?) that spans all segments of the company. We can (and should) have debates about how useful and realistic this practice is, but I think most of us agree that the company has a right to define the behaviors that are expected of successful leaders. 360 processes can be a powerful way to define those expectations in behavioral terms, to help leaders become aware of their perceived performance of those behaviors, to help them get better, and to hold leaders accountable for change.

On the other hand, the symposium papers seem to suggest that leader behaviors should be molded from “the bottom up,” i.e., by responding to the expectations of followers (raters) that may be attributed to their cultural backgrounds and their views of what an effective leader should be (which may differ from the leader’s view and/or the organization’s view of effective leadership).  By the way, this “bottoms up” approach applies also to the use of importance ratings (which is not a cultural question).

My plea to the panel (perhaps to their dismay) was to at least consider the conundrum of the feedback recipient who is being given this potentially incredibly complex task of not only digesting the basic data that Janine was referring to, but then to fold in the huge amount of information created by having to consider the needs of all the feedback providers. Their research is very interesting and useful in raising our awareness of cultural differences that can affect the effectiveness of our 360 processes. But PLEASE acknowledge the implications for putting all of this to use.

The “test” mentality is being challenged.  I used the panel discussion to offer up one of my current pet peeves, namely to challenge the treatment of 360 Feedback as a “test.”  Both in the workshops and again at the panel, I suggested that applying practices such as randomizing items and using reverse wording to “trick” the raters is not constructive and most likely is contrary to our need to help the raters provide reliable data. I was gratified to receive a smattering of applause when I made that point during the panel.  I am looking forward to hopefully discussing (debating) this stance with the Personnel Testing Council of Metropolitan Washington in a workshop I am doing in June, where I suspect some of the traditional testing people will speak their mind on this topic.

This year’s SIOP was well done, once again. I was especially glad to see an ongoing interest in the evolution of the field of 360 feedback judging from the attendance at these sessions, let alone the fact that the workshop committee identified 360 as a topic worthy of inclusion after going over 10 years since the last one.  360 Feedback is such a complex process, and we are still struggling with the most basic questions, including purpose and use.

©2011 David W. Bracken

Vibrating 360′s

I have one of those razors that not only has multiple blades but vibrates as well. And I really like it. Looking into this a little, having multiple blades evidently has been shown to provide a closer shave and the government permits advertisers to say so. (I remember an early Saturday Night Live spoof after the Trac II came out with two blades showing how three blades were going to be even better, and that was funny at the time.)

But the government won’t let the razor companies say that vibrations cause a closer shave.  With 5 blades (or whatever number), it’s hard to imagine that vibrations can realistically be expected to help the closeness factor. But shaving is also about comfort. I must not be the only user who finds the vibrations make shaving more comfortable. Maybe it’s psychological (heaven forbid), but it works for me and others as well.

Another characteristic of these electrified razors is that they have a battery in the handle. It is possible (even likely) that the added weight helps balance the razor and makes it more solid. It just feels better in the hand. Compare that to the other extreme, multiple blade disposable razors. They are inexpensive and feel like it.

Vibrating razors are relatively more expensive and they don’t necessarily do a better job. But they feel better, and comfort is probably a valid criterion for many, if not most, users.

I want a vibrating 360. Using this metaphor, creating behavior change with a 360 is like getting a close shave, each being the sine qua non for consumers.  If the main achievement of a vibrating razor is to make it “feel good,” as in a more comfortable experience, what would be the analogy in a 360? (Let me insert that some 360 processes are all about comfort, not effectiveness. They may vibrate but have no blades. Sounds like another blog sometime soon.)

How do we make our 360 processes more comfortable for our stakeholders (participants, raters, supervisors, management, the organization) without sacrificing effectiveness (i.e., sustainable behavior change)?

©2011 David W. Bracken

Worst is Not First

I was looking back at the list of potential topics for this blog that I constructed when I started about 9 months ago. This is one that was on the list, so it has been festering for awhile.

I thought of it again this week when I had to dive into a set of feedback reports in preparation for phone coaching sessions with these leaders next week. And I do what I suspect most leaders do when they get their own reports, i.e., go first to the write in comments and then the Top and Bottom lists (with more attention to the Bottom unfortunately).

The instructions for interpreting the results and my personal counsel is to then use this information to begin looking for themes to begin the process of narrowing down the possible areas for developmental focus.  But is that what the typical leader does? Of course not. We’re human, and, for some reason, we have this need to focus on the negative.

That results in two unproductive results for many 360 participants. First, we obsess on negative write in comments, giving WAY too much weight to single comments. Then we try to figure out who said it and/or assume we know both who said it and why they said it. One of my big challenges as a coach is to try to head that off before something bad happens, even if it is as innocuous as creating a development plan that is misguided (or the other extreme of “going postal”).

The other problem is the tendency to automatically assume the lowest scores are the ones we need to “fix.”  This relates back to my earlier blog about computers telling us what to do. Not only can computers “go too far” in telling us what is most important to work on, but coaches can also “go too far” in assuming we know what is most important.  While we’re at it, we sometimes also put raters in that position and “going too far” by asking them about importance.

Who knows best regarding importance and priorities? The leader and his/her boss.  Together they know the context of the data, past feedback, and future development needs for the leader and the organization.

We should be looking for themes in the data, both on the strength and development needs sides of the equation. A starting point might be to note if the Top/Bottom scoring items are largely from the same dimension (e.g., Communication). We should also go to the item level data to see if one group of raters is driving the high or low results (e.g., are peer scores much different from direct reports?).

Speaking of strengths, that is probably the best place to start a review discussion with a leader. That said, the total focus on strengths in some media is way overblown. I think I am in the vast majority of practitioners who see a total focus on strengths and the ignoring of development needs is a deadly prescription.

Let’s remember that the worst results are not necessarily the first, and the best don’t mean ignore the rest.

©2011 David W. Bracken

Can you change a culture?

Us folks at OrgVitality have a view of the “vital” organization that includes concepts of ambidexterity, agility and resilience. These concepts can be operationalized to promote the creation of a culture that makes those characteristics a way of life in the organization.

I found a recent article (Lengnick-Hall, Beck and Lengnick-Hall, 2010) titled, “Developing a capacity for organizational resilience through strategic human resource management.”  Their message of creating and sustaining a culture through human resource processes is a powerful concept.

These authors define resilience as:

“…a firm’s ability to effectively absorb, develop situation-specific responses to, and ultimately engage in transformative activities to capitalize on disruptive surprises that potentially threaten organization survival.”  They go on to propose that resilience should be created through individual knowledge, skills, and abilities and organizational routines and processes.

This is good stuff but I think they have missed an opportunity to talk about creating a culture through behavior change. Culture has a lot of definitions, but a couple are consistent with this view of behavior being a key factor. I have been drawn to an observable and measurable definition of culture offered by Bossidy and Charan (2002) in their seminal book, Execution: The discipline of getting things done,:

“The culture of a company is the behavior of its leaders. Leaders get the behavior they exhibit and tolerate.”

While many traditionalists will argue with such a “superficial” treatment of culture, it was foreshadowed by Kotter and Heskett (1992) who refined their definition of culture with this statement: “…culture represents the behavior patterns or style of an organization that new employees are automatically encouraged to follow by their fellow employees.” (p. 4)

This definition is too limiting in not directly acknowledging that the “fellow employees” who have the most impact on creating the culture are the leaders of that organization.

Let’s return to the resilience article. I looked for statements of behaviors that might be useful for creating a culture of resilience, particularly defined in terms of leader behavior that could easily be fodder for a 360 or upward feedback process. Fortunately for me, there is a section called, “Behavioral elements of organizational resilience.” Their language is somewhat academic (e.g., “nonconforming strategic repertoires”), but here are some examples of behaviors that I propose support their conceptualization of resilience:

• Encourages new solutions to problems
• Finds new strategies that are different from the past and industry norms
• Takes the initiative and moves quickly to overcome challenges
• Ensures that new and creative solutions are consistent with organizational goals and values
• Challenges the status quo
• Encourages the discarding of obsolete information and practices
• Recognizes and rewards behaviors that demonstrate flexibility and resourcefulness

They list a whole raft of HR policies, principles and practices that can support the development of resilience, including things like after-action reviews, open architecture, broad job descriptions, employee suggestions, and cross-departmental task forces. They reference a need to include performance reviews (“results-based appraisals) that encourage the right activities.

But nowhere is 360 feedback mentioned as a potentially powerful tool to reinforce and create culture change. Here are a few ways that 360 processes can be integral parts of a culture change initiative:

• Defines the construct (e.g., resilience) in behavioral terms
• Communicates the construct as an organizational priority (i.e., is being measured)
• Potentially communicates to all employees (raters, ratees) on a repeated basis
• Creates a metric for tracking progress over time
• Creates a metric for identifying individual, team, and organizational gaps in performance
• Creates accountability for behavior consistent with organizational needs
• Supports aligned HR practices when integrated with other HR systems (e.g., development, staffing, succession planning, performance reviews, high potential development)

This list makes some assumptions about the design and implementation of 360 processes that support culture change. That is such a large topic that it would require an entire book. Stay tuned for that.

I am amazed and disappointed that a major treatise on what is in effect culture change would not include 360 feedback as at least worth consideration as a supporting HR practice. It makes me wonder why that is.

References

Bossidy, L, and Charan, R. (2002). Execution: The Discipline of Getting Things Done. New York: Crown Business.

Kotter, J.P., and Heskett, J.L. (1992). Corporate Culture and Performance. New York: Free Press.

Lengnick-Hall, C.A., et al. (2010). Developing a capacity for organizational resilience through strategic human resource management. Human Resource Management Review, doi:10.1016/j.hrmr.2010.07.001.

©2011 David W. Bracken

What is the ROI for 360′s?

Tracy Maylett recently started a LinkedIn discussion in the 360 Feedback Surveys group by asking, “Can you show ROI on 360-degree feedback processes?” To date, no one has offered up any examples, and this causes me to reflect on this topic. It will also be part of our (Carol Jenkins and myself) discussion at the Society for Industrial and Organizational Psychology (SIOP) Pre-Conference Workshop on 360 Feedback (April 13 in Chicago; see www.siop.org).

Here are some thoughts on the challenges in demonstrating ROI with 360 processes:

1)      It is almost impossible to assess the value of behavior change. Whether we use actual measurements (e.g., test-retest) or just observer estimations of ratee change, assigning a dollar value is extremely difficult. My experience is that, no matter what methodology you use, the results are often large and cause consumers (e.g., senior management) to question and discount the findings.

2)      The targets for change are limited, by design. A commonly accepted best practice for 360’s is to guide participants in using the data to focus on 2-3 behaviors/competencies. If some overall measure of behavior change is used (e.g., the average of all items in the model/questionnaire), then we should expect negligible results since the vast majority of behaviors have not been addressed in the action planning (development) process.

3)      The diversity of behaviors/competencies will mean that they have differential ease of change (e.g., short vs. long term change) and different value to the organization. For example, what might be the ROI for significant change (positive or negative) in ethical behavior compared to communication? Each is very important but with very different implications for measuring ROI.

4)      Measurable change is dependent on design characteristics of each 360 process.  I have suggested in earlier blogs that there are design decisions that are potentially so powerful as to promote or negate behavior change. One source for that statement is the article by Goldsmith and Morgan called, “Leadership is a contact sport,” which can be found on www.marshallgoldsmith.com.  In this article  (that I have also mentioned before), they share results from hundreds of global companies and thousands of leaders that strongly support the conclusion that follow up with raters may be the single best predictor of observed behavior change.

Dale Rose and I have an article in press with the Journal of Business and Psychology titled, “When does 360-degree Feedback create behavior change?  And would we know it when it does?” One of our major objectives in that article is to challenge blanket statements about the effectiveness of 360 processes since there are so many factors that will directly impact the power of the system to create the desired outcomes. The article covers some of those design factors and the research (or lack thereof) associated with them.

If anyone says, for example, that a 360 process (or a cluster, such as in a meta analysis) shows minimal or no impact, my first question would be, “Were the participants required to follow up with their raters?” I would also ask about things like reliability of the instrument, training of raters, and accountability as a starter list of factors that can result in unsuccessful ability to cause and/or measure behavior change.

Tracy’s question regarding ROI is an excellent one, and we should be held accountable for producing results. That said, we should not be held accountable for ROI when the process has fatal flaws in design that almost certainly will result in failure and even negative ROI.

©2011 David W. Bracken