Strategic 360s

Making feedback matter

Posts Tagged ‘Josh Bersin

Crowd-sourced: Drive-thru Feedback?

leave a comment »

danger-fast-food

(co-authored with Dale Rose)

There were a couple of interesting webinars in the last 2 weeks on the topic of performance management trends.  One was hosted by AON (Levi Segal and Seymour Adler) and the other by Talent Quarterly (Dave Ulrich, hosted by Marc Effron).

I (Dave) am particularly interested in this topic at this moment because I will be hosting my own discussion/debate on this topic at SHRM Florida on August 31 in Orlando. There I will be joined by Keith Lykins (Lykins International) and Joann Gamicchia (Orange County Clerk of Courts) to share our perspectives and engage the audience in an exchange.

As a result, I recently became aware of work by Gerry Ledford regarding trends in the field of performance management (http://goo.gl/lpv8OZ).  He writes about “cutting-edge performance management,” which is characterized by three things: Ongoing feedback, ratingless reviews, and crowd-sourced feedback.

While there has been a lot of banter recently about how to create ongoing discussions between managers and their direct reports, what really caught my attention was this statement about crowd-sourced feedback (CSF):

There is very little written about and almost no research on this growing area, but we think it may replace traditional 360 feedback over time. It uses a technology (social media) that most employees know, it is delivered in real time rather than annually, and the feedback is free form and therefore less artificial than a 360 rating form.

It is interesting to hear a well respected author suggest that a feedback method with a fairly sizable research base might be replaced by another method because the new method is 1) familiar, 2) faster, and 3) easier to do.  This sounds a little like replacing a healthy nutritious meal with fast food. It’s not that fast food is without any merit – certainly we’ve all traveled enough to know that sometimes you just need something quick and easy.  But let’s not jump too quickly into assuming that fast-food-feedback will serve the same needs as 360° feedback.  Gerry is certainly correct that crowd-sourced feedback does not qualify as 360° feedback, especially if you compare it to the definition that we (Bracken, Rose & Church, in press) have proposed:

360° Feedback is a process for collecting, quantifying, and reporting co-worker observations about an individual (i.e., a ratee) that facilitates/enables three specific data-driven/based outcomes: (a) the collection of rater perceptions of the degree to which specific behaviors are exhibited; (b) the analysis of meaningful comparisons of rater perceptions across multiple ratees, between specific groups of raters for an individual ratee, and changes over time; and (c) the creation of sustainable individual, group and /or organizational change in behaviors valued by the organization.

At this point, it is difficult to make generalizations and comparison with 360° Feedback because CSF comes in many different forms. Josh Bersin’s review of the emerging feedback market has no clear category for the type of feedback system Ledford describes. Just from what we have read in various articles, we see that CSF might be:

  • “Push” feedback (ratees asking for feedback)
  • “Pull” feedback (raters provide feedback on their own, at their own initiative)
  • “Event” oriented (e.g., how did I do in a presentation?), though this is not really “ongoing”
  • Totally unstructured (open ended comments on whatever topic occurs to the rater)
  • Open ended but requires attaching comments to rating dimensions
  • Monitored by the organization or unfettered
  • Only for ratee or shared with/used by the organization (manager, HR, other decision makers)

We see potential value in many of these types of feedback, but they clearly do not provide the same benefits to a leader or organization that 360° Feedback can provide.

If we can make some comparisons between true 360° Feedback and CSF, we see these differences of some significance:

  1. Open-ended feedback (which CSF relies on) is highly skewed to a narrow set of content areas (Rose et al, 2004)
  2. Self-selection in crowd sourcing causes sampling bias
  3. CSF makes no allowance for “opportunity to observe” error/bias, i.e., the competence and motivation of the source (rater)
  4. CSF has no method to track individual or group change over time
  5. By using standardized survey content, 360s allow strategically-aligned behavior change across the system
  6. Use of feedback to create real change is greater with 360s (until proven otherwise)
  7. Well done 360s have safeguards against retaliation and misuse
  8. Normative comparisons to other company leaders is an option with 360s
  9. 360s can be aggregated to view company-wide or system-wide trends that can be compared over time (crowd sourcing cannot)
  10. Unlike CSF, 360s allow for census participation – all leaders can be directed to participate in a standardized process; allowing leaders to create organization-wide shifts in behavior and culture.

CSF’s are equivalent to 1-2 item 360’s in most cases where the rater is providing feedback on a very narrow set of behaviors (which may or may not be specified, may or may not be actionable). They are narrowly focused on content that may or may not be aligned with organizational competencies and/or values. They may be more timely than regularly scheduled 360’s, but not necessarily so (CSF may not be timely, and 360’s do not have to be just annual events). The opportunity for timeliness may be an illusion, an opportunity offered but not always fulfilled.

Dr. Ledford’s call for more research needs to be answered.  Here are some things we would like to know:

  • What are the various contexts in which CSF is collected? (We certainly should combine different methods in examining the effectiveness of the feedback, though we could compare methods).
  • Do ratees actually use the feedback (i.e., change their behavior, let alone pay attention to it)?
  • Does the novelty wear off over time?
  • What types of individuals avoid CSF vs. those who use it frequently? (are high performing early career employees more likely to use CSF than veterans with a long track record of success?)
  • What is the differential effect due to type of CSF?
  • What are the opinions of CSF? For users, nonusers and other stakeholders (e.g., HR, management)?

While we are certainly encouraged that there is so much interest in finding ways to improve employee feedback, it’s worth recognizing that 360° Feedback has a long history of success helping leaders to learn from their environment.  Further, there is a fair amount of research and consensus around best practice in 360° Feedback.  Hopefully researchers and practitioners will take a careful look at new feedback methods like CSF.  Until we have a longer track record and much more experience with CFS, it may be a bit premature to assume that CFS will fully serve an organization’s need for valid feedback that is useful for guiding a wide range of talent decisions.

This is not necessarily an either/or choice between using 360° Feedback and CSF. But we don’t think it should ever be a “CSF only” choice.

 

Bersin, J. (2015). Feedback is the killer app: A new market and management model emerges. Forbes, August 26.  Retrieved at http://www.forbes.com/sites/joshbersin/2015/08/26/employee-feedback-is-the-killer-app-a-new-market-emerges/#41bf71036626

Bracken, D. W., Rose, D. S., & Church, A. H. (in press). The evolution and devolution of 360° feedback. Industrial and Organizational Psychology: Perspectives on Science and Practice.

Rose, D. S., Farrell, T., & Robinson, G. N.  (2004). Are Narrative Comments in 360-Degree Feedback Useful or Useless?  Technical Report #8253. Berkeley, CA: Data Driven Decisions, Inc.

Advertisements

No Fighting in The War Room!

leave a comment »

[tweetmeme source=”strategic360s”]

My apologies (or sympathies) to those of you who have not seen the black satire, “Dr. Strangelove: or How I Learned to Stop Worrying and Love the Bomb,” which contains the line, “No fighting in the War Room!”  I was reminded of this purposively humorous contradiction in reading an otherwise very insightful summary of the state of feedback tools by Josh Bersin that I hope you can access via LinkedIn here:  https://www.linkedin.com/pulse/employee-feedback-killer-app-new-market-emerges-josh-bersin.

Mr. Bersin seems quite supportive of the “ditch the ratings” bandwagon that is rolling through the popular business literature, and his article is a relatively comprehensive survey of the emerging technologies that are supporting various versions of the largely qualitative feedback market.  But right in the middle he made my head spin in Kubrick-like fashion when he starts talking about the need for ways to “let employees rate their managers,” as if this a) is something new, and b) can be done without using ratings.  Instead of “No fighting in the War Room!”, there is “No rating in the evaluation system!”   I’m curious: Is an evaluation not a “rating” because it doesn’t have a number? Won’t someone attach a number to the evaluation? Either explicitly or implicitly? And wouldn’t it be better if there were some agreement as to what number is attached to that evaluation?

What I think is most useful in Bersin’s article is his categorization and differentiation of the types of feedback processes and tools that seem to be evolving in our field, using his labels:

  • Next Generation Pulse Survey and Management Feedback Tools
  • “Open Suggestion Box” and Anonymous Social Network Tools
  • Culture Assessment and Management Tools
  • Social Recognition Tools

I want to focus on Culture Assessment and Management Tools, in the context of this discussion of ratings and performance management, and, in doing so, referencing some points I have made in the past. If you look at Mr. Bersin’s “Simply Irresistible Organization” (in the article), it contains quite a few classic HR terms like “trust,”, “coaching”, transparency,” “support,” “humanistic,” “inspiration,” “empowered,” and so on, that he probably defines somewhere but nonetheless cry out for behavioral descriptors to tell us what we will see happening when they are being done well, if at all. Ultimately it is those behaviors and the support for those behaviors that defines the culture. Furthermore, we can observe and measure those behaviors, and then hold employees accountable for acting in ways consistent with the organization’s needs.

To quote from Booz & Co in 2013:

On the informal side, there must be tangible behaviors that demonstrate what the culture looks like, and they must be granular enough that all levels of the organization can exhibit the behaviors.”

“On the formal side — and where HR can help out — the performance management and rewards systems must reward people for displaying the right behaviors that exemplify the culture. Too often, changes to the culture are not reflected in the formal elements, such as the performance-management process. This results in a relapse to the old ways of working, and a culture that never truly evolves.

Of course, all that requires measurement, which requires ratings. Which, in turn, begs for 360 Feedback, if we agree that supervisory ratings by themselves are inadequate. My experience is that management demand ratings. My prediction is that unchecked qualitative feedback will also run its course and be rejected as serving little purpose in supporting either evaluation or development.

There may be a place for the kind of feedback that social networks provide that is open and basically uncontrolled in providing spontaneous recognition. But I totally disagree with Mr. Bersin who states that any feedback is better than no feedback.  I have and still do counsel against survey comment sections that are totally open and beg for “please whine here” types of comments that are often not constructive and not actionable.

Mr. Bersin brings up the concept of feedback as a “gift” that I recently addressed as going against the notion that feedback providers need to have accountability for their feedback and see it as an investment, not a gift, especially a thoughtless gift (https://dwbracken.wordpress.com/2015/04/06/feedback-is-not-a-gift-its-an-investment/).

There is a very basic, important difference in how the field of feedback is trending, i.e., more quantity, less quality, too many white elephants. We need more 401Ks.

©2015 David W. Bracken