What does “beneficial” mean?
My friend, Joan Glaman, dropped me a note after my last blog, (https://dwbracken.wordpress.com/2011/08/30/thats-why-we-have-amendments/ ) with this suggestion:
“I think your closing question below would be a great next topic for general discussion: ‘Under what conditions and for whom is multisource feedback likely to be beneficial?’”
To refresh (or create) your memory, that question that Joan cites is from the Smither, London and Reilly (2005) meta analysis. The article abstract states:
“…improvement is most likely to occur when feedback indicates that change is necessary, recipients have a positive feedback orientation, perceive a need to change their behavior, react positively to the feedback, believe change is feasible, set appropriate goals to regulate their behavior, and take actions that lead to skill and performance improvement.”
Before we answer Joan’s question, we should have a firm grasp on what we mean by “beneficial.” I don’t think we all would agree on that in this context. Clearly, Smither et al. define it as “improvement,” i.e., positive behavior change. That is the criterion (outcome) measure that they use in their aggregation of 360 studies. I am in total agreement that behavior change is the primary use for 360 feedback, and we (Bracken, Timmreck, Fleenor and Summers, 2001) defined a valid 360 process as one that creates sustainable behavior change in behaviors valued by the organization.
Not everyone will agree that behavior change is the primary goal of a 360 process. Some practitioners seem to believe that creating awareness alone is a sufficient outcome since they do not support any activity or accountability, proposing that simply giving the report to the leader is going far enough and in fact discourage the sharing of results with anyone else.
If you will permit a digression, I will bring to your attention a recent blog by Sandra Mashihi (http://results.envisialearning.com/5-criteria-a-360-degree-feedback-must-meet-to-be-valid-and-reliable/) where one of her lists of “musts” (arrrgh!) is criterion related validity, which she defines as, …does the customized instrument actually predict anything meaningful like performance?” Evidently she would define “beneficial” then to not be behavior change but to be able to measure performance to make decisions about people. This testing mentality just doesn’t work for me since 360’s are not tests (https://dwbracken.wordpress.com/2010/08/31/this-is-not-a-test/) and it is not realistic to expect them to predict behavior, especially if we hope to actually change behavior.
Let’s get back to Joan’s question (finally). I want to make a couple comments and then hopefully others will weigh in. The list of characteristics that Smither et al provide in the abstract is indeed an accumulation of individual and organizational factors. This is not an “and” list that says that a “beneficial” process will have all these things. It an “or” list where each characteristic can have benefits. The last two, (set goals and take actions) can be built into the process as requirements regardless of whether the individual reacts positively and/or perceives the need to change. Research shows that follow up and taking action are powerful predictors of behavior change, and I don’t believe that it is important (or matters) to know if the leader wants to change or not. What if he/she doesn’t want to change? Do they get a pass? Some practitioners would probably say, yes, and point to this study as an indication that it is not worth the effort to try to get them to change.
I suggest that this list of factors that lead to behavior change are not independent of each other. In our profession, we speak of “covariates”, i.e., things that are likely to occur together across a population. A simple example is gender and weight, where men are, on average, heavier than women. But we don’t conclude that men as a gender manage their weight less well than women, it’s due to being taller (and other factors, like bone structure).
My daughter, Anne, mentioned in passing an article she read about people who don’t brush their teeth twice a day having a shorter life expectancy than those who do. So the obvious conclusion is that brushing teeth more often will make us live longer. There is certainly some benefit to regularly brushing teeth, but it’s more likely that there are covariates of behavior for people that don’t have good dental hygiene that have a more direct impact on health. While I don’t have data to support it, it seems likely that people who don’t brush regularly also don’t go to the dentist regularly for starters. It seems reasonable to surmise that, on average, those same people don’t go to their doctor for a regular checkup.
My hypothesis is that 360 participants who aren’t open to feedback, don’t perceive a need to change, don’t feel that they can change, etc., are also the people who are less likely to set goals and take action (follow up) if given the option to not do those things. In other words, it’s not necessarily their attitudes that “cause” lack of behavior change, but the lower likelihood that they will do what is necessary, i.e., set goals and follow through, in order to be perceived as changing their behavior. Those “behaviors” can be modified/changed while their attitudes are likely to be less modifiable, at least until they have had a positive experience with change and its benefits.
One last point of view about “beneficial.” Another definition could be change that helps the entire organization. That is the focus of the recent publication by Dale Rose and myself, where (in answer to Joan’s question) we state:
“…four characteristics of a 360 process that are required to successfully create organization
change, (1) relevant content, (2) credible data, (3) accountability, and (4) census participation…”
We go on to offer the existing research that supports that position, and the wish list for future research. One way of looking at this view of what is “beneficial” is to extrapolate what works for the individual and apply it across the organization (which is where the census (i.e., whole population) part comes into play.)
I will stop there, and then also post this on LinkedIn to see if we can get some other perspectives.
©2011 David W. Bracken