Mar 8, 2007

How are stories of successful companies leading us astray?

"The Halo Effect…and the eight other business delusions that deceive managers," by Phil Rosenzweig, is something you really have to read.

The most popular and inspiring books on business success, like “Good to Great,” “Authentic Leadership,” and “In Search of Excellence,” are based on faulty research and conclusions, says Rosenzweig, formerly on the faculty of Harvard Business School and now a professor at IMD in Lausanne. He says the flawed thinking behind such books leads to delusions that won’t help managers succeed.

Rosenzweig says such books, though they stand on mountains of research, are based on biased data and unscientific analysis. They perpetuate the idea, according to Rosenzweig, that just a few factors, usually vague things like “culture” and “leadership,” will give you the keys to business success.

You probably remember a couple of rules that these best sellers can violate: correlation does not equal causation, a inspecting a “sample” consisting only of good parts will not yield good information about a process, and anecdotal evidence is interesting but not acceptable as proof of a hypothesis. Yet even people with science-based engineering backgrounds fall for the arguments in these books.

So what are the eight delusions?

1. The halo effect: When we choose to study companies based on performance – the “best” companies, the factors we think contribute to their success are usually things we simply attribute to them. The business press loves success and publishers jump at the chance to publish more books about companies recognized as “good” ones. But all this means is that we’ve selected only good parts to inspect, not a representative cross-section of all parts. The same companies can go downhill the next year, and the business press will reverse everything they said. The company praised for confidence is now criticized for complacency.

2. The delusion of correlation and causality: Whatever we ask employees or business analysts, the answers are likely to be colored by how well the company is performing at the time. People are likely to feel more satisfied or empowered, or more admiring of leaders, in a company that is doing well than one that is not. So is employee satisfaction a key to company success, or an effect of it? You can’t say.

3. The delusion of the single explanation: Looking at several hypothesized causes of good performance and singling out the one that appears most highly correlated with performance metrics is not going to reveal the meaning of the universe. For one thing, it confuses correlation with causation. For another, what about the factors you didn’t look at? And what about the companies that would score high on that metric, but were performing poorly at the time and weren’t included in the study?

4. The delusion of connecting the winning dots: If we compare all the winning companies, we can connect whatever dots we want but we won’t get an accurate picture unless we compare them to losers. When you search for stars, you find halos – the metaphor is a bit mixed, but the message is clear.

5. The delusion of rigorous research: Some authors describe the sheer volume of research material they gathered and we believe that since they worked so hard, they must have the right answers. But if hundreds of books and thousands of articles are all tainted by the halo effects they are reflecting, what happens to our data? We have quantity but not quality.

6. The delusion of lasting success: You’d think some of these flaws would be overcome by studying companies with good performance for many years, as “Build to Last” tried to do. How many companies on the S&P 500 in 1957 were still on it in 1997? Try 74. How many outperformed the S&P 500 over that period of time? Only 12. So does looking at a biased sample of 12 tell us something about the other 426? It might, but how much of the performance of those 12 of the 500 might have been caused by random factors that affected the whole market? Are we comparing winners and losers with enough statistical validity, or seeing what we want to see in the winners?

7. The delusion of absolute performance: A company may have high inventory turns, rising revenues and growing market share. That should qualify it as a winner. But if another company improves faster, it won’t matter. Performance is relative, not absolute. By looking at its own absolute performance, a company can think it’s winning but won’t see the competitor gaining advantage.

8. The delusion of the wrong end of the stick: In “Good to Great,” Collins picks a stick with “focus” on one end and “pursuing many directions” on the other. Based on the companies he studies, he concludes that focus is always the right end of the stick. Yet he fails to discuss the risks inherent with focus, such as not seeing that you’re on a dead-end road and that you need to take a risk on another direction.

9. The delusion of organizational physics: social sciences have long argued that they are just as valid as the physical sciences, and will soon figure out the immutable laws of human organizational behavior. Collins claims to be on the track. But organizational behavior is an experiment without a control. You can’t take 100 identical organizations and manage 50 with version A of a single factor and version B of another. In the real world you have an incalculable number of variables, all interacting with each other. If there are laws, it would be surprising if we were anywhere near understanding them.

Throughout the book, Rosenzweig provides data from studies that contradict the most popular, and goes to great lengths to show the flaws in that research as well. He shows some ways that meaningful research could be conducted, even if it hasn’t been done before – graduate students take note.

“The Halo Effect” is a warning to companies that think “lean” is going to cure all their ills, embark on a few superficial improvement initiatives, and get demoralized when it doesn’t turn them into overnight successes. The factors in implementing lean are subtle, interactive, and cumulative. It takes finesse to apply them in the right order to the right things. A company can be doing all the “right things” and turn out like Delphi.

Are the many case studies of success without merit? It depends on how you read them. If you’re looking for the “immutable laws,” you’ll be falling into the eight delusions. Still, success stories give people hope, a big ingredient in the struggle for change.

The book has a nice numbered list of “delusions,” which makes it easier to grasp them and keep them in mind. Despite the emphasis on the scientific method and statistical rigor, the writing is lively and conversational. It’s a compelling and quick read.

However, by picking a simple list of eight factors and attacking just a few books, Rosenzweig might be falling into his own trap. It’s a short book, so it can’t dig deep into the subject matter. It’s up to readers to apply Rosenzweig’s challenge to business writing and draw their own conclusions.

Read "The Halo Effect" straight through, then reread the list of delusions and reflect on how your reading and exposure to trainers and conference speakers may have planted some questionable ideas in your mind. Ask yourself questions about what you believe about business success. It doesn’t mean you should give up on making your company better, just don’t look for quick fixes and keep in mind that business is about probabilities, making choices, and improving your odds.


Mike Gardner said...

I suspect the uber-gurus such as Jim Collins and Tom Peters will simply brush this book off as unworthy of a response. What would be really nice is to see a real debate between the differing parties, including only those who have some level of credibility, not the average fluff book author. That probably won't happen, though.

curiouscat said...

Good post. There is a good article talking about how to improve management "theories" by Paul Carlie and Clayton Christensen. They discuss descriptive “theory” versus normative theory: The Cycles of Theory Building in Management Research.

My simple explanation is that descriptive theory reports on what is seen. This can be interesting, but has problems when people assign causation based on just observation (without experimentation). Normative theory involves testing theories (such as is done with the scientific method).

More: thoughts on Carlie and Christensen's paper - Seeing Patterns Where None Exists

Karen Wilhelm said...

Good points guys. I think Mike is right on the money with the idea of a debate. We could call it business guru showdown. Something for the AME 2008 conference in Toronto?

Thanks for the tips on more reading, Cat. I have a couple of other contrarian writings to bring up in future posts.

Stephanie West Allen said...

I think this review of THE HALO EFFECT is very good, too:

Here's my review of the February HBR article on authentic leaderhip:

Ron Pereira said...

Hi Karen, great job with this blog! I have added a link to your blog from mine. Keep up the great work!

Ron Pereira

Copyright @ 2005-2014 by Karen Wilhelm