Mar 25, 2011

Lean assessment basics

I spent a “Learning Friday” morning with some colleagues from the Michigan Lean Consortium at a session about lean assessments led by Debra Setman of Johnson & Johnson. Debra’s been involved lean assessments for 20 years at hospitals, manufacturing companies, IT organizations and J&J, and had a brisk, interactive, and fact-based introduction to them.

Debra says she doesn’t even like to call these 2 ½ day visits “audits” or “assessments.” They are coaching opportunities for the visiting team and learning experiences for the site being visited. The key is to have valid and consistent criteria that can be used to characterize a level of lean … what should we call it? Implementation? thinking? … for the purpose of sharing recommendations and recognizing strengths.

J&J uses a maturity model, starting with Stability, Flow, Pull, and Integration stages (as defined by J&J) of a number of categories of lean practice and thinking. For “Site Leadership Attitude,” for example, the evidence of a site’s stage of maturity is:

Stability: Excited and proactive about lean in the lean pilot area (LPA)
Flow: Excited and proactive about lean throughout the site value stream
Pull: Excited and proactive about lean throughout the value stream and at the key external partners
Integration: Excited and proactively involved with lean throughout the extended value stream. Servant-leader mentality exists throughout.

Debra emphasized that your organization could have different categories and definitions, depending on your company’s goals and understanding of lean. In fact, as requested beforehand, several people brought their own examples of assessments. Mike Taubitz from Lean Journey LLC had a simple Yes/No inventory checklist. Murray Sittsamer of The Luminous Group shared a fairly quick and easy scoresheet on a “worst practice” to “best practice” scale for a number of lean hallmarks.

It’s easy to get the criteria underlying several recognized awards-related assessments:

The Shingo Prize

The Manufacturing Awards of the Association for Manufacturing Excellence

The Baldrige Award is more oriented to classic quality objectives than lean, but many companies like using it for evaluation.  It has versions for education and healthcare.

For assessing your personal lean maturity level, you can review the AME/SME/Shingo Prize Lean Certification criteria that have been adopted by NIST and by many companies. There are a lot of related documents on the SME website: (Go to the two lower boxes in the right-hand column for the links.)

What’s the value for the company of going through an assessment? Michael Bremer asked that question on the AME LinkedIn group -- you can add your thoughts to if you go to “AME just sent out a call for applications for its Manufacturing Excellence Award -- What do you think are some of the benefits a company gets when it applies for the award?

Not giving you the usual lengthy article about the presentation today -- if you want more information contact me at Karen.m.wilhelm …. At …. or dsetman … at ….


Tom Robinson said...

Karen, This is a packed post -- very helpful to all of us trying to learn and improve lean efforts. You walked the talk in providing such a succinct post!

Michel Baudin said...

Dear Karen:

All the checklists I have seen have in common that they are lists of practices with check boxes or scores. You get so many points for having andons, posting work instructions above work stations, or using kanbans.

This treats Lean like regulatory compliance, and rewards plants for looking Lean. It is tantamount to measuring a chess player by pawns moved rather than games won.

If you use a checklist to assess a plant, it should be a checklist of what is wrong with it, based, for example, on Ohno's seven categories of waste. Helping people recognize waste is exactly their purpose.

Eliminating waste in any specific factory does not require implementing all the practices in the assessment checklists, and plants should be praised rather than penalized for focusing on what is relevant to their circumstances.

For the supplier support organizations of major manufacturers, relying on compliance checklists is a way to avoid learning the business and technology specifics of hundreds of suppliers. Unfortunately, it also transforms Lean implementation from an improvement opportunity into a cost of doing business.

Unsurprisingly,as I researched on Shingo Prize winners a few years ago, the companies that do best on these checklists do not perform better than their competitors.

Best regards.
Michel Baudin

Unknown said...

Hi Tom and Michel,

Thanks for reading the blog and commenting on this post. Tom, succinct is not my strength, but I'm working on it.

Michel, your comments are great cautions to the dangers of checklists and boiling down complex thinking to a few points of evaluation. Debra did a great job of recognizing the pitfalls and sharing the importance of using the tool - which is all it is - to find the teachable moment. She also talked about how important the composition of the team is: good communicators, good experience, the right balance of outside vs. inside team members, and good feedback.

Your comment about the relationship of award winning to long term success reminds me of a book that really impressed me a couple of hears ago - the Halo Effect by Phil Rosenzweig (

Your research resonates with his - that a spot measurement of a company is not statistically very good at separating the successful from the average. So the "best practices" that are widely imitated aren't necessarily going to work at your company. Thanks for sharing that insight.

Joe said...

Terrific post, Karen, thank you.

Thinking of Michel's comments, I make one observation. If I am a new golfer, having a list of checkpoints (stance, grip, club selection, etc) will move me ahead faster than just "trying hard" to play golf.

And once I consistently hit the ball straight and reasonably far, I may need more than mere checklists. To a really good coach/trainer with an eye to subtle problems with my swing.

Given that most of us are not as far along the Lean journey as we think we are, the checklists are still useful.

Jamie Flinchbaugh said...

I just completed an assessment at a major automaker in Bavaria. One of the things that I see in assessments is people not understanding the difference between an assessment and an audit. An audit is to see if you're complying to a standard. 100% is the goal (for the most part) and only failures are noted. An assessment, when done well, should help expose the opportunities to move forward and to improve. Even when the recipient of the assessment doesn't understand the purpose, it can still fail to be effective.


Unknown said...

Joe and Jamie,
As always, thanks for reading Lean Reflections and for sharing your counsel. With our social training in feedback and criticism as something to be avoided, we need to constantly help people remember its purpose -- keeping our eyes open to further ways to improve.

Jess said...
This comment has been removed by a blog administrator.
Jaxon Oakley said...

Lean assessment has been the base of the mathematics and all such subjects. The concepts of the lean have been identified for the formation of the right and wonderful abstracts for the individuals.

Copyright @ 2005-2014 by Karen Wilhelm