Qualitative metrics and how they can work – an unusual example and one closer to home

As I said in a previous post, I was listening to “Decisive” on my way to work last week and thinking about a question that one of my classmates presented, but I don’t believe we discussed in class.  The question was about developing qualitative metrics.  The unusual example was in the audiobook and the one closer to home is, of course, one that we use at Vermont Yankee.

The unusual one actually seems like a wacked out antic from a diva rock star.  When David Lee Roth was with Van Halen, there was an unusual demand to have M&M’s backstage.  However, all the brown M&M’s had to be removed.  This was actually written into their contracts.  What they also had written into their contracts was that, if any brown M&M’s were found, the band could refuse to play and still get paid in full.  Sounds pretty crazy, right?  It was actually a very specific qualitative metric.

Van Halen had a very elaborate stage show which required a lot more specific safety and other measures to be followed, so their contracts were much larger than most other bands.  The M&M clause was inserted to ensure that the venue had read and understood everything in the contract.  If any brown M&M’s were found, David would freak out and demand a full quality check of every other detail in the contract.  If there were no brown M&M’s, David could safely assume that the venue had lived up to the contact and he could just perform.  Although they had the right to refuse to play with full pay, doing so would alienate their fans, so I don’t think they ever did.  However, David was accused of being a diva as a result of this unusual qualitative metric.

The one closer to home is for Design Engineering’s specific product, Engineering Changes, or EC’s.  During implementation of EC’s, there are sometimes changes required to the EC.  These are called ECN’s, or Engineering Change Notices.  While we do keep a metric on the number of ECN’s, we also code each ECN with a reason for change code.  If the change is required because of a parts shortage, implementation preference, or unforeseen interference in areas of the plant where we don’t have access to during normal operation, then the ECN doesn’t really affect our performance.  However, if the ECN is needed because a design error or mistake was made in the original EC, that does affect our performance.  This qualitative metric then feeds into another quantitative metric of percentage of design error ECN’s per ECN.