“The Easier It Is To Quantify, the Less It’s Worth”

This quote from Seth Godin (Linchpin, 2010), sparked my thinking about metrics and outcomes. It parallels what I’ve quipped in presentations, “I’d rather have a fuzzy measure of something important than a precise measure of something unimportant.” In Measuring and Managing Performance in Organizations (1996) Rob Austin discusses the difference between desired outcomes and performance measures—and that those are likely to be in conflict with one another. Because of the difficulties in finding good measures for desired outcomes, we often substitute things that are easy to measure, but that cause long-term damage. Assessing the desired outcome of programmer productivity by measuring lines of code is a classic misuse of a metric.

Historically we assess project success, or failure, by measuring scope, schedule, and cost—especially cost and schedule because they are easy to measure (think earned value analysis). However, the desired outcome, from a customer perspective is to receive a releasable product, of acceptable quality, within a set of constraints (schedule or cost for example). For most projects, rather than asking, “Did we implement all the requirements (a scope question)?” the question should be “Can we release this product now?” Accessing release-ability (a value-oriented question) and quality may be “fuzzier” than measuring cost and schedule, but just maybe they are closer to representing the customer’s true desired outcome.


    1. Jennipher Marie says:

      Great post Jim.

      Realizing this can be slow going for a lot of people (especially when all they have known is the waterfall approach to product development.) As a Product Manager I’ve seen the harm that can be caused by only talking about a date being meet with the predefined feature set. We product managers should be worrying about the quality and value of the product but some times it can be hard when your management doesn’t agree.