Hidden: What’s Beyond Measurement

About 550-600 million years ago, just prior to the Internet, the earth experienced the Cambrian explosion in which flora and fauna species proliferated. Wild and weird life forms emerged. Then, with a bang, as happened to the dinosaurs, 96% of all those species vanished. It wasn’t the survival of the fittest, but the survival of the luckiest—says Steven Jay Gould, author of the fascinating book, Wonderful Life: The Burgess Shale and the Nature of History. Conditions changed so radically that species that were “fit” before, were not “fit” anymore. Rewind the tape of history, says Gould, and play it forward time and time again—and life would evolve differently every time.

How many major, and minor events, have shaped your life, your project, your company? A major job change in 1981 started me on the software methodology path. A 5-pound weight swing made the difference in becoming an Air Force officer—or not. A vividly remembered 3 seconds in 1973 was the difference between living and probably dying in a car accident. What is hidden from view in the future and what is garbled about our past remembrances? How much of what we are today resulted from planning, and how much was just the accidental unfolding of our past into our future?

In his fascinating and insightful new book, Creativity, Inc.: Overcoming the Unseen Forces That Stand in the Way of True Inspiration, Ed Catmull, president of Pixar Animation and Disney Animation, devotes an entire chapter to the Hidden and the importance that leaders must place in creating the culture and the practices to uncover the Hidden. The Hidden includes all the things, the reality of the real world, that for whatever reason, we fail to see. None of the flashy Silicon Valley startups Catmull talked about planned to be one trick ponies—but many succumbed to the lack of sustainable innovations because of hidden problems the leaders never saw coming. “I spend a lot of time thinking about the limits of perception. In the management context, it particularly behooves us to ask ourselves constantly: How much are we able to see? And how much is obscured from view? Is there a Cassandra out there we are failing to listen to? In other words, despite our best intentions, are we cursed too?” says Catmull.

About 15 years ago, before the popularity of Agile methods, a major software development magazine published an article about a large aerospace company and how they had managed to be “on-time” and “within-budget” 95% of the time. The article, of course, heralded their accomplishment as a triumph of a standardized, metrics-driven, waterfall methodology.  Who would argue with a 95% success ratio on projects? Well, me of course. My first reaction was that 95% in this instance was a terrible thing, not a positive one. I considered that 1) someone was cooking the books, 2) they were only taking on projects with very little uncertainty, or 3) they were drastically over-estimating the planned numbers.  The problem in any case was that innovation was very unlikely in this organization—no one could afford to make a mistake when management was so enamored of non-failure on projects. In constantly striving to achieve high levels of project success based on limited measurements, management was hiding future problems with innovative new products.

On the other hand, maybe my mental model of the situation was all wrong. Maybe for an aerospace company that depended on government military contracts, innovation wasn’t a significant problem—even in its future. Maybe predictability was key to keeping contracts flowing.

In this brief blog, we’ve now uncovered a range of problems with measurements: 1) picking the specific thing(s) to measure, 2) their accuracy, 3) the mental models we use to interpret them, 4) organizational pressure for the measure to show “success”, 5) the context of the company’s situation, 6) the complexity of the company’s situation. Given all these problems with measurement—do we give up on them? Probably not, but maybe the most important caution—one that Ed Catmull repeats often—is that whatever we choose to measure hides both what we don’t measure and what we can’t measure. Furthermore, a critical role of leaders, and specifically those who want their companies to sustain themselves over time, is to constantly seek out problems hidden behind the measurement veil created by complacency, confidence, and past success.