One of the big problems with successfully executing projects is that while we know projects are very different from each other, we often manage them and measure their success in the same way. Think for a moment about the oft quoted Standish reports in which project success is measured on the traditional iron triangle basis of meeting scope, schedule, and cost plans. In their scheme, all projects, of any type, are successful or not based on the same criteria.
A friend of mine worked on a project recently where the client said, “I have a fuzzy vision of what we want. I don’t have any idea what the detail requirements should be. I need results fast.” The client even refused to participate in a story identification process, leaving that up to the development team to experiment. Managing this project against traditional measures of scope, schedule, and cost would damage any chance of success. This is a different kind of project. The team evolved the product from a vision and evolutionary learning and the client was very pleased with the results, because he understood it was a different type of project.
In Agile Project Management I wrote about assigning an Exploration Factor (EF) to each project (or release of a product). The EF attempts to identify a level of uncertainty and risk for a project by looking at both technology and requirements. Technology can run the gamut from “well-known” to “bleeding edge” and requirements from “stable” to “erratic.” Combining the two factors yields EFs (see figure) of from 1 (very low uncertainty) to 10 (very high uncertainty and risk). EF 1 and EF 10 projects are very different—they need to be managed differently and they need to be assessed differently. For example, if the requirements are uncertain, measuring progress against a scope plan is ridiculous. This doesn’t mean the project is uncontrollable, just that we have to control it differently.
So, before we think about how to measure success on projects with high EFs, we need to understand what makes these projects successful. The pressures on high EF projects are usually uncertainty (about either requirements, technology, or both) and speed. Typically high EF projects are also strategic, customer facing, web or mobile, etc.—all aspects that lead to uncertainty. Sponsors want them done quickly. So how are projects like this managed successfully? By using a combination of a well-articulated vision, quick iterations, evolving functionality to a minimal viable product and beyond, being adaptable to learning from customers as the product evolves, focusing on value delivery, and time boxing.
One important aspect of controlling high EF projects is to view scope, schedule, and cost as constraints—not plans. Because the project requirements are not known it is often difficult to estimate time. However, given the lack of specificity the team needs constraints, for example a time box of 3 months that limits exposure. At the end of this timeframe the project sponsor can evaluate the value delivered, the questions answered, and then decide to invest in the next increment or not. The question to be asked at every iteration and release is, “What is keeping us from deploying this product now?” This is very different from the normal progress question of “Have we developed all the planned scope yet?”
Conversely, a very low EF project, say a 1 or 2, could be managed with scope questions because the requirements are knowable in the beginning (or at least people think they are even though we know they are often wrong). I might add that software development projects are rarely low EF projects.
The bottom line is:
- All projects are not the same.
- Projects with different Exploration Factors should be managed differently.
- Performance measures should be different for high EF projects than for lower ones.