Iterative Delivery, Waterfall Governance

As agile methods become widespread in organizations, the debate over serial, waterfall life cycles versus iterative life cycles is moving from an engineering-level to an executive-level discussion. In terms of project governance, executives are interested in two things—investment and risk. They have to answer two basic questions: What is the projected value or return on investment? What is the probability that this return can be achieved? Investment decisions are linear: Spend some money, receive some results, decide on the next investment increment. Dollars and time are not iterative: When they are spent, they are gone.

Operational delivery is about defining the best methods of delivering project results. Engineering, whatever the product, is inherently iterative— think a little, try an experiment, observe the results, revise. Sometimes the iterations are long, sometimes short, but engineers have never really operated on a linear, waterfall model—unless forced to by an organization process. When development was sequential, the need to differentiate between governance and operations was masked. However, as organizations began to implement iterative methods, the disconnect between governance and operations began to cause friction between executives and project teams. The critical issue for organizations, then, is bridging this seeming gap between linear investment decisions and iterative/agile product development. The solution is separating governance from operations, as shown in the figure, and then loosely coupling them—abandoning the tight coupling that led to the trouble in the first place.

Product development balances opportunity and risk—the opportunity to generate significant ROI, balanced with the probability that something will intervene to undermine the opportunity. That a huge percentage of new product development (NPD) projects fail speaks to its difficulty. Executives use various data to make project funding decisions. Some known information can be assembled into planning documents, but the crux of product development rests on discovering the unknowns, the solutions to problems that have yet to be identified.

For an uncertain project, executives might authorize $100,000 expenditure for a concept phase to gather enough information to reduce the risk of making a five million dollar full product development decision. Executive decision making follows this scenario through the various phases—define an information gathering strategy based on key risks areas, then decide which activities to fund based on mitigating those risks as early and with as little expenditure as possible. From an executive perspective this model is serial—spend money and time, obtain information, decide on continuing the project.

However, from an engineering perspective, a linear model doesn’t serve well. In the example, a serial model would have focused initially on specifying the requirements for the entire product in detail. At the end of this requirements phase (at which point possibly 20-25% of the cost would be expended), the executives would have relatively complete (in theory) information on the product’s requirements, but no information on critical design issues. After expending $1 million, they wouldn’t have reduced the key risk. The team that actually did some design work, constructed a prototype, and tested it might find the key problem was in a component that they hadn’t suspected. In a serial project, the team might not have discovered this critical issue until 80% or 90% of the project had been completed and would probably cause significant delays. By developing an iterative prototype, the problem might be uncovered 10% of the way through the project for $100,000 and not caused a project delay.

The funding model for projects should focus on what executives need to carry out their oversight and fiduciary responsibilities. They need a systematic way to view information gathered at key intervals to make the best investment decisions based on their understanding of the risks involved. The delivery model should be the best way of generating that information. In the past both models have been waterfall, but a better solution is to use an iterative approach to generate the investment and risk data that executives then use on a regular basis to make project continuation decisions.

(For additional information on this topic, see Chapter 12 of Agile Project Management.)

    Comments

    1. Great article. Brings to light some of the fundamental issues we face, particularly in digital agency production where clients demand fixed requirements, fixed fee upfront. Estimation periods are short and we’re working with an inherently iterative software development process.