Skip to content

The hidden problems with waterfall

There have been numerous posts recently about whether or not to waterfall. For many, the issue boils down to which approach to use under various conditions of uncertainty. While uncertainty is certainly one consideration, the more serious underlying problems with waterfall are its impact on organizational structure and communications.

I received a phone inquiry in the early 90s.

“We need a requirements definition class,” said the development manager.

Rather than just recite what was included in my class, I asked, “Why do you need the class?”

“Well,” said the manager. “We want to outsource development to India and need to make sure the specs are complete.”

“How complete are your specs today?” I asked. “And, how complete do you think they should be to outsource programming?”

“I would estimate we are about 50% complete today and I would like to get over 80%.”

“Well, a couple of industry studies and my observations would indicate your percentages are significantly overestimated. One final question, “How long does it take for new hires in your firm to learn enough about your environment to be productive?”

“Our environment is pretty complex, so about six months,” was the reply.

“If the accuracy and completeness of specifications are in the 20-25% range, at best, and it takes six months when the people are sitting in your office to learn the context for a set of specifications, then how will you convey this context information to the staff in India and answer the inevitable list of questions?”

He mused, “I hadn’t considered those issues.” We hung up and I never heard back. I just knew producing a better requirements document wasn’t going to solve his problem. This and other such incidents influenced my thinking about functional silos, the effectiveness of documentation, and the need for collaboration.

A 1970 paper by Winston Royce was credited with starting the “waterfall” trend. In fact, Royce was an advocate of iterative development for anything more than simple projects. Royce recommended a type of iterative model but given the serial “hardware” mindset of the time, it’s easy to see how his serial waterfall diagram took hold even though he explained its dangers in the text.

Since waterfall-like stages were becoming widely used in management practices at the time, their growing use in software development wasn’t surprising. It’s also interesting that Larry Constantine and Ed Yourdon considered waterfall “training wheels” for budding software developers and not for use in serious projects.

Waterfall thinking extended far beyond software, making the transition to iterative methods difficult. The Figure shows the software waterfall lifecycle and organization structure diagrams. The waterfall lifecycle was greatly influenced by the surrounding ethos of the times—functional organization hierarchies and serial approaches to project management. Waterfall fit right in. IT was organized into functional groups aligned to the waterfall structure—requirements groups, design groups, programming groups, and more. Over time these groups became isolated from each other groups, each working well within their designated group but erecting barriers between them. Analysts feuded with designers, programmers feuded with testers, everyone outside IT feuded with IT.

A waterfall assumption was that documentation was sufficient to communicate between groups. The belief was that documentation, or drawings, could be complete and accurate with no need for further explanation.

So, if you have a project with low uncertainty, consider using a serial lifecycle, but be sure to use it with a collaborative, cross-functional team!

** This post is an edited excerpt from my latest book, “Wild West to Agile” (linkage in comments).

© 2023 by Jim Highsmith