Project Managers have been taught predictive planning approaches for many decades now. However, the genesis of the approach has a rather interesting past. There are two somewhat seminal events that led to this and should also lead any PM to question the premise of predictive planning. One is PERT and the other is DoD-Std-2167 from 1985 which promulgated the Waterfall approach.

An excellent book to find out more about these two events is The Heretic’s Guide to Best Practice by Kailash Awati and Paul Culmsee.

In Chapter 3 Awati and Culmsee explain that PERT was actually an after-the-fact public relations tool, as opposed to being the predictive planning tool described in project management books. PERT was an invention of the Polaris submarine project in the late 1950’s that showed a revisionist (and apparently compelling) history of their project. It was created to paint a rosy picture of what the team had supposedly accomplished thus far so that they could continue to get money from the US Congress.

Indeed, PERT worked so well as a management abeyance practice that the Royal Navy employed it with equal success at securing additional funding for their projects. PERT is still a core topic in most of the traditional project management books and training – too bad it’s an illusion.

The second event was the release of DoD-Std-2167 in 1985 which launched waterfall to stardom – of course PERT fits quite nicely with the illusion of predictive planning that waterfall would have us believe. It was a match made in an illusionary predictive heaven. 2167 was based on Winston Royce`s 1970 paper ‘Managing the Development of Large Complex Systems’ in which he was actually promoting an incremental approach) as he felt that waterfall “was risky and invites failure’.

2167’s authors seemingly never read beyond the top of page 2 which depicted the now infamous waterfall diagram.

Awati and Culmsee tell the story of how, in 1996, the lead author for 2167 essentially admitted that had he known about incremental development, he would never have promoted waterfall. So in essence he was saying, “ah, big oops, sorry about that folks!”

Business analysts should also read The Heretic’s Guide to Best Practice as they get special mention. The BABOK is based primarily on waterfall, notwithstanding their recent Agile extension. The scary part? The BABOK came into existence about a decade (2005) after the 2167 lead author had already labelled it as a mistake in 1996! Ouch…

Just how far back does the recognition that waterfall was not a good idea go? In the History of Iterative and Incremental Development Craig Larman and Vic Basili cite the recollections of Gerald M. Weinberg, who worked on Project Mercury in the 1950’s:

We were doing incremental development as early as 1957, in Los Angeles, under the direction of Bernie Dimsdale [at IBM’s Service Bureau Corporation]. He was a colleague of John von Neumann, so perhaps he learned it there, or assumed it as totally natural. I do remember Herb Jacobs (primarily, though we all participated) developing a large simulation for Motorola, where the technique used was, as far as I can tell, indistinguishable from XP.

When much of the same team was reassembled in Washington, DC in 1958 to develop Project Mercury, we had our own machine and the new Share Operating System, whose symbolic modification and assembly allowed us to build the system incrementally, which we did, with great success. Project Mercury was the seed bed out of which grew the IBM Federal Systems Division. Thus, that division started with a history and tradition of incremental development.

All of us, as far as I can remember, thought waterfalling of a huge project was rather stupid, or at least ignorant of the realities. I think what the waterfall description did for us was make us realize that we were doing something else, something unnamed except for software development.”

Royce’s son on his father and the incremental approach (cited by both Larman and Basili and Awati and Culmsee):

“He was always a proponent of iterative, incremental, evolutionary development. His paper described the waterfall as the simplest description, but that it would not work for all but the most straightforward projects. The rest of his paper describes [iterative practices] within the context of the 60s/70s government contracting models (a serious set of constraints).”

According to Larman and Basili

“This was an ironic insight, given the influence this paper had as part of the bulwark promoting a strict sequential life cycle for large, complex projects.”

Another phrase I came across that represents the insanity of continuing to believe in predictive planning and waterfall is called “consensual hallucination” which is premised on the “almost universal belief that it is possible to predict ahead of time how long a project will take, how much it will cost and what will happen along the way” – down to the day and the dollar.

PM’s, teams, and the executives who prefer predictive planning and estimates to satisfy their tic-and-bop decision-making models over dealing in reality need to stop hallucinating and believing in illusions.

NOTE: The above is an excerpt from an upcoming book by myself and Jen Stone called “Agile Value Delivery: Beyond the Numbers