October 18, 2018
It has been common for years to use project success measures based on the triple constraint. Reporting on whether the project is on time, on budget and on scope (OTOBOS) has strong appeal and most organizations still use this approach. But these project success measures are out of date and fail to report on what is really important—whether or not the project is delivering the desired business outcomes.
I first learned of the triple constraint back in my first PM class back in the early 1990s. I learned that the three constraints of time, scope and budget formed the basis of every project. And I leaned that if any single constraint changed, the other two had to be evaluated for impact.
Though it was simple and perhaps convenient to consider only those three constraints, most people today recognize that there are more than three constraints in a project (resources, quality and risks are also frequently mentioned). Unfortunately, most people still use just cost, time and scope as a project success measure or when reporting project status. People have limited their definition of success or failure to whether the triple constraint was managed. And this misses the entire point – OTOBOS is not the goal of the project.
With all the focus on OTOBOS, many people think that it is the goal of the project. Many project managers certainly believe that is their goal to deliver the project on time, on budget and on scope. They have lost sight of the real project goal—and that is delivering some sort of business value. That should be our first goal, and not an afterthought.
PMI seems to be slowly addressing this. A Guide to the Project Management Body of Knowledge (PMBOK® Guide) has included guidance on success and benefits management measures. In my experience, these measures are rarely put in place. Very few people are actually keeping an eye on the achievement of benefits—it is a second-class citizen compared to OTOBOS.
The most recent PMI Pulse of the Profession acknowledged that only now is benefit realization being included in success measures. For the first time in 2017, PMI included benefits realization maturity along with the traditional project success measures in Pulse of the Profession[i].
Perhaps measuring benefits as a component of success isn’t more common because the realization of project benefits often lags the completion of the project. Unless the PMO or other organization is measuring it, it may not happen since teams are quickly disbanded and project managers are typically reassigned.
That is a big problem—the separation in time of the execution of the project and the realization of the benefits. The longer the timeframe, the higher likelihood that people will lose interest. The original assumptions may be forgotten. Or, so many other things changed in the environment that the original benefits can’t be accurately measured.
Rarely if ever does anyone go back after the end of the project to see if the promised business benefits were achieved. In fact, I’ve seen large cost-reduction projects that were supposed to result in headcount reductions that resulted in more people afterward!
Note: It is also a problem that the people tracking the realization of benefits (the PMO) are separate from those responsible for delivering it (the PM and project team).
There are situations where our project delivery measures like OTOBOS are exactly opposite the business values measures. Consider the following two examples from Jim Highsmith’s book, Agile Project Management[ii]:
So, it is possible to have projects that are successfully delivered and fail to deliver benefits—and projects that fail to be successfully delivered and yet yield business benefits. What is success then?
Similarly, some organizations measure “scope creep” and view that as a negative. But if a change in scope serves to improve the benefits of the project, shouldn’t that scope change be valued? Wouldn’t it be more important to change our plan to achieve our benefits, rather than simply adhere to the plan?
The Iridium problem of faulty assumptions is one that should have been uncovered early in the project. That particular project was planned in detail over the course of several years! There was plenty of time to validate the assumptions about the business value.
The problem was that the team believed that if you built it, they would come. This Field of Dreams thinking is problematic. We cannot assume that just because we deliver a project that it will result in business benefits. We should not trust our assumptions about business values—they are frequently wrong. And focusing on OTOBOS is a distraction from what is really important.
In 1961, President Kennedy proposed to the Congress that the United States should put a man on the moon before the end of the decade. The unstated reason was dominance in science and technology and a response to a threat from others. The Soviet Union had successfully launched a man into space just weeks prior.
This was a massive undertaking (what some might call a “Big Hairy Audacious Goal”). It took nearly a decade to accomplish it. There were lots of assumptions about the feasibility of this project. The term moonshot was coined from this mission.
Most organizations today try to avoid moonshot projects. While big goals are important, few want to bet the company on a make-or-break project. Betting the company on a moonshot project like Iridium can result in the death of the company.
Instead, organizations take calculated risks. They make assumptions about the benefits that they can get by implementing a project. And they continually test those assumptions.
The 2005 book The Lean Startup describes this process of challenging and validating assumptions. In the Lean Startup Method, there is little or no focus on traditional project measures like OTOBOS. Instead, there is a continuous focus on learning and on validating the assumptions about business benefits—and continuous monitoring of those business benefits. If the business goal is to increase customer revenue, the organization continuously measures customer revenue and tests different actions to determine what it takes to get that revenue.
Similarly, agile projects focus on delivery of business benefits. They leverage incremental and iterative development based on a “product backlog” of business-related features. Every product backlog item should have business value. Agile projects follow a “build, measure and learn” cycle to constantly inspect and adapt. For those projects where the outcome is fuzzy or the path to get there is risky or uncertain, it is important to have many of those build-measure-learn cycles.
Using traditional OTOBOS measures with agile projects is ineffective and wasteful. Measuring variance from the plan is a distraction, and trying to minimize variance may be futile. Rather, the focus should be on measuring the achievement of business value. This is often referred to as value-driven development.
Given the above, what should project managers do—especially when organizations seem to be rigid about reporting projects using OTOBOS measures? I’ve outlined five recommendations that I believe may be helpful:
[i] Project Management Institute, Pulse of the Profession (2017), Project Management Institute, downloaded from: https://www.pmi.org/learning/thought-leadership/pulse/pulse-of-the-profession-2017
[ii] Jim Highsmith, Agile Project Management (2009),
[iii] Jim Johnson, Chaos Collection 206, The Standish Group International, Incorporated