The most recent Standish Group Chaos Study results show Waterfall and Agile project success and failure rates. It should come as no surprise that Agile projects are statistically 2X more likely to succeed, and 1/3 less likely to fail than waterfall projects. Read on for the most current statistics from the Standish Group Chaos Studies which back this up.
I’ve been a follower of the Standish Group Chaos Studies for a long time. The Standish Group has conducted surveys of IT project success and failure rates every 2 years since 1994. The data included here is from the most recent report published in 2018.
Initially, the success and failure statistics were pretty poor with IT project success rates measured at less than 20%. Thankfully things have improved somewhat. The most recent report (2018) from the Standish Group showing technology project success rates at 36% with agile projects succeeding more frequently. I believe that it is agile project success that is lifting all IT project success.
What is the Standish Group definition of agile project success and failure?
Initially, the Standish Group definition of project success was limited to the triple constraint, which has been the standard for the Project Management Institute for a number of years. Using the triple constraint, the Standish Group evaluated projects as successful, challenged or failed.
Successful – A successful project was one that met all three of the triple constraints: schedule, cost, and scope.
Challenged – A challenged project would have met two out of three constraints, for example, delivered on time and on budget but not with the desired scope.
Failed – A failed project is one that is canceled before it is completed, or completed but not used.
Challenges to the Standish Group definitions
The Standish Group took some heat over the years for their strict definition of success, challenged and failure. The triple constraint measures were really a measure of the project management ability to: 1) estimate and plan the project and 2) deliver according to the original plan. In addition to the obvious difficulty of delivering to the original plan, this success measure ignores whether or not the project delivered anything of value to the customer. Delivering a project perfectly on time and on budget without achieving the business objectives would be a huge waste and misses the point entirely!
So the Standish Group enhanced their measures of success to include the delivery of customer value, alignment to strategic goals and customer satisfaction. You can read more about the changes here.
While we can squabble about the specific definitions of success and failure, it is clear that even groups like the Project Management Institute recognize that failure rates are too high. In the 2020 PMI Pulse of the Profession, PMI survey respondents estimated that only 64% of the projects met their goals and a full 29% of all projects could be deemed failures.
What is interesting to me and perhaps not very surprising is the difference between success and failure rates between Agile projects and more traditional Waterfall projects. Let’s explore this in more detail.
Comparing Waterfall and Agile Project Success Rates
The most recent report from the Standish Group covered projects they studied between 2013 to 2017. For this time period, the overall breakout of success, challenged and failure is shown below for agile and waterfall, with Agile projects being roughly 2X more likely to succeed, and 1/3 less likely to fail.
From the Standish Group 2018 Chaos Report:
The results for all projects show that agile projects enjoy a 60% greater chance of success than non-agile projects. Looking deeper, we find that “waterfall” projects are three times more likely to fail than agile projects.
— Standish Group CHAOS Report Series, Decision Latency Theory
Project Size Also Affects Project Success Rates
Another key finding of the Standish Group is that larger projects have higher failure rates. This should not come as a surprise to anyone. What was surprising to me was the extent to which the smaller projects reduced the risk (see project failure rates chart below). The key takeaway here is that smaller projects will maximize agile project success.
Looking at the chart, you can see that Agile projects have a significant edge over Waterfall Projects for every project size. Of the two project factors, size and approach, the Standish Group says that size has more of an impact on project failure rates than agility. Both together have the greatest impact.
From the 2018 Chaos Report:
When we break “agile versus non-agile” by size, we find some really interesting data. Large agile projects succeed at twice the rate of non-agile projects, and fail half as often. “Medium-agile” projects do not fare that much better (31% versus only 19% for non-agile projects). Only in the small category does non-agile come close to agile. Therefore, we conclude that size trumps agile methodology—but in combination, they can be deadly.
— Standish Group CHAOS Report Series, Decision Latency Theory
Takeaways from the Agile Project Success Rates
- Agile Project Success Rates are Higher. While Agile approaches are not necessarily a silver bullet, the data shows they can help to reduce project risk. The Standish Group data shows that Agile projects success rates are roughly double those of traditional projects. And agile projects 1/3 less likely to fail than waterfall projects. In my opinion, the primary reasons for this is the amount of user collaboration on Agile projects which helps to ensure the team is building the right solution and incorporating feedback. Agile teams also develop in short iterations and take items all the way to done within a sprint to shorten feedback loops and further reduce risk.
- Smaller projects succeed more often than big projects. Duh! Even though this may be common sense, many people ignore it. Organizations that take the time to break big initiatives down into manageable chunks find that they can better manage the schedule and risk, and often deliver something of value with each small chunk. Some organizations deliver a Minimum Viable Product and use that to get feedback or validate their assumptions about the end users or customers. Read more in my book, Agile Project Management.
3. All Projects Are Getting More Successful. When I first published this article back in 2016, I used the Standish Group statistics from 2011 to 2015. The most recent article from the Standish Group provides project data from 2013 to 2017. In almost all cases, the data shows improvements in both traditional and agile project success rate. There are also correspondingly declines in project failure rates. The chart below shows success and failure rates for the previous period with data from 2011 to 2015.
- There is still room for improvement. There are still a lot of projects that are not considered “successful”, both Agile and Waterfall. And there are still a lot of projects that fail, which represents a tremendous cost to organizations. It also has a huge personal cost to the careers of the project managers and leaders who are responsible for these projects. I think organizations need to learn from the past and leverage the findings from the Standish Group. We can continue to use agile methods and break big projects down into smaller ones in order to continue to increase success and reduce failure.
How to Transition from Waterfall to Agile
Making the transition from Waterfall to Agile is not trivial, but it is certainly possible. Many organizations have tried and failed, while others have succeeded with Agile approaches. Here are some resources that you might find helpful:
How to Successfully Transition from Waterfall to Agile and Scrum – This article provides an overview of the steps you need to consider when moving to Agile and Scrum which can help you create your own agile transformation roadmap. It also includes links to free resources like planning checklists and a detailed whitepaper on how to successfully transition to Scrum.
If you are wondering if your organization is too big or too old for Agile, read this Case Study about how Bank of America successfully made the transition to Agile and Scrum. Bank of America is both big and old and if they can make the transition to Agile, so can you.
The Leaders Role in an Agile Transformation – Agile leaders play the most important part of the Agile Transformation. Learn more about how to cast a vision for an Agile Transformation and lead through change. You can also visit our ultimate How To Guide for Agile Success which is aimed to give you everything you need to succeed with Agile and Scrum.
Agile and Scrum Related Training Courses
Training for Agile and Scrum – Our Agile Training page lists the courses we offer to help everyone in the organization with the agile learning they need to make the transition to Agile and Scrum.
We partner with Northwestern University to offer the Agile for Practitioner Training for those project managers and other practitioners who want an in-depth understanding of multiple agile frameworks and processes. This course includes the content the PMI Agile Certified Practitioner exam and may be used to prepare you to get your PMI-ACP certification.
Our most popular course is Agile and Scrum for Teams. This course is designed to be delivered to new Scrum teams to provide them both the theory and the hands-on experience that they need to hit the ground running and succeed with Scrum.
Finally, we provide what we believe to be the best Agile Training for Leaders on the market. Our Agile for Leaders course provides both a high-level overview of various agile frameworks like Scrum and Kanban, as well as an in-depth understanding of the cultural change and mindset needed to support successful Agile adoption in organizations.
You can read more about the Standish Group Chaos studies here.