Beyond the Hype – Where are the Benefits of Agile?

I was puzzled by the recent announcements of layoffs of the agile delivery teams at Capital One. Capital One was one of the first large banks to fully embrace agile ways of working across technology. What happened? Why did Capital One choose to eliminate those jobs?

I don’t know. I can only speculate that they did not think there was a benefit to be gained from having those agile delivery roles. This makes me wonder if the benefits of agile are living up to the hype of agile. I don’t believe the majority of people who adopt agile ways of working are getting the full benefits.

And I have a recommendation for those that don’t think they are getting the full benefits.

Separating the Hype from the Benefits of Agile

Those of us who have practiced agile long enough have seen the benefits including responsiveness to change, higher productivity, predictability, and quality. It is not only possible, it is likely, and there is plenty of case studies and anecdotal evidence to support that.

But where is the hard data?

I conducted a Google search for studies on agile benefits and was surprised to come up short. Oh there are lots of results that talk about the benefits of agile, but I could not find a single study or research project. The closest I could find were the Standish Group Chaos studies and which focus on project success and failure rates. They are backed up with real data. Unfortunately, they don’t track productivity, predictability, quality or other benefits of agile.

Even the widely cited report from digital.ai, The Annual State of Agile Development contains just a list of benefits reported from those that responded to the survey. This is simply confirmation bias.

Why aren’t we collecting real data about the outcomes and benefits? Are we just putting a marketing spin on it and puffing it up?

Consider Jeff Sutherland’s 2014 book, Scrum The Art of Doing Twice the Work in Half the Time. The promise of Scrum benefits in the title is compelling. [And BTW, I received an early signed copy of Sutherland’s book at Agile 2014, and I thoroughly enjoyed reading the book.]

The problem is, most people using Scrum today are not getting twice the work in half the time. I would argue that for many of them, their productivity actually declined after implementing Scrum as I will explain below.

I don’t fault Sutherland for the hype. I do wonder what stops others from claiming even greater benefits of agile. Why not hype agile even more?

Fans of the 1998 movie There is Something About Mary may recall the scene where they mention the 7-minute abs. Who would want to use 8-minute abs when you could have 7-minute abs?

So what is to stop some creative individuals from claiming that Scrum provides TRIPLE the work in a THIRD of the time? I don’t have plans to do it, but the opportunity is certainly there. And it would hurt the credibility of all of us.

It would be a race to the bottom. And I am surprised it hasn’t already happened.

Who is Not Getting the Benefits of Agile – Double the Work in Half the Time?

As I mentioned above, I don’t think there are many organizations that doubled their work in half the time by adopting Scrum.

What I see today is that organizations don’t fully implement Scrum as designed because of constraints in their organization. They have implemented Scrum while having to contend with other teams using waterfall, or annual budgeting cycles that lock in budgets or a myriad of other constraints. They use what they can and adapt to their actual conditions.

Well-intentioned leaders that implemented Scrum this way may have actually decreased productivity:

1. Those that added Scrum without creating true Scrum Teams will have team members running to multiple standup meetings every day and double or triple end-of-sprint meetings. Others that didn’t bother to remove other non-Scrum meetings simply added the Scrum overhead to their existing overhead. The result of that overhead is lower productivity.
2. Teams that were taught to spend time estimating in story points are burning valuable team time. Story points are still guesses, and the velocity they use to forecast completion is based on their guesses, not data. (Note that estimation and story points, though widely used by Scrum Teams, are not part of the Scrum Framework.)
3. Lacking better measures, teams use velocity as a proxy for value delivered. Since it is easily gamed, increases (or decreases) in velocity are meaningless and a distraction from real productivity measures. Velocity is not productivity.
4. Some teams use sprints as a block of time and don’t focus on producing a shippable increment each sprint. Testing is sometimes intentionally scheduled for the sprint following the development, reflecting a waterfall mindset.
5. Some teams never learn to work together as a team to produce solutions. Each individual on the team focuses on ‘their stories’ and being efficient, without regard for team throughput or effectiveness. They could use cycle time and WIP aging to fine-tune their delivery and maximize their productivity.
6. Some teams don’t leverage the Sprint goal. Instead, they just try to deliver as many backlog items as possible, churning out more outputs without necessarily creating value.

The intent is positive but these teams are NOT going to double their productivity and halve their timeline. 

It would be easy to say to those people – you aren’t doing Scrum correctly so that is on you. That is a common criticism, and it falls short for many managers and leaders in organizations that can’t create the exact conditions to generate the promised benefits.

Instead, we need to help those leaders with the art of the possible.

How to Avoid the Agile Hype AND Get the Benefits of Agile

So what can we do to avoid agile hype and get the true benefits of agile ways of working? I think it comes down to running experiments and using data to drive improvements.

Run Experiments

I had a refreshing conversation with a technology leader in pharmaceuticals last week who shared with me some of the techniques her teams were using to deliver technology solutions. She said that her team focused on using lean, visualizing the work, and paying attention to flow. They ran experiments. That helped them make dramatic improvements even on their non-sexy platforms like SAP and Peoplesoft. She never once used the word agile, which was notable.

Perhaps that is the way to do it. Run experiments, learn, and improve.

Isn’t that what the first statement of the agile manifesto is all about…

We are uncovering better ways of developing software by doing it and helping others do it.

–Manifesto for Agile Software Development

Using Data to Make Better Decisions

We should also be collecting and using hard data to make better decisions on where to improve.

I used to be a fan of estimation and story points. I liked the conversation they generated. Lately though, I’ve come to see that story points and velocity are rooted in human estimates. They are subjective, not objective. They get abused in all kinds of ways. And they may hurt more than they help.

Rather than using those widely abused story points, how about using flow metrics and throughput accounting? How about tracking WIP aging to spot flow problems and problem features during the sprint? How about using Monte Carlo simulation to create better forecasts?

This is just one example. Another is gathering actual data about customers. If you release a new feature, do you track whether or not customers are using the new feature? Do you survey your customers to determine their level of satisfaction and track that over time?

You can also measure the costs of your development and compare that to industry benchmarks. Or use agility assessment tools like Comparative Agility to compare your team’s agility to the performance of thousands of teams in their database.

We Can Help Without the Hype

If you find yourself constrained in less-than-perfect conditions for Scrum, we can help you get the benefits of agile. We’ve been there.

Vitality Chicago offers a range of different team and organizational assessments that will help you use real data to drive decisions and focus your continuous improvement efforts. We are offering a limited-time promotion for an Agility Assessment for one team that will deliver a set of recommended action steps that will get you the benefits you are seeking from agile. Contact me for details.

You can also read about how we used agility assessments to help a client succeed and get the benefits of agile in this customer success story: Case Study of Agile Adoption at ASCP.