Agile Teams - Deciding What to Measure

Anthony Mersino
August 30, 2016

In my previous post, I talked about some of the ways that Agile leaders foster incorrect behaviors by measuring the wrong things. The recent shenanigans at Wells Fargo serve as a great example of what can go wrong when measurements are set up poorly. For Agile and Scrum teams, I recommended that the leaders stop measuring the easy things like hours worked, since it leads to undesirable behaviors and ultimately decreased productivity. In this post, we will look at what should be measured instead.

Organizations are in business for a reason. For most, it is about serving a particular market or customer in order to make a profit. The focus on activity measures is misguided because not every activity leads to providing value to customers or making profits. Some activities lead to outputs, and some outputs lead to impact. Some of those impacts serve the customer and contribute to profits.  So a great first step is to stop measuring activity, and focus on either outputs or (preferably) impacts that you are trying to make. Doing the right activities should lead to value and profits, but more of the wrong activities would not.  


Organizations measure activities because it is easy. Output and impacts are more difficult to measure so most organizations don't even try. This is a classic example of the streetlight effect which is an observation bias.

Tracking hours worked doesn't necessarily lead to greater business value. While showing effort is good, we should not confuse activities or outputs with impact. More activity is not necessarily good, especially if you can achieve the same business impact with less activity by working smarter. If anything, I find people today burnt out from frantic activity and a false sense of urgency. They are showing a lot of activity, but they are not doing the right things.

So activity measures aren't great, but what about output measures? Unfortunately, output measures can also be misleading and a distraction. If we ran 50 test cases, was that good? If we identified 100 defects, is that good? If we produce 100 lines of code, is that good?

Establishing Meaningful Measures

Here are some practical steps to take when setting up measurements for Agile teams that you hope will become high performing teams.

1. To start, help the team connect their work – their activities and outputs – to the business impact that you are striving to make. Give people the big picture. The more that the people closest to the work understand the overall goals, the better decisions they will make. And don't make the mistake that Wells Fargo did. Their business goal should have been more delighted customers, not more accounts set up. Accounts set up is not a business impact.

2. Ask WHY before introducing any new metric or measure. Determine the purpose behind the measure and what you hope to expose. Think through the behavior that you are trying to change or incent, and the outcome you are trying to achieve to make sure your measures don't actually backfire.

3. For Scrum Teams, work with the Product Owner to determine the business impact of the work of the team. While this may be difficult at an individual story level, it may be easier to see at a feature or epic level. How does a particular feature contribute to revenue growth or cost savings? Does another feature impact customer satisfaction or retention?

It's not easy to make the shift. Some teams try to identify impact by assigning business value at the story level, using the Fibonacci series. While this is imperfect and easy to game, it is better than doing nothing.

4. Understand that a valuable business outcome may be overall business agility through team capability. By this I mean that an organization may desire to have teams that are learning more about the domain, about the solution, and more about how to perform well together. That is a worthy outcome! Business agility is not only delivering the XYZ interface in this sprint, it is also having the capability to effectively deliver future functionality. Business agility is about making change easy and cheap.

5. In the lean startup context, validating (or invalidating) assumptions about our customers and solution is a valuable outcome. If we can agree that we really don't know for sure what "products" we need to produce, learning becomes a great outcome. Teams can view each feature as an experiment that helps them learn and validate their assumptions about what the customer or market needs. In this context, teams invest the absolute minimum while trying to maximize their learning.

Leaders of agile teams need to be cautious about measures used. Tracking hours worked undermines the team's empowerment and motivation. Instead, step back and look at the outcomes you are looking for at the end of the sprint, release and year. That is the “what” to be achieved. Let the team figure out "how" to get something done - the hours and tasks associated. If you don't have the right people to do this, or if you have people that you don't trust, then you have hired the wrong people. And that is on you. Your measurements won't fix that.

About Anthony Mersino

Anthony is passionate about helping technology teams THRIVE and organizations TRANSFORM.  He loves partnering with organizations to help teams with Agile thinking and the Scrum Framework.  He teaches Agile and Scrum as well as the cultural elements that are necessary for an organization to gain true business agility. Anthony has  authored numerous articles and two books: Agile Project Management, and Emotional Intelligence for Project Managers.