Poorly chosen measurements can lead to undesirable results. Last week Wells Fargo hit the news with stories about corporate fines, employees fired, and $1.4B in market capitalization being wiped out. All of this was because the company was tracking and rewarding employees for opening new customer accounts. Though it seems like an innocuous thing, the measures and focus on new account growth was in conflict with the best interest of the customers of the bank.
Similar things can go wrong when we track the wrong things with our Agile Teams. Managers and leaders often give little or no thought to what is being asked of teams, or seem oblivious to the impact of their measurements and reports. They want high performing teams but what they actually measure is something else, and the result is anything but high performance.
Don't get me wrong, I am a big fan of metrics and measurements. For example, I marvel at the change in behavior that I see from my friends that are using Fitbits or other personal fitness tracking devices. Those simple devices, and a public "scoreboard", has induced lots of people to adopt healthier habits like going out of their way to walk or run more. The "prize"? To be the current leader on the board for their group. That is the reward. Oh and it also leads to good health. (My wife's company just introduced a form of Schrute Bucks as a reward for physical activity!)
In hindsight, it is clear how the measures and rewards set up at Wells Fargo led to the unethical and fraudulent behavior at all levels of the organization. Though I doubt that any leaders at Wells Fargo said "please act unethically", the message of the measures and rewards was exactly that.
Where do managers and leaders of agile teams go wrong? Here are some of the more problematic measurements that I have seen:
False Productivity Measures - Because it is easier, most organizations measure outputs and treat them the same as impacts. They are different. Examples of outputs include lines of code written by the team, deliverables produced, test cases developed, or time spent on production support. Impacts are things like cost savings, cycle time reductions, or new customers obtained. The more impact you can get with the least amount of output, the better.
Anti-Team Measures - Sadly, I've seen managers who track things that work against desirable team behaviors. Some managers try to track which team members are responsible for each individual story in the sprint, in particular any that don't get completed. I've also experienced functional managers trying to maintain their old reporting lines by asking team members to track things that subverted team goals. I know a Quality Manager in a major bank that insists that all test professionals working as members of agile teams report defects to him on a weekly basis. Why? And what behavior will likely result from it? Will finding a lot of defects make you think that I am doing a good job? How does my defect reporting affect my team?
Similarly, some managers request that agile teams report all their notes from their retrospectives. Or worse, they insist on sitting in on the retrospective to get a feel for how things are going. They cannot imagine doing it any other way.
Red, Yellow Green Status for Agile Teams - Most PMO organizations want to boil the team status down to just one indicator - Red, Yellow or Green. Results? Most teams report Green, right up until delivery time when they are Red. And then you have those watermelon projects that everyone on the team knows are a problem - they are Green on the outside and Red on the inside.
Measuring Hours Worked By Individuals - My all-time least favorite measure is the one that managers and project managers seem to latch on to like a dog with a bone. They want to keep track of time spent on tasks within agile teams in the sprint.
Hours worked by person is easy to measure, especially if teams are using a tool like JIRA. But measuring hours worked is subject to the Goldratt quote above. People will behave in ways that they think management wants. If I think you want high actual hours, I will record that. If I think you are watching the variance between my plan and actual hours, I will make sure my actual hours reported turns out to be as close as possible to the plan, even if I have to inflate the plan to always hit it. Whether or not that is the reality, my behavior will produce the numbers I think you want and value.
Sadly, I know of a company that does this, much to their detriment. The "scrum masters" for each team track the hours worked vs. plan every day, and they produce individual burndowns showing the plan and actual hours worked for the current sprint. These individual reports go out each morning to the team member and their manager. There is an expectation that each person would "burn" the hours they planned, and if they didn't, their managers were supposed to do something about it.
The problem that the organization was trying to address was rampant multi-tasking due to the over assignment of projects to people. The problem was exacerbated by managers who changed priorities to try to put out fires and reassigned people working on teams. Do you think the daily individual burndown reports did anything to help the situation?
As one might expect, most of the reports were simply ignored. They just became more noise in an already chaotic and frantic environment. To avoid trouble, most team members ensured that their hours were "burning down" exactly as planned. The reports went to managers who were busy directing this person to this project or task and fighting fires on all fronts. But nothing changed the fact that there was too much work - a management problem - and not enough motivated people to do it.
Takeaways for Agile Leaders:
#1 - Think before you introduce any type of measure or reward.
#2 - Get honest with yourselves about your own desire to control people. That is all most of these measures are designed to do. Instead of controlling people, lead!
#3 - Empower your teams. Give them the big picture goals and let them figure out the best way to get things done.
What does your organization do about metrics and reporting for agile team members? In my next post on this topic, I will suggest some things that could be helpful to measure instead.