Lately, I’ve been hearing a lot of buzz about agile assessment tools. You know, those tools that can tell you whether you are moving toward business agility? It makes sense that the people who are investing in agile ways of working want to have some measure of whether they are making a difference or not. Or perhaps they want to compare teams within their organization or compare their teams to a benchmark such as a global database of similar teams.
Different reasons exist for measuring agility. Most people want to see some progress; to know whether they are making a difference. Some of the most common reasons I have heard include:
I don’t know if I have met an agile coach that hasn’t developed their own agile assessment tool. I know I did. I am embarrassed to say that over 10 years ago I developed a very crude tool that purportedly measured agile readiness (I don’t think it did). Tom Cagley was mentoring me at the time and provided me a number of pointers. He should have just told me that my baby was ugly (it was) and to give up (I finally did).
A few years later I was working with another coach supporting Global Markets at Bank of America and we (mostly him) developed a spreadsheet that Scrum Teams would use to measure 50 Scrum practices and outcomes. That tool was much better that my original one and I used versions of it over the next few years. It even provided a color coded graphical summary for multiple teams – something that is pretty common today on the modern agile assessment tools. But it was clunky and labor intensive. And the output wasn’t beautiful or intuitive.
Today I suspect there are still a lot of people using homegrown tools. The benefit of the homegrown approach is that you can tailor the assessment to your specific needs. The downside is that your specific tailoring may result in something that is not comparable to other organizations. Or you might have to go through and customize it for every organization. And home grown tools rarely have the ease of use and fancy graphical outputs that the commercial tools have.
So why would you roll your own when there are so many better options available?
What follows is a list of some of the popular agile assessment tools available today. I’ve tried to capture licensing fees, strengths and weaknesses and other factors.
One of the first agile assessment tools that I know of is the Nokia Test. If you are young, Nokia may not be a name you recognize, and it probably isn’t the first company that comes to mind today when you think about agility. But back in the mid to late 2000’s Nokia was the hottest cell phone maker and literally owned 50% of the market. I had the phone below and loved it. It didn’t shatter if you dropped it. But that is beside the point.
In 2005, Bas Vodde created the Nokia test to gauge agility across the agile teams he supported at Nokia. It was subsequently picked up by Scrum co-founder Jeff Sutherland and expanded to include Scrum practices. It remains a simple and quick test of agile and Scrum.
Here is an example of a question and the scoring (0 – 10 points) from the Nokia Test:
Question # 2 – Testing within the Sprint
The great thing about the Nokia test is that it is free. The downside is that you have to do all the work yourself. You can download the questions from Jeff Sutherland’s blog here and either create a written test or make an online version that you would score by hand.
Where I think the Nokia Test could be helpful is to mix up your retrospective routine. Perhaps once a quarter or once every 6 months you could use this to bring fresh insights. By walking through the various questions as a team, you can identify where improvements are needed or possible.
[See our related post on retrospectives: A Quick Tutorial to Improve Your Retrospectives]Mike Cohn and Kenny Rubin created the Comparative Agility assessment in 2007. As the name implies, a key feature of Comparative Agility is the comparison of your team or teams to other teams in their online database. Within five years of the launch, Comparative Agility already had over 5,500 teams in their database. As the database grew, comparisons could be made to teams of the same age, in the same industry, or developing similar solutions. Today over 14,000 organizations have used the Comparative Agility tools.
Almir Drugovic and Jorgen Hesselberg were early adopters of Comparative Agility, ironically when they were at Nokia in 2008. Almir and Jorgen purchased Comparative Agility from Cohn and Rubin in 2015 and have expanded it significantly since then.
Today, Comparative Agility is more of a continuous improvement platform. You can use the original Comparative Agile assessment or create your own unique assessment. You can also choose from one of the 20+ other surveys authored by thought leaders in various fields including Mike Cohn, Dr. Amy Edmondson, Jutta Eckstein, Dr. Rita McGrath, Esther Derby, Gil Broza and others. Here is a view of some of the currently available assessments:
Comparative Agility is a bit unique in that it doesn’t adhere to a pre-defined maturity model. There is no goal of agile nirvana that teams will work toward and arrive at. Instead, they treat maturity as a journey of continuous improvement without a set destination.
Strengths of the Comparative Agility Assessments:
Weaknesses of the Comparative Agility Assessments:
Pricing for 5 and 20 teams:
Learn More: You can check out the Comparative Agility website to learn more about the tool.
AgilityHealth is an Enterprise platform that aims to accelerate Business Agility through measurement, growth and outcome alignment. AgilityHealth was created by Sally Elatta and her team in 2014. (Disclaimer: I worked with and am a big fan of Sally and Agile Transformation.) I was trained and certified in 2014 as an AgilityHealth facilitator though to be honest, I only used it once in 2014 and haven’t spent much time on it since.
The platform was originally launched to support teams assessing and improving their TeamHealth which continues to be their global benchmark radar. I first envisioned it as a tool to support ‘retrospectives on steroids’.
It is recommended that the TeamHealth retrospective be facilitated at the end of each quarter before the next planning meeting. AgilityHealth certifies internal facilitators (AHF) who walk the team through the radar. The facilitators create an environment of psychological safety to encourage people to respond openly and honestly. Then each team member completes the assessment. Results are shown in real-time on the visual radar graph. The facilitator strives for open conversation and reflection and then helps create a clear growth plan with 2-3 items that gets logged into the platform.
Initially, AgilityHealth used Tuckman’s stages of team development like Forming, Storming, Norming, and Performing but found that teams were not as objective as they would have liked. So they moved to the following levels: Pre-Crawl, Crawl, Walk, Run, Fly.
Over time, AgilityHealth has expanded to offer a range of assessments for different agile roles, and different levels of the organization. It moved from being a tool for team measures to a platform that supports measurement and improvement. Organizations can zoom out or zoom in, look from different angles and gain a lot of insights into how things are actually working. Some examples of the different views are shown below.
Noteworthy Features of the AgilityHealth Platform:
Strengths of the AgilityHealth Platform:
Weaknesses of AgilityHealth Platform:
Ballpark Pricing:
Learn more about AgilityHealth on the the AgilityHealth website or check out this 3 minute AgilityHealth Explainer video.
Agile Fluency ® was first published by agile veterans James Shore and Diana Larsen in 2012 and updated in 2018. They created the model based on ways that people practice agile and use it well.
It is positioned as a tool for consultants to use with client organizations. In fact, the home page emphasizes the potential fees to be generated which is a bit of a turnoff.
The tool provides feedback in terms of 5 levels of fluency: Pre-Agile, Focusing, Delivering, Strengthening and Optimizing.
Strengths of the Agile Fluency ® tool:
Weaknesses of the Agile Fluency ® tool:
Ballpark Pricing:
Learn More:
You can check out the Agile Fluency ® site, download the free eBook The Agile Fluency ® Model, or watch this short video about the model which features Diana Larsen and James Shore.
I recently learned about an assessment tool from Michael McCalla and LeanAgileIntelligence.com. Lean Agile Intelligence was launched in 2017 as a continuous improvement platform that allowed agile leaders to easily build custom assessment templates to directly align to their context, change strategy, and desired outcomes. It orients to those desired outcomes (i.e., Customer Satisfaction, Time to Market, Reliability) of the transformation efforts as well as measurement of the specific practices that lead to those outcomes. McCalla created the Lean Agile Intelligence instrument based on his expertise with several enterprise-level transformations with Fortune 500 companies.
One of the things that I really liked about the tool was the ability to customize to the target environment. This includes not only the prompts that are part of the assessment but the ability to group or tag information for better reporting across the organization. The screen shot below shows how easy it is to tailor the language used.
The Lean Agile Intelligence tool provides a holistic view of the overall organization and allows you to drill down into specific departments or individual teams.
Strengths of Lean Agile Intelligence:
Weakness of Lean Agile Intelligence:
Pricing:
This agile assessment tool is priced similar and slightly less than the AgilityHealth assessment. It will be about $1,300 each year for one team to do quarterly assessments.
There are many other lesser-known agile assessment tools from a variety of companies. Some of the more recognizable company names include Thoughtworks, IBM, Eliassen Group, Atlassian, and NetObjectives. There is also the Scrum Survey which is a scrum-flavored assessment that offers free and paid functionality.
Do you have experience with one or more of these tools that you would like to share? Please leave your comments below.
Hat Tip – I’d like to thank Randall England, Vince Yabut, David Shapiro, Jorgen Hesselberg, Michael McCalla and Sally Elatta for taking the time to speak to me about the assessment tools above. Also, a belated shout-out to Ben Linders for sharing his exhaustive (exhausting?) list of self-assessments. Check them out here: Self-Assessments list from Ben Linders