9 Empirical Metrics for Agile Success

Whether your organization is trying to go Agile or has been practicing Agile for some time, defining the metrics used to gauge the success of your efforts can be a tricky endeavor. But it’s an important exercise to undertake, not only so that Agile teams can measure their progress and improvement over time, but also, and more importantly, to establish a common vocabulary between the business and IT. If one of the aims of an Agile transformation is to better align IT with the business, then it behooves IT to communicate progress and improvement in a way that’s meaningful and actionable to the business. Doing so helps to establish the business case for undertaking an Agile transformation, as well as for scaling or investing more in Agile improvement efforts for an organization with more maturity.

So what metrics should you be paying attention to? Matthew Badgley shared some empirical tips for measuring Agile success in an article on the VersionOne blog that were culled from the 9th Annual State of Agile Survey and based on the responses from nearly 4,000 respondents. These tips, then, come from a survey question that allowed respondents to make multiple selections.

On-Time Delivery
58% of respondents reported they use on-time delivery to measure the success of their Agile initiatives. According to Badgley, “on-time is generally measured in context with the expectations about what will be delivered.” Basically, is what you’re committing to at the beginning of a sprint being delivered at the end of the sprint? Badgley recommends using either the burndown or burnup rate to measure.

Product Quality
48% of respondents are using measures of product quality to gauge success. The metrics used to measure quality range from customer satisfaction to revenue growth, which are both fairly abstract for the Agile team, to monitoring testing trends throughout the life of a product. The basic question you’re trying to answer here is whether or not you’re building quality into the product up front, and how successful those efforts are.

User Satisfaction
44% of respondents are using this one to gauge success. Measures include usage statistics of product, number of support calls vs. number of features delivered in a time period, sales figures, and the Net Promoter score.

Business Value
44% also regarded business value as important. Business value is a murky one, unless you’re in a contract work scenario. For this one, collaboration with the business is essential to defining how business value will be measured and what the units will be. To some extent this will be arbitrary, in the way that story point estimating is somewhat arbitrary – the units themselves are not as important as everyone having a contextual understanding of what they represent.

Product Scope
39% are using product scope for measuring Agile success. According to Badgley, “Setting a goal around what to get done over the next three months, then tracking status, and getting it completed is hugely rewarding. Actually having real-time feedback as to the progress of work is valuable to everyone on the team, from the engineers to the program managers.”

Project Visibility
30% of respondents track visibility, tracking metrics that compare progress against the plan, or said another way, whether you’re over or under scope. “The other reason visibility is important is because we need to have alignment among internal teams so they can best manage their work in relation to component or service dependencies,” adds Badgley.

Productivity
29% are gauging success through measures of productivity, but it’s important to remember that in Agile it’s the outcomes which are measure, as opposed to output. According to Badgley, “Simply looking at a burnup of count of stories or features over time is a great way to understand how much the team is actually delivering.”

Predictability
25% are gauging success using predictability. For this, Badgley recommends using velocity as the measurement, “A velocity that wildly fluctuates might reflect a team that is changing, work that is unpredictable, or simply a team that is still getting used to defining work small enough to complete in an iteration.”

Process Improvement
For this last one, 23% are measuring process improvement to gauge success. For the mature Agile team, this is a good one to track, as part of Agile is about continuously improving over time. In addition, measuring process improvement forces the team to think about the flow of value through the system, which is a key tenet of Lean thinking. Two ways to go about measuring this flow over time are keeping an eye on cumulative flow over time to help identify bottlenecks in the system, as well as cycle time. According to Badgley, “Cycle time is a great metric to view over time to see if process tweaks and adjustments are having an impact on productivity.”

Read more about how to bring database changes into Agile workflows by downloading our white paper, Making the Database Agile: Best Practices.