Agile Metrics: The Dos and Don'ts

Veronique Skelsey, Senior Consultant

In the fast-paced world of asset and wealth management, Agile metrics are a strategic asset. They provide crucial insights that help firms become more efficient, improve client satisfaction and build robust operations that that can adapt to market changes, client needs and regulatory requirements. This article explores the fundamental ‘Dos and Don’ts’ of Agile metrics to support your strategic decision-making and operational workflows.

Agile Metrics support what are known as the "Pillars of Scrum" (which are equally relevant whether you're working in Scrum, Kanban or any other framework):

pillars of scrum.png

These principles should be at the heart of everything you do with Agile metrics.

 

How to use Agile Metrics as a Team

Do:

  • Have the whole team look at your metrics at the end of every iteration (this fits very well as the opening section of the Retrospective)
  • As a team, reflect on the root cause of changes to trends or areas of concern in the metrics
  • Brainstorm improvements and prioritise the most urgent or impactful improvements (to team or organisational processes, behaviours, technologies etc.)
  • Do something about it! e.g. Use the metrics to inform how you plan your next sprint (are you adding too much to a sprint? Are your stories too big and always rolling over?). Add at least one improvement to the next sprint/iteration with an owner (who will be responsible for following them up with the right people to make sure they happen, NOT necessarily actively making the improvement themselves)
  • Finally, reflect on whether the existing metrics are sufficient or whether additional metrics might be useful to track and address areas particular to your team

Don't:

  • Try and make the "numbers" better - the metrics are a tool to help you figure out where problems exist and improvements can be made, not an end in themselves
  • Try and compare yourselves to other teams - their experience is not the same as yours and what works for them may not work for you
 

How to use Agile Metrics as a Manager

Do:

  • Take an interest in your teams' metrics and reinforce their importance in a team's awareness and ownership of the continuous improvement of their product, processes and practices
  • Offer your support and help if metrics are trending downwards
  • Own and escalate organisational improvements identified by your teams

Don't:

  • Ask (or imply) that teams should make the "numbers" better - the metrics are a tool to help them figure out where problems exist and improvements can be made, not an end in themselves. When teams are asked to improve numbers, they often feel obliged to "game" the system to give you what you asked for. This corrupts the data and reduces transparency, so you can no longer see where issues are occurring and get a true picture of the team's health
  • Use the metrics as a comparison between teams. Teams have different challenges. Context is key - look to understand what metrics are highlighting, what the root cause might be, and what might need to change for a team to improve
 

Why it's important to use Agile Metrics as a Tool and not a Target

Simply stated, Goodhart's Law says: “When a measure becomes a target, it ceases to be a good measure".

Think about what you're measuring and the insights and outcomes you'd like to achieve.

There are 5 key metrics that can help ensure good team health and effective delivery: 

  • Say/Do ratio - what a team thinks they'll deliver at the start of the sprint vs what is actually delivered (particular to Scrum as it operates with a timeboxed iteration approach)
  • Throughput - how many items a team complete in the sprint / iteration
  • Cycle Time - the average time taken from when a team starts working on an item to it meeting their definition of done 
  • Defects - the number of escaped defects live in production 
  • Happiness - the happiness and satisfaction of a team

A team should be looking at improving the outcomes NOT the numbers.

 

Let’s take a look at what can happen when a team is asked to address the metrics rather than the outcomes...

The Say/Do Ratio compares what a team initially put into a sprint to what they completed by the end. It is desirable to get the number of items completed as close to the number committed as possible because it improves predictability by ensuring fewer items roll over to subsequent sprints; thereby making it easier to forecast when backlog items will be completed, resulting in happier stakeholders. 

A team who is asked what is causing their Say/Do Ratio to be so divergent and to think of ways that they could improve the situation may conclude that their stories are too big and need to be broken down to make it easier to track their progress and complete them within a sprint, or they may be adding items to a sprint that still have outstanding questions or unresolved dependencies which means that they're not really "ready" for them to start work and results in incomplete items at the end of sprint. They may be doing a lot of support work which is unaccounted for in their sprint planning or saying "yes" to mid-sprint requests without considering the priorities and the impact. They can come up with ideas to try and address the problems: ensuring stories are small enough to fit comfortably in a sprint (or preferably within a few days), or creating and mandating a Definition of Ready, or including capacity considerations during sprint planning, for instance.

A team who is asked to get their Say/Do ratio closer to 100% will often do exactly as they're asked, by trying to fix the numbers. They may do this by becoming overly risk-averse and only putting a small number of items into a sprint at the start, and then pulling items in as the sprint progresses. They may "close" items in the sprint before they're truly complete (i.e. before they've met the Acceptance Criteria and Definition of Done) and then create a duplicate or "part two" item in the next sprint. These "fixes" affect transparency and have a detrimental effect on predictability, making it hard to understand what is complete and releasable, and hard to forecast when items will be delivered. 

As Steve Jobs said: “Incentive structures work. So you have to be very careful of what you incent people to do, because various incentive structures create all sorts of consequences that you can’t anticipate.

 

Conclusion

Agile metrics are essential for continuous improvement of teams and processes but remember the goal is to improve outcomes, not just the numbers. Delve deep into what they reveal to not only predict outcomes more accurately but also enhance operational agility and client satisfaction. At Liqueo, we proudly specialise in asset and wealth management and are ready to guide you through the complexities of becoming Agile throughout your organisation. Our team’s deep domain expertise means you can trust our recommendations are tailored to meet your specific needs and will always be rooted in our understanding of the impact on the intricacies of your trade flow and broader operating model. If you are interested in how we can help you implement successful programmes or want more information about Ways of Working, contact us.

 

Our use of cookies

We use necessary cookies to make our site work. We'd also like to set analytics cookies that help us make improvements by measuring how you use the site. These will be set only if you accept.

For more detailed information about the cookies we use, see our Cookies page. Cookie Control Link Icon


Necessary cookies

Necessary cookies enable core functionality such as security, network management, and accessibility. You may disable these by changing your browser settings, but this may affect how the website functions.