How to measure an Agile team's effectiveness and efficiency: Right measures
As a line manager, CTO, CEO, Product Owner, or Scrum Master, … We are supposed to optimize something — “Optimize: To make it as perfect, effective, or functional as possible(Merriam-Webster)” — For example, more features within less time or making more money with fewer features, or producing more features with fewer resources, …. But What should we optimize in agile teams? And who should optimize what?
So, in software development(regardless of Traditional or Agile), there are four key areas for optimization. Or on the other hand, you can be an: 1- Activity Optimizer, 2- Output Optimizer, 3- Outcome Optimizer, 4- Business Impact Optimizer, or a mix of optimizers.
For example, traditional project managers mostly act as Activity (What they are doing)and Output(What they are producing) optimizers. As an Activity optimizer (AKA resource management), they focus on managing individuals and their workload; the popular measures in this area are the Work Log and the number of codes committed per day or week. (e.g. Yesterday, I worked 3 hours on this bug, and 2 hours on the planning meeting and …)
But as an Output Optimizer, They usually focus on the team's throughput. Put more simply, they are responsible for making this idea happen: Produce more features within less time and cost. To do this job, they usually use measures like Lead Time and cycle time to find bottlenecks, try to remove or reduce them, and optimize the Output of the team.
Contrarily, the outcome is about the users and customers. Our target users/customers have a couple of problems/challenges today, so they will hire a product to do their job. Outcome optimization is about building features that customers/users want to use and love. Therefore, feature usage and customer satisfaction are the key indicators of outcome optimization.
People don’t want a quarter-inch drill, they want a quarter-inch hole. Ted Levitt
But what about business impact? We should build a feature that customers want to use, AND they want to pay for it or bring any value to the business.
Anti-patterns for Agile teams:
Just over-focus on optimizing flow or delivery
Today most agile coaches, scrum masters, or delivery managers just focused on Kanban and Lean practices; the main idea is to optimize the flow. I am a fan of Lean and Kanban, but we should consider that if lean is about optimizing the flow and delivering faster and smoother, Agile or lean-startup is about learning fast and eventually delivering the right product.
So an agile team shouldn’t only focus on optimizing the flow of the delivery; they’re responsible for the outcome too, which means building features that customers/users want to use and love. Sometimes this anti-pattern is also known as Feature factory too. The main idea of mixing Agile and Lean is to deliver faster and more frequently to Learn Faster, and build a product that customers want.
As described by Satya Nadella, CEO of Microsoft, leaders must shift from a “Know it all” to a “Learn it all” mindset. In a feature factory, we assume that we know it all, and the only problem is a smooth and fast delivery production line.
In scrum teams, we have a dedicated role called Product Owner. Their mission is just one thing: How to Optimize the outcome with Less Output, or in other words, How to Make Customers/users happy with Fewer features.
But they should think about business impact too because we are not a charity or a non-profit organization to make people happy for free, we should make money too, or make the market share bigger or …, so the question will be “How can we make a product with a minimum set of features(Output) that customers love to use(Outcome) and pay for it(Impact)?”
There are a few anti-patterns about product owners:
- Product owners do not have direct contact with end users, so they do not have a good grasp of users' needs
- Product owners do not have enough domain knowledge, and they only act as backlog managers
- Product owners do know anything about the competitors
- Product owners only act as output optimizers
Just optimize Business Impact
Just-optimize-Business-Impact, AKA leaky bucket too; instead of creating a high-quality product, you focus on forming and hiring a powerful marketing and sale team. A good signal to identify this syndrome is the customer/user churn rate; if the user churn rate is too high, it can signal that you are optimizing the wrong thing now.
Put in practice:
The Scrum Master is accountable for the team’s effectiveness. They do this by enabling the Scrum Team to improve its practices. The best thing for scrum masters is to help their teams grasp the current situation based on data by creating information radiators like a dashboard and facilitating and guiding them to decide the next wise action. The team should own its measures and set a direction based on them.
The Product Owner is accountable for optimizing the product's value resulting from the Scrum Team's work, AKA Outcome optimizer:
Discovery Stage:
In this stage, the Product owner should ensure that they understand the issue rather than merely assuming it. It involves speaking to and spending time with people affected by the issues. The gathered insights can help you to define the challenge differently.
Delivery Stage:
Give different answers to the clearly defined problem, seeking inspiration from elsewhere and co-designing with various people. It involves testing out different solutions at a small scale, rejecting those that will not work, and improving the ones that will.
Validation stage:
This stage is often overlooked. You can use a measure like a Feature usage index to learn whether your customers use your features or not in the real world.
In the above image, we have released two new features, “Backup” and “Screen sharing”, that nobody uses. The product owner should ask herself why? Then, she can start to talk with prospects. She can generate some assumptions and, based on these assumptions, begin to tweak the features, for example, showing a reminder for backup. If it does not work, She can think about either more tweaks or removing it from the product.
Final Notes for Scrum masters and Agile Coaches:
You would like to focus on outcome and impact and not act as just output optimizers. But optimizing output by creating a smooth flow(Like measuring lead time and cycle time and introducing WIP limit, …) can be a great short-term win to find this opportunity and privilege to focus on other areas.
Do not hesitate to share what you think about this story :)
Regards
Asad Safari