Metrics are valuable tools for guiding testing strategy toward business value
Measuring progress in a software engineering team is done in various ways. The most common method in the industry is through story points. Story points measure effort, not time. More emphasis on time can lead the team to focus on low-value stories to complete the sprint. This needs to be more accurate because the primary focus of a delivery team should always be on business value. Managers need to understand the actual meaning of the metric. They should not use it as a goal or target but as a tool to inform their testing strategy.
Acknowledging the progress of a software system is achieved by delivering new functionality at the end of each sprint. Working software is the only tangible proof of progress, not any burnout chart. Teams need to prioritize such actions to conserve the company’s resources.
The issue of measuring effort becomes even more complex when dealing with bugs. Measuring progress based on activity creates incorrect incentives for prioritization. Bugs arise when our understanding of the problem is more profound than anticipated. This problem is inevitable, as software is never “done” since it requires ongoing maintenance. We can measure our work in a way that maximizes predictability. Activity metrics help assess team collaboration but do not indicate real progress. Focusing on the behavioral changes a feature or application aims to influence is more valuable.
Before selecting metrics, it is essential to understand the reasons for measuring. Google employs the Goals/Signals/Metrics (GSM) framework to guide metric creation. Start by defining the goal you want to achieve. The purpose should describe what you want to understand at a high level and not include specific measurement methods. Define signals, which are indicators of goal accomplishment.
Choose metrics that serve as proxies for the signals. Some signals may have multiple metrics. If the data supports the expected result, determine the action to take. If no action is taken based on the results, there is no point in capturing that metric. Consider whether appropriate action will be taken if a negative impact is obtained. If other factors override a negative result, it may not be worth measuring. Determine who will make decisions based on the results and when they will do so. “The goal of measuring our software process is to help people make business decisions. What do story points count? Activity metrics like velocity or the number of completed stories do not accurately measure progress. This information needs to be more precise and based on quantifiable data.
Impact refers to a change in customer behavior, and this goal should provide value to the business. Consequences are not the same as product features. Product features represent a set of behaviors that the system supports. Therefore, impact describes the outcome of a solution. A better behavior change to tell would be “being able to find information faster.” the steps in between are called deliverables. “Better mobile search” is not an impact but a deliverable. Agreeing with the client on what must be done and, more importantly, what needs to be done significantly impacts your development’s success.
