All decisions should be fact based and in software this presents a problem because facts are often elusive. The answer is of course to use metrics; however I learnt in college that information must be:
  • Timely
  • Useful
  • Accurate
The Agile methodologies promote metrics as an important part of the planning and estimating process. However software creation and testing is only part of the process. We need metrics that address the whole of Application Lifecycle Management (ALM), which is a daunting task because of the potentially large amount of information and scope implied. However by the simple process of:
  1. Using a ticket management and capturing all issues, not matter where in our lifecycle they occur
  2. Following a defined process, or processes (usually one process does not fit all needs), to address our issues and using our ticket system to track the process.
we can capture some valuable information which we can use in a variety of simple ways to obtain metrics. So what are useful ALM metrics? Here are some example suggestions.
  • Rework rate and costs -- by documenting each issue via the ticket system we can get some idea of how much we are spending on rework. In particular we can capture information to tell us:
    • Where we are finding issues (in UAT, Regression or production). This may tell us that specific parts of our processes need attention (e.g. UAT or Unit Testing)
    • Which sub-systems require most rework. This might imply that some parts of the system under discussions are fragile and need to be re-architected
    • Classification of root causes (e.g. Poor requirements/user stories, poor unit testing, etc)
  • Process instrumentation. By looking at the amount of time spent in each process we can find efficiencies and cost savings
  • Trend analysis, which allows us to spot potential problems before they become too large and test our assumptions about where we should deploy our effort. It is possible to trend all sorts of data, provided that the information is initially captured in our system (e.g. Severity, cost to fix, time to fix, business feature impacted, time to test etc.)
However the core principal is that it should be simple and useful. Also note that as the lifecycle progresses, and remember we are also talking beyond the lifetime of a single development project, which metric is useful will change and it will be necessary to re-tool the metric reports. Make sure you capture that in a ticket as well! And finally remember, "You get what you measure"

powered by performancing firefox