Goal and success metric: writing down “why” so acceptance has something to compare

A user story has three parts: who, what and why. The third is the one most often dropped, and when it is written, it is written so that it cannot be checked: “to make it more convenient”, “to increase loyalty”. A month after release nobody can say whether the task succeeded. The feature works — so it succeeded. But whether it works and whether it helps are different questions.

Goal, task and metric

Three notions that specs usually merge into one.

• Goal — the change in the world everything is done for: customers call support less often to ask “where is my order”.
• Task — the means: notify the customer when the order status changes.
• Metric — the number that shows whether the goal was reached: “where is my order” contacts as a share of orders.

A task without a goal gets done formally: the notification is sent, the box is ticked. A task with a goal lets you notice that notifications go out and calls have not dropped — for instance, because the status in them makes no sense to the customer.

How to write a goal so there is something to compare

Goal          customers ask support less often where their order is
Metric        “where is my order” contacts / orders per week
Now           6% (measured over 4 weeks before release)
Expected      no more than 3% four weeks after release
Must not      unsubscribe rate from notifications — no higher than 2%
get worse
Reviewed by   process owner, at the review one month later

The “now” line matters more than it seems. The baseline has to be measured before release: afterwards there is nothing to compare with, and the past cannot be reconstructed from memory. If there is nothing to measure it with, that is a finding too: the task gains a requirement to collect that number.

Metrics that mislead

The counterweight is mandatory. Any metric can be improved at the cost of something else. The second line — what must not get worse — protects against a solution that meets the goal on paper and does harm in practice.

What to do if the metric did not move

A metric that did not move is not a failure but information: the hypothesis linking task and goal was not confirmed. The bad part is not that, but that usually nobody notices — there was nobody looking. So it pays to write two decisions into the spec in advance:

• when we look — a specific date or number of days after release;
• what we do if it did not move — investigate the cause, try a different means, roll back.

A decision made before release is made calmly. After release people get used to the feature, and giving it up gets harder even if it does nothing for the goal.

When a metric is not needed

Not every task has a number. A defect fix, a legal requirement, a library upgrade — their goal is obvious and checked by the fact. A metric is needed where the task is a hypothesis: “if we do X, Y gets better”. Such tasks are fewer than half, but they are the most expensive when they do not work.

The sign there was no goal: half a year later the feature cannot be removed, because nobody knows whether anyone needs it.

A ten-minute check

Take three tasks in the current sprint that change something for users. For each, try to name the number that should change and its current value. Tasks with no number are either obvious, or hypotheses nobody will check.

What next

The goal is the top link of a chain that runs from why to what, from what to code, and from code to the check. For that chain not to break at the joins, its links have to be tied together. That is the subject of traceability.