Measuring outcomes and defining metrics

Shipping is not the end. Outcomes are the changes in user behaviour and business results that a design causes. Outputs are the things you made. Teams should be judged on outcomes.

Types of measure

• Behavioural: completion rates, conversion, time on task, retention, feature adoption, error rates.
• Attitudinal: satisfaction surveys, ease ratings, Net Promoter Score, qualitative feedback.
• Operational: support calls, refunds, manual processing, cost per transaction.

Measuring well

• Define success measures and a baseline before launch.
• Watch for guardrail metrics: things that must not get worse, such as complaints or accessibility.
• Allow enough time: some effects take weeks to appear.
• Account for other changes, such as marketing campaigns or seasonality.
• Combine numbers with qualitative insight to understand why.

Honest reporting

Report what happened, including disappointing results. A change that did not work is still valuable learning.

Measuring outcomes proves the value of design, guides what to do next and builds credibility with leadership. Without measurement, teams cannot tell success from failure and keep building on guesses. Outcome evidence is also what makes portfolio case studies convincing.

A team redesigns a council tax direct debit sign-up. Before launch, they record a baseline: 61% completion, 340 related phone calls per month. Guardrails: complaints and accessibility issues must not increase.

Eight weeks after launch, completion is 78% and calls have fallen to 190 per month, with no rise in complaints. A short survey shows people value the clearer payment date choice. The team shares the results, including that mobile completion is still lower, which becomes the next focus.