Measuring ROI is often elusive because there is a disconnect between the expected results and the direct outcomes of the program meant to deliver them. Without realizing it, we are talking about two different measures; and often the final measure identified to evaluate the “expected results” is not the best reflection of those outcomes.
While the mystery of ROI calculations leads some organizations to opt out of measuring program impacts altogether, a better understanding of what we are really asking for can draw a measurable link between programs and results.
To start, the redefinition of ROI must focus on outcomes, often observed as behaviour change, rather than attempting to move directly to impacts.
An impact value chain links the direct program outcomes to a series of progressive metrics, usually in the form of operational measures (consequences), and then on to strategic metrics (impacts).
Arguably, the most challenging part of measuring ROI is building a valid, reliable impact value chain to bridge the gap between the effects of a program at the individual level and their long-term, cumulative impact on the business.
With proper impact value mapping, the on-going work of measuring ROI is about data collection, consolidation, and analysis.
Without an impact value chain in place, that same data becomes meaningless, unhelpful, and at worst, misleading.
Before mapping the chain of impacts, let’s address the inconsistencies in language and terminology used throughout evaluation literature. Measuring ROI is a cross-functional endeavour; and aligning on shared definitions of terms like impact, metrics, KPIs, and measures before you start will improve the validity of the final impact value chain.
Measures, Metrics & KPIs
Often used interchangeably, these three terms in fact refer to very different things.
Let’s use the goal of “fitness” to see how these three inputs work together.
Measures are independent observations of what’s going on (e.g., resting heart rate, daily exercise, calorie intake)
Metrics pull these observations together, clarifying how the system works (e.g., body composition, cardiorespiratory capacity)
KPIs help us to understand what it all means relative to a goal (e.g., cardiorespiratory capacity/VO2 max)
ALL THREE of these inputs are necessary to calculate ROI and should appear in the impact value chain.
The importance of KPIs (they’re the vital signs!), combined with the reality of resource constraints often leads to prioritizing KPIs over other metrics – sometimes going so far as to assume other metrics cannot offer valuable business insights.
On the contrary, non-KPI metrics are imperative for true business insight, and here’s why:
While KPIs provide a direct line of sight into progress against an objective, other metrics work together to tell the whole story about what is going on.
Insight is context-dependent - if KPIs aren’t being met, it’s important to have other metrics in place to help us understand why.
How do we choose the right measures, metrics, and KPIs? How do these translate into outcomes, consequences, and impacts? And how can we set ourselves up to more easily measure ROI in the future?
In the final post in this PROVE IT! series, we’ll work through a step-by-step framework to map the impact value chain you can use to measure ROI. We’ll also consider the benefits of a measurement strategy in being proactive about measuring ROI.
**This is part of the PROVE IT! Series of posts on measuring and using ROI to generate organizational insights and drive strategic decisions.