Humans tend to equate numbers, graphs and percentages with intelligence. however in the era of PowerBI, HubSpot , google analytics it is easy to get carried away with dashboards, charts and some intelligent sounding acronyms like LTV, AOV, MRR and CPA.
In almost evert company, everyday, strategy is being hijacked by numbers. Because strategy is abstract, business managers often mentally replace it with the hard metrics meant to assess whether the organisation is succeeding at it
But do these metrics and KPIs tie back to your strategy? are you tracking metrics that matter to business? Are you chasing vanity metrics, are you falling prey to surrogation? is your metric a perverse incentive ?
read on to find out more..
- How to figure out your set of key metrics, as well as your target metric
- How a hierarchy of metrics that complement one another can help you measure the health of your business.
- A simple method for building your own key metrics framework
What are Product Metrics?
Product metrics are quantifiable data points and KPIs about your product or service and customer behaviour that help you gauge the success of your product or service. A business should actively track and monitor such metrics to analyse the health of your business and pre-empt future challenges for example :
- At the current rate of customer growth how do your systems need to scale in 6 months ?
- Which marketing channels are generating more traffic and how you can design campaigns for these channels
Examples of product metrics include Net Promoter Score (NPS), Customer Lifetime Value (LTV), churn rate, monthly recurring revenue, etc... These can define the direction, focus and profitability of a product so they should tie back to your product strategy.
Don’t be fooled by Vanity metrics
The metrics that matter to a business depend on the type of business and the maturity of the business.
Vanity metrics are things like registered users, downloads, and raw page views. They can be easily manipulated, and do not necessarily correlate to the numbers that really matter: active users, engagement, the cost of getting new customers, and ultimately revenues and profits. The latter are more actionable metrics
An e-commerce start-up will want to leverage venture capital to acquire new customers quickly, whereas a 100-year-old bank will be more focused on retention and upselling within an already large customer base. Page views could be considered critically important for a publisher generating Ad revenue, but the same metric would be irrelevant or a lead generation website.
But there are thousands of metrics, how does a business know which metrics truly represent success of a product or service and which one to track and monitor.
Surrogation
Surrogation is a psychological phenomenon found in business practices whereby a measure of a construct of interest evolves to replace that construct.
Willie Choi, Gary Hecht, and Bill Tayler coined the term in their paper “Lost in Translation: The Effects of Incentive Compensation on Strategy Surrogation” The papers shows how managers tend to use measures as surrogates for strategy, acting as if measures were in fact the strategy when making optimisation decisions. This appears to occur even if a measure-maximising choice ultimately works against the strategy.
Here’s a common scenario: A company selects “delighting the customer” as a strategic objective and decides to track progress on it using customer survey scores. The surveys do tell managers something about how well the firm is pleasing customers, but somehow employees start thinking the strategy is to maximise survey scores, rather than to deliver a great customer experience.
An extreme example of surrogation was observer in the Wells Fargo Fake accounts scandal. Wells Fargo uncovers up to 1.4 million more fake accounts
Executives at Wells Fargo decided to track cross-sales to customers to measure performance on the bank’s strategy of building long-term customer relationships. The focus on cross-selling goals led employees to open 3.5 million accounts without customer consent, which, with brutal irony, severely damaged the long-term relationships the bank sought.
“Eight is great”. A manta that was the foundation for an aggressive cross-selling target scheme advocated by Former CEO John Stumpf. The scheme set the target for accounts per customer at eight. Employees had to reach “eight is great” targets in order to earn commissions and avoid termination. The scheme turned out to be a perverse incentive for employees to meet targets by opening fake accounts. Perverse incentives. Refers to incentives that have unintended and undesirable results. Some economists argue perverse incentives were partially to blame for the recent 2008 financial crisis.
Need for a KPI framework
Defining a robust measurement framework is the first step on the road to accountable and data-driven performance marketing.
A measurement framework defines what success looks like for your business. By connecting objectives to metrics, it becomes easy to understand how strategy is driving your business forward.
If a company’s metrics are set up well, there is a natural hierarchy and upward flow of impact. An individual achieving their goal should advance the team toward its goal,
TEAL Framework for KPIs
If you’re experienced with analytics, you’ve likely heard the term “north star metric” before. A single north star can be limiting and prone to surrogation snare. Businesses need to have a hierarchy of metrics that drive what call the Target metric that works in concert with other KPIs. Single-mindedly maximising your Target metric can hurt your business. Target metric should be the top priority, not the sole priority.
Engagement & Activation metrics should either directly contribute to the Target metric or act as a check to make sure the product or service is growing in a healthy direction.
Leading indicators give early indications of performance. These indicators “lead” to results by showing the progress you’re making toward your goal. Typically, leading indicators are metrics that will help keep you on track so that you hit your strategic Target objectives.
The key metric categories
Reach
Reach is the total number of people who have used the product or service in a recent time period. These can be existing users who have logged in / opened the app in a week or new users who have registered or downloaded the app. Reach is important because it represents the maximum amount of users who could reasonably become active, whether organically or through campaigns.
Activation
How many website have you created an account and then never really used any services. To successfully onboard a new user we need to ensure that we activate their experience on the platform. Activation is a foundational step that primes a new user to become an active user. Facebook identified adding 7 friends in 10 days as their activation metric when they were a start-up. They found it was a key milestone in driving long-term usage.
For other types of products, activation could be defined as registering, making a first purchase, viewing five videos, or making two deposits within a specified time period.
Active users
Active users are people who have taken a key action and realised value from your product or service in specific timeframe. Value could be defined as an action, posting a 2 photos or messages, or a set of actions, such as 5 likes or 3 comments on a post.
Products that promote habitual usage, such as Twitter or Instagram, look at DAU (Daily Active Users). Business software is better viewed through the lens of WAU (Weekly Active Users) since it’s not always used every day, especially not on the weekends. MAU (Monthly Active Users) would be a good fit for a bill payment portal, since bills are usually due monthly.
Engagement
Engagement is different than active usage because it measures a deeper level of commitment to the product or service. It accounts for both the frequency and involvement of users in your product or service to complete key actions.
Engagement could be defined as the number of key actions taken, minutes spent browsing or number of posts liked or transactions completed.
Retention
Retention is the metric that shows how sticky is your product or service. Think about what drives retention in two ways: How can we make existing people stick around and How do we attract users that will stick around longer.
When deciding on time frame for retention goals, pick a range that is long enough to capture the reasonable repeat visit cycle of your customers, yet short enough that teams can get feedback to iterate quickly. Length of time window depends on the type of product or service.
Conclusion
To avoid traps such as surrogation and perverse incentive, track multiple metrics of a strategy rather than just one. This approach highlights the fact that no single metric completely substitutes the strategy.
Also it is important to remember that the goal of measurement isn’t just to track change over time; it is to effect change as well with rigour, discipline, review and reflection.
If you’re using performance metrics, surrogation is probably already happening even without a perverse incentive, it is enough to induce some level of the behaviour. So it’s time to take a hard look internally to see which metrics are alined with your strategy and which might be prone to surrogation and if we are measuring what really matters.
