In this series…

• Part 1: WTF is a Metric? — introduction to the basics of metrics, what it is, and why it matters.
• Part 2: 4 Steps to Defining GREAT Metrics for ANY Product — how to develop a metric from scratch.
• Part 3: WTF is Your Metric Doing to Your Customers? — how to avoid the unintended consequences of the “one metric that matters.”

As a Product Manager, picking the right metrics may be the single most important thing you’ll do for your product. Having the right metrics can mean the difference between success and failure, not least is the fact that they MEASURE your success. The metrics will also become the ball you obsessively keep your eyes on. Metrics will likewise drive your team’s priorities, values, and culture.

You may find yourself asking this question:

“What does success look like?”

Every single strategic initiative or decision I have encountered has been accompanied by discussions addressing this question. We fundamentally understand success to be strongly correlated with setting achievable goals with concrete, quantifiable metrics. After all, we can’t improve what we don’t measure. Metrics—whether they be revenue, customer satisfaction, or engagement—become the driving force of our success.

For all their importance, how do I pick the RIGHT metrics? In the course of trying to answer this question, I realized I had to first define what a metric was. It was at this point I also realized that not once in my entire professional life has anyone explicitly told me what a metric is.

What constitutes a metric?

To most, the definition of the term “metric” may seem obvious, even trivial. However, when I attempt to seek out a clear and established definition of what a metric is, I am not able to find anything sufficiently substantive. Some resources related to the topic will mention the term’s definition, but only vaguely and casually in passing. Searching mainstream references like Dictionary.com and Wikipedia only resulted in lengthy articles about the metric system. Most relevant articles will mention something about quantitative measure or performance. They generally leave me desiring more depth and precision.

So, lacking an authoritative definition, I will, here and now, attempt to establish the meaning of the term:

metric

/ˈmetrik/ noun

In the broadest terms, a metric is any collectable, quantifiable measure that enables one to track the performance of an aspect of your product or business over time.

With the goal of being clear-cut and exhaustive about this definition, I would also like to be explicit about what I mean by each of the phrases used in this definition:

• “…collectable, quantifiable measure…” This means a metric must be a value you can somehow collect by instrumenting your product.
• “…track the performance…” You can measure a lot of dimensions of your product, but not all of them are tell-tale of its performance. Metrics must measure components of your product that are relevant to the product’s success (more on this in a subsequent part of this series).
• “…an aspect…” It’s important to note that rarely does one single metric give you a sufficient look into the health of your product. A product will typically have multiple metrics to represent its various aspects, and this suite of metrics together paints a complete picture.
• “…over time.” Metrics should capture changes in your product over time to help you set baselines with past data, to deduce trends, and to enable projections.

For example, a common metric called lifetime value (LTV) is calculated based on two commonly collected data points: number of payers and total amount in revenue they pay over their lifetime. Per the definition above, this metric tracks the performance of your product over time.

How are metrics special?

You may ask yourself: “Based on that definition though, can’t any number be a metric?” Not exactly. To further understand what a metric is, it’s also important to understand what a metric is not:

A “metric” is not an “analysis.”

This is a hard thing to pick apart, because you can analyze a metric, and metrics invariably contribute to your analysis. However, a metric is most commonly concerned with a high-level measure that provides a pulse to your product.

In contrast, an “analysis” (in quotes because I’m yet again referring to the specific context of a very broadly defined word) can any numerical value that is derived to provide insights beyond monitoring performance over time. An analysis can quantify as a trend in your metric for a specific period of time (e.g. LTV has on average increased 5% week-over-week for the past 3 months) or a comparison of your metric across different dimensions (e.g. users who sign up during the weekend are 1.28x more likely to buy during their first visit).

Metrics also tend to be simple combinations of data, i.e. one or two arithmetic transformations from raw data, whereas analysis are typically more involved and sometimes will require intensive calculation or statistical correlation.

A good mental model to apply when distinguishing between a metric and an analysis is this:

A metric will tell you that something is happening, while an analysis will tell you why something is happening.

If your body is a product, your body temperature, heart rate, blood pressure, and glucose levels would be metrics. If any of those metrics appears out of the ordinary, you would then run analyses, like blood tests or CT scans, to ascertain why

Check out “part 2” of this series, where we discussed HOW metrics developed to ensure a success of a product!