We dominated the match. Seventy percent possession, six hundred passes, nine corners. We lost 3-0.
A launch review sounds pretty similar. Forty thousand visits the first week, two thousand sign-ups, a LinkedIn post with three hundred reactions. Applause, screenshot to the Slack channel, someone suggests ordering pizza. Three months later someone asks how many people are still using the product and there's that silence we all know, until someone else says "we need to look into how we measure that".
Eric Ries popularized the name, first on his blog and then in The Lean Startup: vanity metrics. They're the ones that make you feel good and don't help you decide anything. There's a pretty simple question to spot them: what decision do we make with this number? If nothing comes to mind, it's vanity.
There are examples everywhere. Total sign-ups, when what matters is how many people came back the second week. App downloads, when what matters is thirty-day retention. Landing page visits instead of purchases, followers instead of conversations. And my favorite: features shipped in the quarter, instead of what user behavior changed because of them.
The business metrics are the second half of each pair. Smaller and almost always more uncomfortable, but they're the ones that tell you whether you're winning.
Why do we like the other ones so much?
First, because many of them are cumulative. Total sign-ups can keep going up even if fewer people come back every month, and you can't see that on the slide. It's very tempting to show that chart and leave the uncomfortable question for later.
And second, because they're safe. Nobody asks you for explanations about visits. They ask when retention drops, and for that someone first has to dare to look at it.
Still, there's a logic to possession. A team that has the ball most of the match has a better chance of scoring. The drama starts when having it becomes the goal and the team starts passing backwards so as not to lose it.
Metrics work the same way. Visits are useful if they help you understand why people don't convert. Sign-ups are useful if you follow them until you see who stays. The same metric can be a vanity metric or a business metric depending on the question you ask it.
And if on top of that you set a target on a vanity metric, Goodhart's law kicks in: the team finds a way to inflate it and then it tells you nothing at all. Of all the forms of quantophrenia, this is the friendliest one, because it comes with applause.
So next review, when that rising chart shows up on screen, ask what decision it helps us make. If nobody knows, it's possession. And possession is useful when it gets you closer to the goal, the rest is passing backwards.