How Startups Get Their Metrics Wrong

Y Combinator is hosting two Demo Days this week for startups in its latest batch to pitch throngs of investors. As is typical, some begin bringing investors on board before the rush, and so I have had a series of hurried meetings this past week with many of them.

By and large, the teams I have been meeting are excellent and inspiring. The level of talent seems to be increasing cycle to cycle. That said, in rapid-fire meetings with these and some other young companies over the last few weeks, I have been surprised how few can reduce their mission to a single understandable and achievable metric.

When it comes to pitching metrics to investors and setting goals for their businesses, founders make several mistakes over and over again. (To discuss this column with me and other subscribers, tune in here Monday at 9 a.m. PT.)

1) Irrelevant Metrics of Scale

Companies continue to set goals around their monthly or weekly active users. But those who do so are either hiding something, are uninformed or are optimizing for the wrong thing. In a mobile world, where consumer apps are always available and need to be highly engaging to hope to scale, all that really matters is daily active users (DAUs) or engagement measured by daily active users divided by monthly active users.

But even DAUs can be deceiving. Recently I have seen several companies grow to hundreds of thousands of DAUs from zero in weeks. This is impressive if truly organic; however, frequently when you see this pattern it is from promotion in app stores or unsustainable viral flows, so it is something investors and people managing business need to be leery of and dig into further.

Companies also make mistakes by targeting a certain number of deals or integrations. If there is some reason that achieving a certain number of deals or integrations would denote meaningful progress, this would be valid. For example, if going from 10 deals to 100 or 500 would require some massive technology or approach change, the number would reveal a certain type of progress. But many companies play up integrations without explaining how they drive their ultimate goals.  

2) Metrics You Can’t Directly Control

The second mistake people frequently make is picking metrics that are highly sensitive to things beyond their control.

Perhaps the easiest example of this is companies in the energy industry whose models and goals are highly dependent on energy prices. There is nothing wrong with banking on future pricing if you have a strong opinion about it and there is some consensus about what will happen; however, a metric you don’t control is a metric you can’t manage your team around. That means there is little for your team to do but pray, which is not usually a good use of funds. Another example would be a company that was betting too heavily on the the price of Bitcoin or currency exchange rates.

3) Complicated Metrics

The third metric mistake people make is just having too many. When you have too many metrics you almost always end up in a world where you don’t know what to optimize for. That doesn’t mean you can’t have a metric and a counter metric you are trading off against. I have seen that work well, like a subscription business that wants to acquire new users from new channels without increasing churn.

That said, companies frequently say in the next period we want to grow DAU to X, bring on our first Y customers, and get revenue to Z.  

Even if multiple metrics don’t directly conflict, at the end of the day, focus is ultimately a tradeoff. Having complicated metrics means you and your team don’t know where to focus or what is the most important thing to do moment to moment. If you feel like you must have multiple metrics to hold yourself to, then either you aren’t being draconian enough about what needs to happen. Or, frankly, you have too many different things that need to work for your business to succeed, and it therefore probably won’t.

4) Money, the Ultimate Metric

My favorite metric is money. Within reason, it is very hard to cheat yourself or others on. There are lots of ways to artificially close deals or get users, but artificially getting more revenue-per-unit is pretty difficult. The other nice part about money is that it is the currency that everyone else gets paid on. It is completely normalized, so targeting money and its direct drivers effectively ends up yielding you the currency you need to build your business.

Historically there was a pretty strong argument that money wasn’t the right initial target for companies because making money online was tough while ads and payments were immature. And there are plenty of cases where it still isn’t, like mass consumer platforms that need hundreds of millions of people to become viable businesses. But as it has become easier and easier to accept payments online and people have become more and more willing to pay for all sorts of digital goods, I think companies have fewer excuses for not being able to justify their business based on revenue and profit.

Conclusion

I know finding the right metrics is easier said than done. Personally, I have spent years working on projects where picking the right metrics was a constant battle because there was no clear, agreed upon way to evaluate success even if there was broad alignment on theoretical goals.

But at the same time, I think companies truly have to. If you have a smart and dedicated team—and can define your metrics of success clearly—you can use your resources efficiently. You can liberate people to make good decisions without massive overhead and debate.