Real growth vs vanity metrics

Every early company produces numbers that go up. Sign-ups, page views, downloads, followers, pipeline. Most of them mean nothing, and the dangerous part is that they feel like progress. This playbook is a short guide to the difference between growth you can build on and growth that evaporates, told mostly through founders on The Product Market Fit Show who had the vanity kind, recognised it, and gave it up.

What 20 million users taught Keith Peiris

Tome, an AI presentation app, raised a $43M Series B and reached 20 million users. Retention was terrible, and almost nobody cared enough to complain. Keith Peiris turned the product off before he knew what he would build next, cut the company from 70 people to 7, and started over. The new company, Lightfield, an AI-native CRM, has since closed a $45M Series A, and its customers light up the team's Slack at three in the morning. His view now: angry customers are a better signal than 20 million happy ones, and there are businesses that work but will never re-inflect, however hard you optimise the metrics. Keith Peiris, Lightfield

He is not alone.

• Fathom. Rich White got 100,000 sign-ups in 30 days from the Zoom marketplace. About 99.9 percent churned. Most of them did not even have meetings on their calendar. Two years later, after selling a team plan before it was built, Fathom passed $10M ARR. Rich White, Fathom
• Gamma. After its AI pivot, Jon Noronha's company went from hundreds of sign-ups a day to 10,000, then 50,000 in a single day. Most did not stick, and most of the people who paid did not stick either. The term "AI tourist" came out of that period. What turned the company was not more sign-ups but the retention of the ones it had. Jon Noronha, Gamma
• Juicebox. David Paffenholz had a consumer app with 50,000 users and real virality. He shut it down because the retention was not as good as what he had seen working at Snapchat. Two years later Juicebox, an AI recruiting product, was at $10M ARR with a $30M Series A from Sequoia. David Paffenholz, Juicebox

Most people who signed up didn't actually stick with the product, and most people who bought didn't stick with the product. There was an incredible flood of initial interest that was barely manageable, but it was a very leaky bucket.

What makes a metric vanity

A metric is vanity when it can go up while the business gets no better. The test has three parts:

• Does it measure an action a customer took that benefits them, or an action you took? Sign-ups measure your marketing. Weekly active use measures their behaviour.
• Is it a total, or a rate? Totals only go up. Cumulative sign-ups, total revenue since launch, all-time users. Rates can go down, which is why they are useful.
• Could you make it go up in a way that hurts the company? If a bigger ad budget moves it without anything else changing, it is not telling you about the product.

Cumulative charts are the classic offender. A chart of "total users" rises even while the business is shrinking, as long as one new person signs up.

One caveat from the show. Michel Tricot at Airbyte, which raised $150M before it had revenue and then hit $1M ARR in four months, defends GitHub stars for open-source companies. They tell a developer the project is real, and that confidence is worth something. Context decides. Even then, revenue needed a separate motion. Michel Tricot, Airbyte

Revenue can be vanity too

The subtler trap is revenue that looks fine and is not.

Revenue from anyone. Adam Robinson's first company flatlined at $3M ARR. Churn was 15 percent a month because the company had sold to anyone who would sign up: lead-gen firms, publishers, B2B, e-commerce. He calls that growth curve unfixable once you are on it. When he spun out Retention.com with a narrow customer base, demand was, in his word, ferocious. The company now does $25M ARR and $14M a year in profit with no outside funding. Adam Robinson, Retention.com

Revenue in a market that is too small. Kevin Tian took Doppel to $1M ARR selling anti-fraud tools to NFT projects, looked at the size of the market, and pivoted the whole company into cybersecurity. The first post-pivot contract was $5,000 with no product. Doppel has since raised more than $120M. Kevin Tian, Doppel

Revenue from the wrong customers. SolidRoad closed sales teams, then watched them churn. Real usage was showing up in customer support organisations instead. Mark Hughes fired the sales-vertical customers and followed the usage, and got to $1M ARR and a $25M Series A. How SolidRoad found its ICP

Revenue that is slow and steady. Jay Madheswaran was running a respectable AI company at $3M ARR. He fired all of its customers and rebuilt as Eve, legal AI for plaintiff lawyers. The signal that the new market was different: 40 percent of cold outreach converted to a demo. Eve was a $1B company in under two years. Jay Madheswaran, Eve

Revenue from one customer. Dan Novaes grew Mode Mobile's revenue by 32,481 percent in three years and topped the Deloitte Fast 500. Then his crypto advertisers went bankrupt, revenue fell from $25M to $8M, and he cut half the staff. What nearly killed the company was concentration, not growth. Dan Novaes, Mode Mobile

The metrics that predict PMF

For a B2B product before and around product-market fit, five numbers tell most of the story. Pick these, define each one in a sentence, and look at them every week.

1. Activation rate

Of the accounts that started, what share reached the moment where the product did the thing it promises? Define the moment precisely: "imported their first 100 invoices and approved one", not "logged in twice". Fathom's 100,000 sign-ups would have failed this test on day one, because most of them had no meetings to record. Sean Ellis, who ran growth at Dropbox and invented the PMF survey, puts activation first for a reason.

Tactical churn-reduction efforts come too late. By that time you're dealing with people who've given up on the product. Activation is what most likely leads to long-term retention and referrals.

Under 30 percent usually means onboarding is broken or the segment is wrong. Over 60 percent is strong.

2. Retention, by cohort

Of the accounts that activated in a given month, what share is still using the product each month after? Plot it as a curve for each starting cohort. This is the single most honest number a company has. It cannot be bought, and it flattens before revenue does. Zach Lloyd at Warp had hundreds of thousands of free users and no revenue for three years. He tracked one thing before spending on marketing: whether the retention curve would flatten. Once it did, and he pivoted into AI development tools, the first $1M ARR took 300 days and the company now adds $1M ARR every ten days. Zach Lloyd, Warp

3. Conversion against the benchmark

For a self-serve product, what share of free users pay, compared with the norm for your category? Tanay Kothari spent years on brain-wave interfaces before pivoting Wispr Flow to voice dictation. Free-to-paid conversion came in at 20 percent against a freemium benchmark of 3 to 4 percent. A ratio five times the benchmark is a real signal; a raw user count is not. Product-market fit after a pivot

For a sales-led product, the equivalent is demo-to-close. Matt Watson, who hosts Product Driven, says a demo-to-close rate above 40 percent on deals of $1,000 to $20,000 is a clear signal, though he is careful about what it measures.

Demo to close is less a measure of product market fit and more a signal of message market fit. You're saying something that people want, then they actually get the product and use it. Retention is the best long-term metric, but takes time.

4. Net revenue retention

Take the customers you had twelve months ago. What is their revenue today, including expansions and net of churn and contraction, as a share of what it was? Over 100 percent means existing customers are growing faster than they leave, and the business grows even with no new sales. SolidRoad's is 186 percent. Under 80 percent means you are pouring water into a leaking bucket. At early stage, measure it quarterly.

5. Sales efficiency

For every dollar of sales and marketing cost, including your own time at a fair salary, how much new annual recurring revenue did you add? Below 0.5 early on is normal. If it does not improve as you learn, the motion is not repeatable yet.

Set up the measurement in a week

You do not need a data team. You need event tracking on five to ten actions, a spreadsheet, and discipline.

1. Define the core action in one sentence. The thing a customer does that means they got value.
2. Instrument five events: account created, activation reached, core action performed, key secondary action, account cancelled. A free analytics tier or a database query is enough.
3. Build one cohort table: rows are the month an account activated, columns are months since, cells are the share still active.
4. Add a revenue sheet: monthly recurring revenue by customer, so you can compute new, expansion, contraction and churn each month, and see how much of the total sits with your largest account.
5. Write the definitions down and do not change them without noting the date. A metric whose definition drifts is a vanity metric with extra steps.

Reading the numbers without fooling yourself

• Look at rates and cohorts, never totals. If a chart only goes up, it is not helping.
• Segment before you celebrate. A 60 percent activation rate that is 90 percent in one segment and 20 percent in another is a message about who to sell to. That is the SolidRoad lesson.
• Look at who, not just how many. OpenAI signed up for LiveKit Cloud with a personal Gmail address and quietly built ChatGPT's voice mode on it, while the company was deciding whether to sell a licence or shut down. The customer that mattered was invisible in the aggregate. LiveKit is now a unicorn. Startup failure and recovery
• Compare against your own history, not against benchmarks from companies at a different stage.
• Write down the prediction before you look. "I expect month-three retention for the July cohort to be 55 percent." If you are consistently wrong in one direction, your model of the customer is wrong.
• Count customers, not logos. One enterprise pilot that is not yet paying is not a customer.

Vanity in disguise: the sophisticated versions

Experienced founders avoid the obvious traps and fall into subtler ones.

• Blended numbers. An average retention that mixes two segments hides that one is dying.
• Annual contracts counted as retention. A customer on a twelve-month contract who stopped using the product in month two has churned; you just have not been told yet. Track usage, not billing.
• Pipeline weighted by "probability". The probability is the rep's mood. Track stage conversion rates measured from history instead.
• Growth from a single customer. If one account is 40 percent of revenue, the growth chart is that account's growth chart. Ask Mode Mobile.
• Engagement from your own team. Exclude internal and test accounts from every metric, always.
• Survey scores. Net promoter score is a conversation starter, not a predictor. Watson's advice on the show is to prefer the Sean Ellis question, "how would you feel if you could no longer use the product?", and treat 40 percent "very disappointed" as the bar. Then check it against retention, which is the real answer.

When the numbers say pivot

Founders who pivoted rarely regret it. Founders who did not pivot often wish they had.

Most founders report wishing they had pivoted sooner rather than pivoting too fast, and the challenge is in the grey area where you have some traction but don't have true product-market fit.

Pablo Srugo's test for that grey area: until PMF, every day is day zero. If you walked into your company fresh today, with the same team, cash and product but none of the work behind you, would you keep going? If not, pivot.

The show's own guide to the decision draws the line between direction and effort.

The decision is about direction, not grit. Pivot when the evidence says your direction is wrong; persevere when it says your execution just isn't there yet.

The evidence usually looks like this: the market keeps pulling you somewhere other than where you are pointing. Chris Ellis at Thatch pivoted after seven of eight interested prospects asked for a different product than the one he had built. The new product grew 80 times and raised a $38M Series A. Rob Boukine at Noibu had 15 customers and $3,000 in monthly revenue after two years of selling 3D storefronts; discovery calls with those customers surfaced checkout errors as the real pain, and Noibu now does tens of millions in ARR. Chris Ellis, Thatch

And when a pivot is right, the numbers say so fast. Marty Krátký-Katz at Blockthrough was pre-revenue two and a half years in, with 29 days of cash. The fourth version of the product shipped in March 2018 and was doing more than $100,000 a month within eight months. He bootstrapped the company to a $100M exit. Marty Krátký-Katz, Blockthrough

What to show investors, and what to show yourself

Investors at seed have seen every vanity chart. The ones that build trust are the ones that could have gone down: cohort retention curves, net revenue retention, sales efficiency, revenue concentration, and a clear account of churned customers and why. Showing a number that could have gone down and did not is worth ten that could only go up.

Show yourself the same numbers, plus one more: the list of the last ten customers who stopped using the product, with the reason in their words. That list is where the next quarter's roadmap comes from.

Vanity metrics are not lies. They are just answers to questions nobody should be asking. Ask better questions, measure the answers honestly, and the numbers will tell you where product-market fit is before it arrives.

Episodes and posts cited

• Keith Peiris, Lightfield, and the Tome shutdown
• Pivot vs persevere
• Product-market fit after a pivot
• Startup failure and recovery, including LiveKit
• Jon Noronha, Gamma
• David Paffenholz, Juicebox
• Michel Tricot, Airbyte
• Adam Robinson, Retention.com
• How SolidRoad hit $1M ARR, Mark Hughes
• Jay Madheswaran, Eve
• Sean Ellis's step-by-step growth guide
• Zach Lloyd, Warp
• Matt Watson: how to tell if you have true product-market fit
• Chris Ellis, Thatch
• Marty Krátký-Katz, Blockthrough