Your Dashboard Is Lying to You (Just a Little)
- saurav soni
- 1 day ago
- 4 min read
7 numbers that fooled me before I learned to double-check them
Quick story. I was looking at a retargeting campaign that had fired 15 "Purchase" events in 10 days. Solid number, right? Then I checked Shopify. Only 4 of those orders had actually been paid. The other 11 were cash-on-delivery orders nobody had paid for yet, plus one that got cancelled.
The tracking wasn't broken. The pixel was healthy. The number was just... measuring the wrong moment. "Purchase" fired the second someone clicked "place order," not when the money showed up.
This is basically my whole job: not finding bad numbers, but finding numbers that are technically true and still telling you the wrong story.
Here are 7 ways that happens, and what I do about it.
1. Ask "what actually made this number exist?"
Every metric is the result of some process you can't see. Before I trust one, I want to know that process.
Take "ROAS" as Meta reports it. It's real, but it's built from a generous 7-day click window plus some guesswork for iOS users — so it runs hot, consistently. I never look at it alone. I put it next to actual store revenue for the same days. The gap between the two tells me way more than either number alone. A growing gap usually means something broke, not that the campaign suddenly got better.
Takeaway: before you trust a number, ask what had to happen for it to exist.
2. "It's not working" isn't a diagnosis. It's a math problem in disguise.
Meta needs about 50 conversions per ad set, per week, to "learn" properly. Not a suggestion — basically a hard floor.
So before I blame creative or targeting, I do one piece of napkin math: daily budget ÷ realistic cost-per-result × 7. If that's nowhere near 50, the campaign was never going to learn — not because the audience was wrong, just because the budget was too small to ever produce enough signal. I've seen accounts running a dozen tiny campaigns, every single one quietly blamed for "bad targeting" when the real issue was just... not enough money to count to 50.
Takeaway: do the division before you write the theory.
3. Don't let anyone set a finish line your budget can't reach
A client once proposed a perfectly reasonable-sounding goal: 2 purchases a day, 5 days a week, at a target cost. Sounded fair. Except the daily budget could only produce about 1.3 purchases a day even in a best-case scenario. The test was rigged to fail before a single ad even ran.
This happens constantly, everywhere, not just ads. A target gets set, it sounds sensible on its own, and nobody multiplies it out to check if the system in front of them can physically hit it.
Takeaway: before agreeing to a goal, check if the budget can mathematically reach it. If not, that's the actual problem to fix first.
4. Don't bet it all at once. Bet it in stages.
A client didn't want to risk a big daily budget just to chase a conversion threshold — fair, that's real money. So instead of one big leap, we split it into small steps: ₹900/day → ₹1,400 → ₹3,000 → ₹5,000, each one judged on its own for a week before moving up.
Slower, yes. But you find out you're wrong for the cost of one small step, not the whole bet.
Takeaway: almost any "go big or go careful" decision has a staged version that teaches you the same thing for way less risk.
5. The name on the box is not what's in the box
An audience in this account was literally named "IG - Sales 365." Sounded like purchase data. I almost used it as-is. Then I opened the actual rule behind it: turns out it was just anyone who'd ever liked a post. Zero sales data in it. The name was just... aspirational.
Takeaway: names are claims, not proof. Open the thing before you trust it — especially if you didn't build it yourself.
6. Count what's actually in the bucket
I built a "Top 10 Bestsellers" product set for an ad. It had 106 items in it. Turns out each product had up to 7 size variants, and the catalogue counted every single one separately. The ad would've shown the same dress 7 times before showing a different product.
Cheap mistake to catch, expensive to miss. "How many things are actually in here" is one of the easiest checks to run and one of the most skipped.
Takeaway: a label says what something's for. A count tells you what it actually is.
7. Averages are where the real story goes to hide
Overall site conversion looked... fine. Mildly unremarkable. Then I split it by device: 99% of traffic was mobile, converting at roughly a tenth of the rate that desktop did. One blended number flattened a 12x gap into "meh."
Takeaway: any time a number blends two genuinely different groups (mobile/desktop, new/returning, cold/warm), split it. The average is almost never where the interesting decision lives.
So what's the actual rule here?
Honestly, it's just one move, done over and over: before you trust a number, ask what would have to be true for it to mean what you're about to assume it means.
Trace it back. Do the division nobody did. Open the rule instead of reading the label. Count what's in the bucket. Split the average until it stops hiding something.
And yeah — the catalogue mistake (#6) was mine, not a client's. That's the point of leaving it in. The rule isn't "don't trust their numbers." It's don't trust any number — including your own — until you've checked what's actually behind it.
If your numbers aren't adding up and you're not sure which ones to trust, this is exactly the kind of thing I dig into with clients. Book a free strategy call here: https://calendly.com/freelancersaurav11/free-strategy-call
Comments