Nearly every growth engagement we start begins the same way: within two weeks we find that the numbers the business has been steering by are wrong. Not slightly wrong. Wrong in a direction that has been shaping decisions for months.
The usual suspects
- An event that fires twice on one interaction, inflating a step by 40%
- A funnel definition that changed during a release and was never documented
- Bot traffic counted as sessions in one report and excluded in another
- Consent-gated tracking with the gap silently modelled and presented as measured
Why it survives so long
Because dashboards are load-bearing. Once a number appears in a board pack, questioning it has a political cost, and the person best placed to notice the discrepancy is usually the most junior person in the room. So the number persists, and eventually the business builds strategy on it.
Validate against something outside analytics
The only reliable way to check an analytics implementation is to reconcile it against a system with a different failure mode — your billing records, the CRM, the support queue. If the analytics say 1,200 sign-ups and billing says 940, the argument is over before it starts.
Then, and only then, optimise
Experimentation on untrustworthy data is expensive guesswork dressed as rigour. Once measurement is honest, a surprising thing tends to happen: the first two or three experiments become obvious, because the real drop-off is finally visible and it was never where the old funnel said it was.