Measurement after signal loss: three instruments, not one dashboard

For about a decade, marketing measurement had a comfortable answer: a multi-touch attribution model fed by third-party cookies and device identifiers, producing a single number per channel. The comfort came from the single number. Whether it was true was a separate matter, and the industry largely agreed not to ask.

Consent requirements, platform restrictions and walled-garden reporting removed enough of the underlying signal that the question can no longer be avoided. The useful response is not to find a replacement dashboard. It is to accept that three different questions need three different instruments.

The three instruments

• Experiments answer 'did this work'. Geo holdouts, matched-market tests, platform lift studies. Slow, expensive, and the only thing that produces a causal claim. Reserve them for decisions that are worth the cost of being wrong.
• Marketing mix modelling answers 'how should the whole budget be split'. It is coarse, it needs years of data, it cannot see a single creative, and it is privacy-durable, because it works on aggregates.
• Attribution and platform reporting answer 'what is happening right now'. Directional, fast, biased towards whatever is closest to the conversion. Useful for in-flight optimisation and dangerous for budget setting.

Attribution is a speedometer. Mix modelling is a map. An experiment is the only thing that tells you the road actually goes where the map says.

The calibration loop

These three are not alternatives to be argued over. They are a loop. Experiments produce causal estimates for a handful of channels. Those estimates calibrate the mix model, which allocates across everything. The mix model's implications get tested by the next round of experiments. Attribution runs continuously in between and is trusted only within the range the other two have validated.

Run that loop for four quarters and the arguments about which platform is lying largely stop, because you have an external reference. Most organisations never get there, not because the method is hard but because the first two quarters produce no new dashboard and are therefore hard to fund.

The honest caveat

This is more expensive than the thing it replaces, and it produces fewer, slower, less precise-looking answers. It is worth it only if the decisions are large enough. Below a certain spend, the correct measurement strategy is a clean conversion setup, one good holdout a year, and the humility to admit the rest is noise.