Measure by Market: Don't Fool Yourself with Global Averages

Consider a composite scene. A brand selling smart pet feeders into the US, Germany, and Japan opens its quarterly AI visibility review. The global mention rate is flat, and the recommendation is to hold budget. The Germany lead filters the same answers to German-language questions and finds the brand missing from every one for eight weeks. Japan has never reached a shortlist. The global line held because the US line rose far enough to cover both holes.

Per-market measurement is not one report cut into smaller reports; it is the admission that each market has its own denominator, language, and evidence base — and while a single global average allocates the budget, a failing market stays hidden behind a growing one.

A global average fails in two ways

The first failure is weight. The market with the most sampled answers decides the shape of the curve, and one large market’s swing can cover every change in two small ones. Weighted by answers you get one “global” number; weighted equally by market you get another, sometimes further apart than the movement you are tracking. Both are defensible. Publishing one and deciding on the other is not.

The second is mix. Even when every market deteriorates, shifting traffic and budget toward a market with a higher baseline lifts the average. You need two lines — movement inside markets, movement of the mix — or optimization and a portfolio shift look identical.

The judgment: a global number belongs in the board deck, not the budget meeting. Split first, then ask whether rolling up tells you anything.

A translated prompt measures your translation

A German buyer asks about certification, a tax-inclusive price, and the statutory withdrawal window. A Japanese buyer asks about dimensions, noise at one meter, and where the service center sits. A US buyer asks who it beats on price. Translate the headquarters prompt list and you get questions that are grammatically perfect and locally implausible: the measurement then describes your translation, not demand.

The ecosystem differs too — which assistants buyers open, which local publishers get retrieved, how much weight in-platform AI carries. Prompt sets have to grow out of support tickets, site-search logs, review sections, and the local competitor pages that keep surfacing in answers. Local buying language covers what shifts inside the questions.

Then sampling. Print n and the interval beside every percentage. Below roughly 30 answers a month, write “not decidable” instead — 30 is a working threshold, not an industry standard, and its job is to keep a nine-answer cell out of a budget decision. Repeat each question in one window on one assistant version, the way an AI visibility baseline does.

Five steps, and the order is the method

Step	Action	Owner	Passes when
1 Lock the definitions	Markets, languages, assistants, metrics, denominators, on one page	Growth, analytics	A second analyst reproduces the number without asking
2 Build local prompt sets	15–20 questions per market, per stage, in the local language, sourced locally	Local ops, content	No translated phrasing; each question maps to a page
3 Fix the sampling	Same window, same assistant version, repeated questions	Analytics	n and interval printed in every cell
4 Normalize money	One reporting currency, dated exchange rate, tax basis, shipping, returns	Finance, analytics	Orders add up across markets and come apart again
5 Layer the board	Market × stage: mention, citation, shortlist, order	Growth	Movement inside a market is separable from mix movement

Step 1 also decides which markets earn a line: too many thin every cell, so start with the two largest by revenue plus one pilot, ordered by market entry variables rather than by which lead argued hardest. Step 4 is the one teams skip. Yen, euros, and dollars in one column, tax-inclusive mixed with tax-exclusive, and the resulting “AI revenue” explains nothing. Date the rate, state the tax basis, keep shipping and returns separate — otherwise you are comparing accounting habits.

If the language versions share a domain, confirm with the hreflang matrix that each market resolves to its own language page. Point it wrong and every number downstream measures another page.

Example: a 31% global rate, three different markets

Example — the brand, markets, and figures are invented; no client data is described.

Market	Answers sampled	Mention rate	Top-three shortlist	Read
US	350	40%	18%	Hold; keep testing comparison pages
Germany	100	12%	4%	n is adequate; check page and language first
Japan	50	6%	2%	Not shortlisted; rebuild local prompts
Global roll-up	500	31%	14%	Pulled up by US weight

The roll-up is not a lie. It hides that two of three markets sit at 12% or below, and that 70% of the sample is American. Weighted equally, the same table reads 19% for mentions and 8% for shortlist — a spread produced entirely by the denominator. The read is not “keep spending”; it is US hold, Germany diagnose, Japan rebuild.

Keep the stages apart too. Mention, citation, shortlist, and order are four positions, not grades of one figure: mention responds to crawl and content, shortlist to comparison pages and local trust. Per market, one can rise while the other falls, and the pair is the finding.

Where this breaks

Five situations. One market, or a second that contributes a rounding error: split by language and channel, skip the country board. Sample-starved markets: under 30 answers a month, monthly percentages are noise — aggregate quarterly and report direction. Markets sharing a language but not a currency, tax basis, shipping cost, or withdrawal window: the US, UK, and Australia can share English prompts, but “the English market” is the same trap one level down. In-platform AI and agent transactions: the platform’s report is the definition and cannot be recomputed from outside, so record “not measurable here”. Attribution gaps: AI referrals land in Direct or none, so per-market AI revenue is a floor, read beside leading indicators.

There is a sixth, political one. If each market lead picks the metric that flatters them, the board becomes a bargaining table. Central owns the dictionary; regional teams may add prompts, never change a denominator.

This month, recompute the last visibility report by market. Put n and a currency behind every percentage, replace each cell under 30 answers with “not decidable”, then take the market with an adequate n and the steepest decline, rewrite ten prompts in its language, and retest at the same hour in four weeks. The rest of the line is on the measurement and growth hub.