Shortlist Share: A Metric More Important Than Rank

A DTC pool-robot brand sells into the US and Germany. At its July review, one chart: average position in summarized AI answers moved from 3.8 to 3.2. The founder asks a narrower question — of the answers to “which robot cleans an above-ground pool under $1,000 with heavy leaves,” how many name us? The saved answers say none. The brand is climbing in answers it never needed to win and missing from those nearest a cart.

Shortlist share is the share of answers to frozen buyer prompts — one category, one market, one engine — where your brand makes the three-to-five-name list; it beats rank because rank only averages the answers you already won.

Rank is an average of the hands you won

An AI answer is a generated list, not a ranked result: the model fixes budget, setting, and region, then picks a few names and a reason for each. One week four names, the next seven; move a budget tier and the cast changes.

Position is not comparable: second of four and second of nine differ. Worse, the mean covers only answers where you appear; absent answers never enter the denominator. The cheapest way to raise it is to appear rarely and rank high when you do — the opposite of the goal.

Keep rank as a secondary column: order inside an answer, not presence.

Three numbers, three denominators

One percentage cannot carry the diagnosis. Inclusion says whether you are present, seat type says why, reason rate says whether the mention could persuade.

Measure	Formula	Question	Common error
Inclusion rate	Answers naming you ÷ valid answers	Am I on the list?	Counting answers with no brand
Seat type	Tag each inclusion: default, spec, scenario, budget, local	Why am I on it?	Logging presence, not reason
Reason rate	Inclusions with a concrete reason ÷ inclusions	Could it persuade?	Accepting “also offers similar products”

Three rules govern the denominator. One answer counts once, even if you appear twice. A brand folded into “other brands also offer similar products” is not a seat; that is how coverage looks real and reads as filler. Answers naming no brand stay out; they measure how-to content.

Fix seat share for the quarter and never blend it with answer-level share. Reason rate points to action: high inclusion with a low reason rate means you are the fifth name padding the list — an evidence problem, not exposure.

Slice by shelf, or the average lies

Compute by category × market × engine, with a row per price band. Local stock, certification, returns, and plug standard change the reason a model gives, so one product can hold a seat in one market and lose it in another for a reason no page fixes. A blended total survives the meeting and changes nothing.

Step	Action	Owner	Passes when
1 Define the shelf	One cell per category × market × engine; price bands split	Growth	Language, currency, country on the cell
2 Freeze prompts	15–20 prompts by budget, scenario, comparison; 4–6 rivals fixed	Growth	Adding one restarts the quarter
3 Fill and archive	30+ valid answers per cell; save date, region, names, reason	Ops	Old answers reopen

Thirty answers is a working floor, not a guarantee. Store the count beside the percentage, or a later doubling cannot be told from a shrinking denominator.

Example: one product, two markets

Example — brand, markets, and figures invented; only the slicing logic is reusable. Brand B sells one cordless pool robot in the US and Germany, four prompt groups, 20 answers each.

Prompt group	Germany incl.	US incl.	Seat
Budget (under €900 / $1,000)	15%	5%	Budget
Scenario (above-ground pool, leaves, sand)	35%	20%	Scenario
Comparison (two corded models)	10%	0%	None
Category (best pool robot)	20%	15%	Spec

Blended, the cells read 20% Germany, 10% US: Germany fine, US weak, meeting over. Split, one cell deserves the next dollar — scenario plus Germany. It holds a seat but lacks test conditions: heavy leaves and fine sand, as checkable facts. Comparison at zero in the US is not a copy problem but the absence of a like-for-like spec table, the work of machine-readable comparison tables. The 5% US budget cell needs another check: if local stock and delivery time keep the model from committing, fix fulfillment.

How to read the wobble

First, measure list churn. Compare two months of name sets: if two of four names changed, the answer has not settled and the share move cannot be attributed to you. If the set held, an edit can be read.

Second, separate three kinds of drop. Absent means category recognition or comparable fields are missing. Present without a reason means evidence is missing. Present as “the cheaper option” means price band and positioning are talking. Each repair differs; blended, none is visible. This is not share of citation, which asks whether a page was used as a source: different unit, different denominator.

Third, interpret only movement outside the noise band. Three to five points on a small sample is usually noise; freeze the prompts and repeat.

Where this breaks

The category is too small: a handful of real questions a month makes the denominator unstable; keep a qualitative log.

In-platform AI does not reproduce: record whether you appeared and why, never a share.

The brand is not recognized: inclusion is zero and arithmetic will not fix it.

Sales happen mostly inside a marketplace: on-site and listing inclusion are different objects.

It becomes a KPI for content: the list is finite, so a target pushes writers into short answers with empty reasons.

This line belongs to measurement and growth; its output is which cell to fix. The list forms in AI shortlists only 3–5 names; challengers win cells for reasons in the challenger’s opening.

This month, pick one category, one market, one engine. Write 15 real buyer prompts, ask each three times, and log two columns: in the list or not, and the reason. Rerun in four weeks. If inclusion has not moved, wait: half the rivals changing means the answer has not settled; none changing means the gate is in fields and evidence, not article count.