Measuring Agent Traffic: Sessions, Conversion, and Refunds

Late September, quarter close at a DTC brand selling gravel-bike racks into the US and Germany. Agent checkout went live six weeks ago, orders are climbing, and the deck proposes doubling spend. Finance opens the refund tab and stops: reasons cluster on “not as described” and “doesn’t fit my frame.” Support adds that tickets open with “which model did I actually buy?”

Agent traffic is not a new channel but a new order shape — shorter decision, less page seen, longer after-sales tail — and blended into organic it tells you to scale the queue that loses money.

Two traffic types share one referrer

A hit from an assistant may be a person comparing racks or a machine asking for a delivered price; averaged, the ratio describes neither.

Pre-check traffic is the machine: the assistant fetches the page, checks stock, prices shipping, and validates an address so its answer can carry a landed price. It arrives as a one-page deep link or a shipping API call, no add-to-cart, sometimes three times a minute.

Agent checkout is the person. They decided, and the agent finishes the purchase: the order is real, the session may not exist.

Two anomalies follow. Agent orders arrive without sessions, so any rate with sessions in the denominator drifts with agent share. Pre-check calls never order, yet dilute that same denominator until someone calls the channel low quality.

Type	Person behind it?	In the denominator?
Pre-check, comparison	No	No; count quote requests separately
Agent checkout, page	Yes	Yes, in its own column
Agent checkout, API	Yes	Cannot be; compute from orders

The API row is the one teams miss, and reaching it depends on agentic commerce readiness.

Find agent orders on the order record, not in the user agent

Identify agent orders from order-side evidence; the User-Agent string is optional, forgeable, and rewritten in transit.

Signal	Proves	Limit
Caller ID on the order API	Which system created the order	Exposure is your payment provider’s decision
One-time authorization or agent credential	Agent-initiated checkout, approved cardholder	Not every setup offers it; names no agent
Referrer, product landing, no navigation	Likely came from an AI answer	Proves the entry, not a watching person
Repeated address or quote calls	Machine pre-check came first	Rarely ties to one order

Three or more signals mean “agent,” one means “suspected,” and neither enters the main report. A mislabeled cohort costs more than a thin one — the first destroys trust in the whole measurement line.

Four metrics instead of one conversion rate

Judge the channel on margin after refunds; conversion answers “did the visitor buy,” never “did the order pay.”

Metric	Compute	Why the human baseline fails
Contribution margin per order	AOV − COGS − shipping − refund loss − support − fees	Human refunds are already baked into historical averages
Refund rate and reason mix	Refunded ÷ delivered, by reason	Clusters on “not as described” point at an information gap
Tickets per 100 orders	Tickets ÷ orders × 100	Agent buyers never scrolled; they ask what they bought
Repeat and return	Second order or return visit in 90 days	Separates one order from one customer

The reason mix is the diagnostic one. “Not as described” points at the product and quote layers, “doesn’t fit” at feed attributes, “thought I could return it” at policy. An agent that cannot read the return window, the return address, and who pays return shipping — US and German stores differ — folds the worst case into price and declines; one that reads them and finds them wrong sends the refund back as a dispute. See shipping and returns policies.

Example: one quarter, two ways to count

Example — brand, products, and figures are invented.

Metric	Human organic	AI referral (page)	Agent checkout (API)
Orders	1,000	120	60
AOV	$118	$123	$121
Refund rate	8%	11%	19%
Tickets per 100	6	9	22
Margin per order	$31	$28	$14

By conversion rate neither agent column looks alarming: page checkout lands on the product page where the decision is half made, and API checkout has no denominator at all. AOV differs by a few dollars. Add refunds and tickets and the reading flips — margin per order is less than half the human figure. Blending does more than misstate a number; it funds a losing queue, then invites the conclusion that agent buyers are low quality.

Five steps: tag first, merge second

Order matters; a dashboard built before the tags is another table nobody trusts.

Step	Do	Owner	Passes when
1 Tag	Three states — agent, suspected, not agent — from order-side evidence	Data, engineering	50 orders hand-checked; mislabels explained
2 Split the denominator	Pre-check calls logged as quote requests	Analytics	Both numbers recompute for one week
3 Merge	Refunds and tickets joined by order ID	Data, support	One table yields margin per order
4 Freeze the window	Write the refund window down; inside it, nothing counts toward targets	Growth	Definition fits one page
5 Review weekly	Chase only the widest gap to a specific break	Growth, engineering	Each review ends with one fix

Step 4 is easiest to fake. Refunds lag, short windows read low, and agent returns surface after unboxing. Without a fixed window you will keep seeing “refund rate fell” while refunds are still arriving. Step 5 needs restraint: one metric at a time.

Where this breaks

A dozen orders a month is not a cohort. One return doubles the rate; wait 90 days or pool by category.

Marketplaces rarely expose an entry point. Work with what the platform reports — ratings, return reasons, fulfillment — as in marketplace ratings.

The identifier belongs to a provider. Whether a caller ID appears is your payment provider’s call, and the rules move; fall back to order-side evidence rather than rebuilding checkout.

Attribution is not causation. Higher refunds may mean these SKUs are hard to fit; compare same-category human orders, not the site average.

This month, export 90 days of orders into three columns — human organic, AI referral, agent checkout — and compute two numbers per column: refund rate and tickets per 100 orders. Together they say whether the agent line earns margin or manufactures after-sales. Volume comes later: the agentic commerce pilot covers the test, and the measurement hub holds the definitions.