Navigation for AI: Categories, Facets, and Pagination

Two weeks before peak season, a DTC outdoor-furniture seller shipping to the US and Germany watches a buyer in Hamburg ask a chat assistant for a folding table that fits a 24-inch balcony. The answer cites two competitors’ category pages; their own table sits on page four of the filtered results. Engineering checks the index: nothing dropped. The log says otherwise — the grid renders client-side, filters are query strings like ?material=aluminum&width=60, and paging is one “Load more” button. The SKUs are indexed. The road to them is not.

AI does not lose your SKUs; it loses the road from category to SKU, and every filter combination and pagination layer removes another door a retrieval crawler could walk through.

A category page is an answer page; a facet is navigation

Stores treat the category grid as merchandising and filters as a feature. Retrieval reverses the order.

A crawler answering “which folding table fits a small balcony” fetches a handful of pages and needs one HTML list it can read in one pass: titles, prices, key attributes, who each product suits and who it does not. It will not click five dimensions to assemble that list. A facet serves humans already on the site; paths are entrances.

Three checks decide whether a category page is worth fixing, none of them organic traffic. With JavaScript off, do titles and prices appear? Does the page say how to choose within this category? Is the homepage link a real anchor, not a scripted menu item?

The facet trap: combinations, not URLs, are the cost

Faceted navigation splits material, width, use case, and color into combinable filters. The trouble is not ugly URLs but that the entrance count multiplies: six values across five dimensions is 7,776 URLs before sorting and tracking parameters. Arithmetic, not a benchmark; a real catalog is messier. A crawler with fixed attention spends it on shallow, heavily linked pages and skips product pages already hard to find — the first bill in how AI crawlers allocate crawl budget.

Approach	What the crawler gets	Fits when	Cost
Parameter-only filters	Unlimited URLs from one template, most near-empty	On-site interaction only	Budget diluted; products sink
Path-based facets	One addressable URL per combination, self-canonical, room for an answer unit	A dimension buyers name, with supply to fill it	Manual whitelist; open too wide and you manufacture thin pages
Static category with curated entry points	A dozen flagship products and attributes in one fetch	Long-tail dimensions, seasonal edits	Incomplete coverage; selection needs refreshing

The test is one sentence: path only dimensions a buyer would say out loud — material, use case, size band. Margin, supplier, and launch batch are database fields; nobody asks that way, so no answer needs that page.

Pagination, canonical, and rel: three calls

First, a paginated page canonicalizes to itself. Page two holds different products; collapsing it tells the engine those products need not exist alone. The one narrow exception: a different sort order or page size over the same set.

Second, rel=next/prev does not solve discovery. It does not make page five findable or put its products in an index; clickable HTML links do. Rel tags shipped as a patch reopen the same ticket two weeks later.

Third, do not let “Load more” become the only entrance. Keep it for humans, keep a static pagination fallback so no-JS fetches reach the end; past roughly ten pages, let a sitemap carry the rest.

Variants double that arithmetic, so how variants and out-of-stock pages are handled has to be settled before you path anything.

A five-step playbook for one category

Step	Action	Owner	Passes when
1 Inventory	Export crawl records for category and filter URLs; split “returned products” from “returned a shell”	Engineering	Index-candidate and refuse lists, built as in a complete AI crawl audit
2 Whitelist	Path two or three dimensions buyers say; keep the rest as parameters with noindex,follow	Growth and content	Every path page carries its own answer unit, not a template sentence
3 Template	Server-render subcategory and top-product links; clickable breadcrumbs	Engineering	A no-JS fetch returns links and titles
4 Pagination	Fixed page size, static next link, self-canonical throughout	Engineering	Page two loads and is crawlable alone
5 Verify	Compare indexable, crawled, and 200-with-products URLs	Growth	Each gap maps to a template, scored as crawl coverage

Step two is where most teams break: opening every dimension leaves the long tail uncrawled and buries flagship products first. Ship two dimensions, run a cycle, then argue about the third.

Example: one category, three changes

Example — a hypothetical cross-border pet-supply store with six categories, five filter dimensions, six values each. No real brand, no performance figures.

Material and use case become paths: /cat-beds/waterproof/ and /cat-beds/small-space/. The other three stay parameters with noindex,follow. Each page holds a fixed 24 products, paging becomes static next and previous links, and every page canonicalizes to itself. At the top sits a short block on choosing in that category: who it suits, who should skip it, how to measure. It is written for extraction, not conversion.

Acceptance is three checks, not a dashboard: with JavaScript off, a path page shows titles and prices; from the homepage, a flagship product is three clicks away; re-run the same prompts and the cited page is the path URL, not the parameter one. If all three fail, do not roll into a second category.

Where this breaks

Below roughly fifty SKUs, a static category plus a sitemap beats path work; path facets there produce three-product pages, and the model reads “this category is empty.” Categories that convert inside a marketplace need listing, review, and feed work first; site navigation does not buy a platform recommendation. Where on-site search is the primary path, fix category names and attributes; opening results to crawlers creates unbounded URLs and empty pages. Multilingual stores multiply every path by language count, so settle hreflang and canonicals first. The “miss nothing” misuse — every combination open and self-canonical — spends budget on empty results and duplicate parameter orders while flagship products get rarer. The content and semantics hub collects the rest of this line.

This month, take the category with the most SKUs. Fetch it and its page two with JavaScript off; list the products you cannot reach. Convert the two dimensions a buyer would say out loud into paths, mark the rest noindex,follow, and re-run the same prompts. Decide on a second category only after that.