GEO-020·Issue 02·Content & Semantics·20 / 132
Beyond hreflang: Crawl Consistency for Multilingual Stores
hreflang validates clean while every German URL stays unfindable. Align language paths, canonical tags, and sitemaps so AI answers cite the right locale.
Issue focusGet Seen First: Crawling, Rendering, and AI Bot Accessibility
Eight weeks after a US cycling-gear brand opens a German storefront, a teammate asks a chat assistant, in German, which waterproof backpack suits a daily commute. The answer names three brands; none is theirs. Engineering checks hreflang first: zero validator errors, every reciprocal tag present. The problem sits elsewhere: German pages canonicalize to English, the switcher is a JavaScript dropdown, the sitemap lists English URLs only.
hreflang declares which pages correspond; it never makes one reachable — when language path, canonical, and sitemap disagree, the model keeps the English page.
hreflang is a declaration, not a path
The tag does one job: map URLs to the same content in other languages so results can serve the right variant. It creates no page and adds no crawlability.
A German page that lives only inside an hreflang cluster — no HTML link, no sitemap record, still behind a firewall rule — is absent to a crawler, and a tag pointing at an unreachable target turns the cluster into noise.
Retrieval crawlers fetch pages per question from links, sitemaps, and off-site citations, and generally keep the URL they found. Each version must stand alone: own URL, own route in, own readable content. Delete every tag; if the German page can still be linked, fetched, and read, the tags mean something.
When the signals disagree, the canonical wins
The language directory sets URL shape, canonical decides which page is the original, hreflang declares relationships. Canonical carries the most weight: pointing it elsewhere says the page is not the original.
The rule: each version self-canonicalizes, hreflang points at the others, and the cluster closes both ways. Canonical to English plus hreflang to English is a contradiction; engines follow the canonical and the annotations are wasted.
| Common setup | Why it looks fine | What the AI side sees |
|---|---|---|
| Switcher is JavaScript- or cookie-only | Humans can click it | No HTML link, so the crawler never arrives |
| German pages canonicalized to English | Validator reports zero errors | Treated as a duplicate |
| Auto-redirect by IP or browser language | People land correctly | A 302, so the fetch ends |
| Missing translations 302 to homepage | No 404s | A soft 404: this page should not exist |
A redirect by IP or browser language saves a human one tap and shuts the crawler out, which arrives with no location or cookie. A real language page returns 200 to a request with no preference; a complete AI crawl audit shows how to read those 302s.
Every language version needs its own entry
Sitemap alternates are hreflang written a second way: one <url> carrying xhtml:link records, or one record per language. Both work; the test is whether the German URL is a discoverable entry.
- Each language has its own
<url>, not just an alternates reference - Each
lastmodreflects that language’s update, not the English page’s - Sitemaps split by language, so coverage gaps are visible
- After submitting, fetch 20 German URLs: 200, right language
- Alternates match the on-page hreflang
“Submitted successfully” is false comfort: a sitemap solves discovery, not priority, and as the only entrance it queues last. The three-click rule in crawlable category navigation applies to language directories too.
Content parity: machine translation is not coverage
The hard part is not the tags but whether the German version holds anything worth citing. A 200-SKU catalog where 20 products have German copy and 180 get machine-translated pages is not coverage; it pushes 180 thin pages into the German index and dilutes the 20.
| Situation | Do | Don’t |
|---|---|---|
| Local demand, local copy | Standalone German page; sizing, warranty, delivery written locally | Reuse the English answer unit |
| Demand, no copy | Keep the English page and label its language | Machine-translated shells |
| Neither | 404 or 410, or a localized not-yet-shipping note | 302 to the English homepage |
Track the share of SKUs with German content, not the number of German URLs; the second rewards sprawl.
A five-step alignment, and one example
| Step | Action | Owner | Passes when |
|---|---|---|---|
| 1 Inventory | Language × template matrix, one URL per cell, flagging link, sitemap entry, canonical | Engineering | Every cell has a sample; blanks are gaps |
| 2 Fix the triangle | Self-canonical, bidirectional hreflang, x-default to the default language | Engineering | 20 URLs agree both ways |
| 3 Reopen the road | Static-link the switcher, one sitemap per language, no redirects | Engineering | German URLs return 200 without JavaScript |
| 4 Decide missing pages | Apply the three-case table; no unreviewed machine translation | Content | Every German SKU gap has a decision |
| 5 Retest | Re-run the same German questions, record the cited URL | Growth | Citations in the question’s language, monthly |
Step one gets skipped because “we know how many languages we have” sounds like an answer; real catalogs are archaeological.
Example. A hypothetical cycling-accessories brand, 400 SKUs, US and EU, storefront at /de-de/, no figures. Findings: German pages appear only as alternates inside English sitemap records, the switcher is JavaScript, German PDPs canonicalize to English. Fixes: self-canonical, a closed hreflang cluster, /sitemap-de.xml with one record per URL, static <a> links. Only 60 SKUs carry German copy; the other 340 return a German-language 404. Acceptance: the German homepage reaches a PDP with JavaScript off, German questions cite /de-de/, canonical and hreflang agree.
Where this breaks
A single-language market should skip this: crawling, rendering, and answer units decide more. Marketplaces are the second exception: on Amazon or Kaufland, language versions belong to the platform, so feeds matter instead, as the three storefronts compared lays out.
Where local supply is unbuilt, do not ship language pages ahead of it: a German storefront whose payments, returns, and support run from a US team turns AI-referred buyers into returns. GEO amplifies supply; it does not create it. Legacy ccTLDs should not migrate for uniformity: align canonical and sitemap, leave URL shapes alone. And hreflang is not a ranking lever — it selects a variant, not a position.
This month, take one non-default directory. Fetch 20 URLs with no JavaScript, no cookie, and that language’s Accept-Language header; record status, language, self-canonical, closed hreflang. Every “no” is the first fix batch. Re-run the same questions and check the cited URL against the AI visibility baseline. This line continues in the content and semantics hub.