Mobile page not crawlable due to robots.txt
What Mobile page not crawlable due to robots.txt means in Google Merchant Center, why it hits mobile even when desktop passes, and how to fix your robots.txt rules.
Your robots.txt is blocking Google's crawler on the mobile version of a product page, even though the desktop version is fine. Fix it by checking robots.txt allows Google's crawlers on your mobile site or mobile-specific paths, not just desktop ones.
What triggers this error
This is a mobile-specific variant of a robots.txt block. It fires when Google's mobile (smartphone) crawler is disallowed from the landing page, while the desktop crawler may pass the same product without issue. It's especially common on setups with a separate mobile subdomain (like m.example.com) that has its own robots.txt, or sites using dynamic serving with mobile-only crawl rules left over from an old mobile setup.
Because Diagnostics reports desktop and mobile crawl results separately, a merchant can easily see "everything's fine" on one view and miss that mobile is failing quietly for a chunk of the catalogue.
The landing page crawl errors guide covers robots.txt blocks alongside the other crawl failure modes if you're seeing more than one reason string on the same products.
/tools' feed health checks flag when a product's mobile and desktop crawl results disagree, so a mobile-only robots.txt block gets caught before it quietly caps your reach.
Step by step
- Identify which URL is affected. Check whether the flagged link is your main `link` or a separate `mobile_link`. Sites with a distinct m.example.com subdomain, or dynamic serving with mobile-specific rules, are the most common source of this error.
- Fetch robots.txt for the mobile host. If you run a separate mobile subdomain, it can have its own robots.txt with its own rules. Check that file specifically, don't assume the main site's robots.txt covers it.
- Confirm Google's crawlers are allowed on mobile paths. Look for disallow rules scoped to a smartphone user agent or a mobile-only path. Remove or narrow any rule that blocks product pages on mobile.
- Re-test with a smartphone user agent. Fetch the mobile URL specifically as Googlebot Smartphone to confirm it now returns a full working page, not just a 200 for the desktop crawler.
Frequently asked questions
Why does desktop pass but mobile fail on the same product?
Google crawls desktop and mobile separately with different user agents, and many sites have robots.txt rules or a separate mobile subdomain that only blocks one of them. A pass on desktop tells you nothing about mobile.