Learn
How to Fix Crawl Errors in Google Search Console
Crawl and indexing errors quietly keep pages out of Google. This guide explains the Search Console reports and how to fix the most common causes.
Run a fresh DomainLens audit and use the report as your priority list.
What “crawl errors” actually mean
A crawl error means Googlebot tried to reach a URL and something went wrong, or the page was reachable but Google chose not to index it. In Google Search Console these now live mainly in the Page indexing report, which lists why each URL is or isn’t in the index.
Not every excluded URL is a problem — some pages should not be indexed. The goal is to separate intentional exclusions from real errors that are costing you traffic.
Where to find them in Search Console
Open the Page indexing report to see indexed vs not-indexed URLs grouped by reason. Use the URL Inspection tool on any single page to see its last crawl, whether it’s indexable, and any blocking signals. The Crawl stats report (in Settings) shows server responses Googlebot received over time.
Start with the reasons affecting the most URLs — fixing one cause often clears dozens of pages at once.
Common crawl and indexing errors
- Server errors (5xx) — Googlebot couldn’t load the page due to a server problem.
- Not found (404) — the URL is linked or known but no longer exists.
- Redirect errors — redirect chains or loops that never resolve.
- Blocked by robots.txt — crawling is disallowed, so Google can’t read the page.
- Excluded by “noindex” — a meta tag or header tells Google not to index it.
- Crawled — currently not indexed — Google saw the page but didn’t consider it worth indexing.
How to fix each cause
Match the fix to the reason. For 5xx, stabilize the server or fix the application error. For unwanted 404s, restore the page or 301-redirect it to the right URL. For redirect errors, collapse chains so each old URL points directly to the final destination. For robots.txt or noindex blocks, remove the directive only on pages you actually want indexed.
“Crawled — not indexed” usually signals quality or duplication: improve the content, make it unique, and strengthen internal links so Google sees the page as worth keeping.
Mistakes that make it worse
- Mass-requesting indexing instead of fixing the underlying cause.
- Blocking a URL in robots.txt to “remove” it — that just prevents Google from seeing the noindex.
- Redirecting every 404 to the homepage, which Google treats as a soft 404.
- Ignoring “Discovered — not indexed” on large sites, often a crawl-budget signal.
- Fixing one URL at a time instead of the shared root cause.
How DomainLens helps
Search Console tells you which pages are affected; DomainLens helps you see why on a given URL. It reports the HTTP status, redirect chains, robots and noindex directives, and the canonical target — the exact signals behind most indexing exclusions — and explains each in plain language.
Audit a problem URL, fix what the report flags, then use URL Inspection in Search Console to request a fresh crawl and confirm the fix.
Next steps
Work the Page indexing report from the largest issue down, validate each fix, and re-check monthly so new errors don’t pile up silently.
Run a free DomainLens audit on any URL stuck as not-indexed to pinpoint the blocking signal before you ask Google to recrawl.