Fixing Crawl Errors and Indexing Problems in Search Console

The Page Indexing report in Google Search Console is the single best diagnostic tool most site owners never fully open. This walkthrough shows you how to read every status it reports, decide which ones actually matter, and fix the ones that are quietly keeping your pages out of Google.

Updated August 2026 · 14 min read

A few months ago I was helping a friend with a small e-commerce site that had stalled at around 2,000 organic visits a month. Content was fine. Links were fine. But when we opened the Page Indexing report in Search Console, 340 of his 480 product and category pages were sitting in “Crawled — currently not indexed.” He had been publishing for two years and more than two-thirds of his catalog was invisible to Google. Three weeks of cleanup later, 260 of those pages were indexed, and organic traffic climbed about 40% in the following quarter. Nothing magical happened — we just read the report properly and fixed what it told us to fix.

That experience is more common than you’d think. Most site owners check Search Console for clicks and impressions, glance at the coverage number, and close the tab. The indexing report, meanwhile, is quietly listing every reason Google is refusing to show your pages. Learning to read it is one of the highest-leverage technical SEO skills you can pick up, and it doesn’t require any paid tools. If you’re brand new to the platform, our Google Search Console guide covers setup and verification; this article picks up where that leaves off.

What the Page Indexing report actually shows

Open Search Console, look at the left sidebar under “Indexing,” and click “Pages.” You’ll see a bar chart split into two colors: green for indexed pages, gray for not indexed. Below the chart is the part that matters — a table of reasons, sorted by how many URLs each one affects.

The first thing to understand: not indexed does not mean broken. Google deliberately excludes URLs all the time, and many of those exclusions are correct and even desirable. Your site’s internal search results pages, filtered category views, tag archives, staging URLs, and paginated comment feeds probably shouldn’t be indexed. The skill isn’t getting the gray bar to zero — it’s knowing which rows in the table represent real problems and which represent Google doing exactly what you want.

Click any row in the table and you get a detail page: a trend chart for that specific issue, a list of example URLs, and — after you fix things — a “Validate Fix” button that asks Google to re-check. Keep that button in mind. It’s how you close the loop on every repair you make.

Decoding the “Not indexed” statuses, one by one

There are roughly a dozen statuses that show up regularly. Here’s what the important ones mean in plain language, and whether you should worry.

Crawled — currently not indexed

Google fetched the page, looked at it, and decided not to add it to the index — for now. This is a quality judgment, not a technical error. Common causes: thin content, near-duplicate pages, content that doesn’t add anything beyond what already ranks, or a site whose overall quality signals are weak. On my friend’s e-commerce site, the culprit was product pages with the manufacturer’s two-sentence description and nothing else. Google’s own documentation on the Page Indexing report confirms this status usually means the page didn’t clear the usefulness bar.

The fix is to make the pages genuinely better: unique descriptions, real specs, customer questions answered, original photos. If the page doesn’t deserve to exist, consider consolidating it into a stronger page instead.

Discovered — currently not indexed

Google knows the URL exists but hasn’t even crawled it yet. This one is usually about crawl budget or scheduling. Google saw the link, queued it, and keeps deferring. Typical triggers: the site is slow to respond, the server threw errors when Googlebot last visited, or the site has a huge number of low-value URLs diluting Google’s attention. If this count is large and growing, check your server response times and your hosting stability first.

Duplicate without user-selected canonical

Google found two or more very similar pages and picked its own canonical — possibly not the one you wanted. This happens constantly on sites with URL parameters (sort orders, tracking tags, session IDs) and on sites where the same content appears on multiple URLs. The fix is to add a clear rel="canonical" link element pointing to the preferred version, and to clean up your internal linking so you consistently link to the canonical URL.

Alternate page with proper canonical

This one usually needs no action. Google found a duplicate, saw your canonical tag, and correctly consolidated it. Mobile/desktop URL pairs, AMP pages, and parameter duplicates with canonicals all land here. Verify a couple of example URLs to make sure the canonical points where you expect, then move on.

Excluded by ‘noindex’ tag

You — or your CMS — told Google not to index the page. The question is whether that was intentional. I’ve seen entire WordPress sites launch with “Discourage search engines from indexing this site” still checked in Settings > Reading. If the URLs listed here are pages you want ranking, find where the noindex is coming from (a meta tag, an X-Robots-Tag HTTP header, or an SEO plugin setting) and remove it.

Blocked by robots.txt

Your robots.txt file is telling Googlebot not to crawl these URLs. Important nuance: a page blocked by robots.txt can still appear in search results (as a bare URL without a description) if other sites link to it, because Google can’t crawl it to see any noindex tag. If you want a page fully out of search, allow the crawl and use noindex instead. If you want it indexed and it’s blocked here by accident, edit robots.txt. We have a full walkthrough in XML sitemaps and robots.txt, and Google’s robots.txt documentation explains the syntax in detail.

The server and redirect errors that do the most damage

The statuses above are mostly exclusions. The ones below are outright errors, and they’re the rows I check first, because they affect pages you definitely want indexed.

Server errors deserve special attention because they’re often intermittent. Your site loads fine when you visit it, but at 3 a.m. when backups run, or during a traffic spike, it starts throwing 502s — and that’s exactly when Googlebot shows up. If you see a spike of 5xx errors on specific dates, correlate them with your hosting logs. Cheap shared hosting hitting its resource limits is the most common root cause I’ve seen, and the fix is usually upgrading the plan or adding caching, not anything in Search Console itself.

For 404s, apply a simple rule: if the page was removed deliberately and has no replacement, let it 404 — Google will drop it, and that’s fine. If it has backlinks pointing at it or a logical successor exists, 301-redirect it to the closest matching page. Never mass-redirect everything to the homepage; Google treats that as a soft 404 anyway.

A practical triage workflow that takes about an hour

Here’s the exact sequence I run through when I audit a site’s indexing report. If you already use our one-hour SEO audit checklist, this slots into it as the technical section.

• Export everything. At the top right of the Pages report, click the export button and pull the full table into a spreadsheet. The UI only samples 1,000 URLs per status; the export gives you the real picture.
• Sort statuses into three buckets. Bucket 1: “should be excluded” (noindex, canonical alternates, 404s of deleted pages). Bucket 2: “definitely fix” (5xx, redirect errors, accidental noindex, robots.txt blocks on wanted pages). Bucket 3: “improve or merge” (crawled/discovered not indexed, duplicates without canonical).
• Check the trend, not just the count. Click into each bucket-2 status and look at the chart. A sudden spike has a cause — a deploy, a plugin update, a hosting migration. A slow, steady growth suggests a systemic issue like parameter URL explosion.
• Spot-check with URL Inspection. Paste 5–10 example URLs per status into the URL Inspection tool at the top of Search Console. It shows the last crawl date, the rendered HTML, the canonical Google selected, and whether the page is indexable right now. This is where you catch things the report summary hides — like a canonical pointing to the wrong domain.
• Fix in order of traffic impact. A 5xx error on your top category page outranks a duplicate parameter URL on a page nobody visits. Sort your fix list by the organic traffic those sections of the site historically earned.
• Validate and wait. After each fix, hit “Validate Fix” on the corresponding status. Google will email you when validation starts and when it finishes. Expect days to weeks, not hours.

The indexing report doesn’t tell you what to do — it tells you where to look. Every status is a question, and URL Inspection is how you get the answer.

Crawl stats: the report nobody opens (but should)

Hidden under Settings > Crawl Stats is a report that pairs beautifully with Page Indexing. It shows how many requests Googlebot makes per day, broken down by response code, file type, and Googlebot type (smartphone vs. desktop).

Two things to look for. First, the average response time. If it’s consistently above about 600–800 milliseconds, Google will crawl you more slowly and less deeply — which is one of the root causes behind a big “Discovered — currently not indexed” pile. Second, the response code breakdown. If more than a percent or two of Googlebot’s requests are hitting 5xx or 404 responses, you’re wasting crawl budget and signaling an unstable site. On one client site, 30% of Googlebot’s daily requests were going to dead URLs left over from an old site structure — fixing the redirects freed up the crawl for pages that actually mattered.

Sitemaps and indexing: what helps and what doesn’t

An XML sitemap won’t force Google to index anything, but it’s still one of your best diagnostic tools. Submit your sitemap in Search Console (Indexing > Sitemaps), and two useful things happen. First, the “Discovered” count in the indexing report gets a cleaner signal, because Google can distinguish URLs it found from your sitemap versus from links. Second, sitemap coverage errors surface problems your CMS is hiding — like pages included in the sitemap that are noindexed, which is a direct contradiction you should resolve.

A few sitemap hygiene rules: only include URLs you actually want indexed (200 status, indexable, canonical to themselves); keep the sitemap under 50,000 URLs per file and split into sections by content type; and resubmit after major site changes. Google’s sitemaps documentation covers the format if you’re building one by hand, though WordPress and every major SEO plugin generate one automatically.

When “not indexed” is a content problem in disguise

The hardest rows to deal with are “Crawled — currently not indexed” on pages you believe are good. Google is making an editorial call, and there’s no checkbox to change its mind. What works, roughly in order of effort:

Make the page genuinely more useful

Compare the page to what currently ranks for its target query. If the top results have original data, step-by-step screenshots, or firsthand experience and yours is a competent summary of other articles, that’s your answer. Add something only you can provide — your own test results, a real example from your work, a template or checklist people can download.

Strengthen internal links to the page

Pages buried four clicks deep with a single internal link read as unimportant. Link to the page from your strongest, most relevant existing content with descriptive anchor text. This is the cheapest indexing lever you have, and it also helps rankings once the page does get in.

Consolidate instead of multiplying

Five thin posts on overlapping keyword variations often index and rank worse than one comprehensive guide. If you have several pages sitting in “Duplicate” or “Crawled — currently not indexed” that cover nearly the same topic, merge them into the strongest URL, 301-redirect the rest, and watch the combined page’s indexing and impressions over the following month.

Special cases: new sites, migrations, and sudden drops

Three situations deserve their own mention because they follow different rules than steady-state maintenance.

Brand-new sites. If your site launched in the last few weeks and most pages show “Discovered — currently not indexed,” that’s normal, not a bug. Google is cautious with new domains that have no history and few links. Keep publishing useful content, earn a few real links, submit your sitemap, and give it a month or two. Manually requesting indexing on your five or ten most important pages is reasonable here; beyond that, patience is the fix.

Site migrations and redesigns. This is where indexing problems are born. Every URL change needs a 301 redirect from the old address, every redirect needs to land on a 200 page (not another redirect), and your sitemap, canonical tags, and internal links all need to reference the new URLs. After a migration, watch the indexing report weekly for at least two months. The pattern of a botched migration is unmistakable: 404s and redirect errors climbing while “Page with redirect” balloons. Catch it early and it’s a one-afternoon fix; catch it late and you’re rebuilding months of lost crawl history.

Sudden indexing drops. If your indexed count falls off a cliff in days, check three things in order: whether someone added a sitewide noindex or robots.txt block (it happens more than you’d believe, usually during a staging-to-live push), whether the server is throwing 5xx errors to Googlebot, and whether a manual action appears under Security & Manual Actions in Search Console. The first two are technical accidents with quick fixes; the third is rare but serious, and the report will tell you exactly what Google objected to.

Common mistakes to avoid while fixing things

A few traps I see people fall into repeatedly. Don’t use the Removals tool to “fix” indexing problems — it hides URLs from search results for about six months but solves nothing underneath, and it’s meant for urgent removals only. Don’t request indexing on hundreds of URLs manually via URL Inspection; there’s a per-day quota, and if Google is declining to index a page for quality reasons, re-requesting changes nothing. Don’t panic-validate a fix before you’ve actually confirmed the fix works — failed validations reset the clock. And don’t obsess over getting every URL indexed; a healthy site always has some exclusions.

Finally, remember that indexing is a lagging indicator. Fixes you make today may take two to six weeks to fully reflect in the report. Keep a simple log: date, issue, action taken, validation date. When the gray bars start shrinking, you’ll know exactly which change did it.

Sources: Google Search Central documentation on the Page Indexing report, robots.txt, and XML sitemaps (linked inline above). This article is updated periodically as Search Console’s interface and status labels evolve. Related reads on this site: Google Search Console guide, XML sitemaps and robots.txt, and the one-hour SEO audit checklist.