Fix indexability first, or your best SEO work may sit in the dark. A page can have great content, fast speed, and lovely titles. But if Google cannot find it, crawl it, or index it, it is basically a billboard in a locked basement.
TLDR: Indexability means a search engine can discover a page, crawl it, understand it, and store it in its index. If your page is not indexed, it cannot rank in normal search results. For example, a small shop found only 780 of 1,240 product pages indexed, which meant about 37% of products had no chance to bring organic traffic. The fix started with robots.txt, noindex tags, poor internal links, and messy canonical tags.
What indexability really means
Think of Google as a very busy librarian. Your website is a giant pile of books. Indexability is the process that lets the librarian find a book, open it, read it, and place it on the right shelf.
If any step fails, the page may vanish from search.
- Discovery: Search engines find the URL.
- Crawling: Bots visit the page.
- Rendering: Bots load the page content.
- Indexing: Bots decide if the page belongs in search.
A page can be discovered but not indexed. It can be crawled but blocked. It can load for users but fail for bots. Annoying? Yes. Common? Very.
Signs that pages are not indexed
You do not need to guess. Search engines leave clues. Some are obvious. Some are weird little traps.
Watch for these signs:
- Traffic drops on pages that used to perform.
- New pages get zero impressions after several weeks.
- Google Search Console says “Discovered, currently not indexed.”
- Search results do not show the page when you search the exact URL.
- Important pages are missing from the sitemap report.
- Crawlers find many blocked pages or redirect chains.
Use the site: search operator for a quick sniff test. Search this:
site:example.com/your-page-url
If nothing appears, do not panic yet. This check is imperfect. Google Search Console is better. Still, it is a fast way to spot trouble.
Start with Google Search Console
Google Search Console is your first stop. It can be clunky. Honestly, it feels like one report takes five seconds longer than it should. But it is still worth using.
Open URL Inspection. Paste the page URL. Check the result.
You may see messages like:
- URL is on Google: Great. It is indexed.
- URL is not on Google: It is not indexed.
- Crawled, currently not indexed: Google visited it but skipped indexing.
- Discovered, currently not indexed: Google knows it exists but has not crawled it yet.
- Blocked by robots.txt: Bots are not allowed to crawl it.
- Excluded by noindex tag: The page tells Google to stay out of the index.
Then click Test Live URL. This checks the current page, not old data. That matters. Search Console reports can lag.
Check the robots.txt file
Your robots.txt file tells bots where they can and cannot crawl. A tiny mistake here can block a whole section. It drives me crazy that one short line can ruin hundreds of pages.
Look for lines like this:
User-agent: *
Disallow: /products/
That example blocks bots from crawling the product folder. If your money pages live there, ouch.
Visit your file here:
example.com/robots.txt
Check if key folders are blocked. Also check staging rules. Sometimes a site goes live with old development settings. That is like opening a store and leaving the front door chained.
Find noindex tags
A noindex tag tells search engines not to add the page to the index. This can be useful. For example, you may want to noindex thank you pages, login pages, or thin filter pages.
But noindex on a main service page? Big problem.
Look in the page source for this:
<meta name="robots" content="noindex">
Also check HTTP headers. Some sites add noindex there instead. SEO crawling tools can find this fast. Screaming Frog, Sitebulb, and similar tools are useful here.
Inspect canonical tags
A canonical tag tells search engines which version of a page is the main one. This helps with duplicates. But a bad canonical tag can point Google away from the page you want indexed.
Example:
<link rel="canonical" href="https://example.com/other-page/">
If that sits on your target page, Google may index the other page instead.
Check for these issues:
- Canonical tags pointing to the wrong URL.
- Canonical tags pointing to non-indexable pages.
- Canonical tags pointing to redirected pages.
- Missing self-referencing canonicals on key pages.
Canonicals are suggestions, not commands. Still, search engines often listen to them.
Review your sitemap
Your XML sitemap is a map for bots. It should include your best pages. Not every page. Not junk. Not old redirects. Not broken URLs.
A good sitemap includes:
- Indexable URLs only.
- Canonical versions only.
- Pages that return 200 OK.
- Fresh pages you want found faster.
Submit your sitemap in Search Console. Then review the report. If Google sees 2,000 submitted pages but indexes only 900, you need to dig.
Common sitemap messes include old URLs, parameter URLs, duplicate pages, and pages blocked by robots.txt. It is messy, but fixable.
Check internal links
Search engines find pages through links. If a page has no internal links, it is an orphan. Orphans are easy to miss.
Make sure key pages are linked from:
- The main menu.
- Category pages.
- Related blog posts.
- Breadcrumbs.
- HTML sitemaps, if you use them.
Do not bury key pages ten clicks deep. If users cannot find them, bots may struggle too.
Use a crawler to find orphan pages. Compare your sitemap URLs with crawled URLs. Pages in the sitemap but not found during a crawl may have weak internal links.
Look at page quality
Sometimes nothing is “blocked.” Google just does not think the page is worth indexing.
This often happens with:
- Thin pages with very little content.
- Duplicate product descriptions.
- Near-identical location pages.
- Tag pages with no value.
- Filter pages that create thousands of weak URLs.
If Search Console says “Crawled, currently not indexed,” quality may be the issue. Not always. But often.
Improve the page. Add clearer answers. Add unique copy. Add helpful images. Add FAQs. Show real details, specs, comparisons, reviews, or examples.
Test rendering problems
Some pages depend heavily on JavaScript. Users may see content. Bots may see a blank shell or delayed content.
Use the URL Inspection live test. Check the screenshot. Check the rendered HTML. If core content is missing, rendering is the problem.
Watch for:
- Text loaded only after user clicks.
- Blocked JavaScript files.
- Slow API calls.
- Content hidden behind scripts.
- Infinite scroll without crawlable links.
Check status codes and redirects
Search engines like clean signals. Status codes are signals.
- 200: The page works.
- 301: The page moved permanently.
- 302: The page moved temporarily.
- 404: The page is missing.
- 500: The server had an error.
Your indexable pages should return 200 OK. Avoid long redirect chains. Example: URL A redirects to B, then C, then D. Bots can follow some redirects, but chains waste crawl time.
A simple indexability checklist
Use this when a page will not index:
- Inspect the URL in Google Search Console.
- Run a live test.
- Check robots.txt.
- Check for noindex tags.
- Check the canonical tag.
- Confirm the page is in the sitemap.
- Confirm it has internal links.
- Check the status code.
- Review rendered content.
- Improve page quality if Google has crawled it but skipped it.
Final tip
Indexability is not magic. It is plumbing. When the pipes are blocked, traffic cannot flow.
Start with technical blocks. Then check links. Then improve the page. Keep the process simple. If a page deserves to rank, make sure search engines can find it, read it, and trust it first.
