Is your website encountering the “Crawled – currently not indexed” status in Google Search Console? This Page indexing status indicates that Googlebot (Google’s URL crawl bot) has visited a particular web page (for a verified website property in Search Console), but hasn’t yet added the web page to its index, which also means the web page won’t show up in Google Search.
Video Tutorial How to Fix Crawled Currently Not Indexed Error
Fix All Submitted Pages NOT All Known Pages
You NEED NOT Review robots.txt Rules & NOINDEX Meta Tags
Do not waste time using robots.txt testing tool in Search Console because Google CAN crawl the web page. You need to NOT check <meta name=”robots” content=”noindex”> meta tags because Excluded by ‘noindex’ tag error reports are for those types of problems.
To Be Able to Remedy Crawled – currently not indexed Status
We have to understand the difference between Google’s CRAWLING versus INDEXING process, and also what is URL?
URL stands for Uniform Resource Locator. A URL is nothing more than the address of a given unique resource on the Web.
Knowing this, the way Google search engine bots traversing the internet to find content (that is your web page in our example) by following links. Therefore, link is the URL which Google tries to access/crawl.
If we don’t understand the basic concepts, we’ll be wasting time trying to get Google to show web pages in search results. So, I’ve created an image because it will be easier to understand.
Crawling in Google Search Console Reports Simplified
Crawling is just the process of finding webpages (new or updated webpages) so that Google can add the contents found on the URL in Google index. Basically, it is Googlebot sending a HTTP request to the web server for a given URL, and then first checks the rules defined (if any) in robots.txt (mainly for legal reasons) file to see IF IT IS ALLOWED TO ACCESS THE web page. If so:
Indexing = a Web Page Contents are in Google Database
After Google crawler called Googlebot is allowed to index the URL (if Googlebot is not blocked through robots.txt file rules), Google algorithms analyzes the content, analyzes the meaning of the keywords it encounters. Then, if Google algorithms deems the content worthy (according to its latest guidelines) as in the content is useful, helpful, reliable, created considering people first (instead of quick and easy low-quality content for the sake of keyword ranking). Google then stores the worthy content in its database (Google has global Data Centers).
What Does Google systems Mean in Crawled – currently not indexed Status?
As detailed in the video tutorial for fixing crawled but currently not indexed errors the problem with Page indexing Crawled – currently not indexed is determined by Google’s ranking algorithm, as opposed to a technical problem encountered on a Website. But the confusing part for most website owners is the fact that Google Systems also evaluates the website.
How to Fix Crawled Currently Not Indexed Errors
All you have to do is adhere to Google Search Essentials, as in, ensure that the web site and web pages follow Google guidelines. You can check out the latest course for SEO by RankYa where I show you how to get top Google ranking results and higher website conversions.
3 Golden Rules of Search Essentials
- Technical requirements: What Google needs from a web page to show it in Google Search. Basically means, fast loading web page that is mobile friendly and doesn’t have broken design elements.
- Spam policies: The behaviors and tactics that can lead to lower ranking or being completely omitted from Google Search results. Basically means spam, if you’ve hired spammy SEO experts, or, trying to outsmart google with keyword stuffing, backlink spam etc. this then will apply. Most websites need not worry about this but be aware of not to do.
- Key best practices: The main things that can help improve how your site appears in Google Search results. Basically means on page optimization while ensuring content is created for the best interest of people as opposed to search engine traffic.
Other Factors to Consider When Fixing Crawled Currently Not Indexed Problems
Let’s imagine your website has great content, or the website is an e-commerce store, and, example URLs are for large amount of web pages, this could also mean Google Systems are picking up other more technical issues which needs to be identified and addressed. Basic example: if the webpage has original unique and useful content BUT the web page loads too slow AND there is design issues, Google Systems may in this example also not index a web page because the web page will provide bad user experience. Accordingly:
Make Sure There are NO Core Web Vital Errors
Although content is critical, Google no longer just wants great content only, it wants great content on a website that provides great user experience. Accordingly, websites must fix Core Web Vitals errors so that poor user experience is eliminated and Mobile Usability errors are fixed when a web page is viewed on mobile devices. These are part of latest Google guidelines. So let’s find out how you can fix crawled but not indexed status identified by Google systems.
Provide Content for People
As a website owner, it’s crucial to ensure that web pages create people-first content.
This is a major change Google has now made, thus, page indexing shows crawled currently not indexed identified by Google Systems, stating that content is poorly created, thin content or Google determines the content not worthwhile to put in its index.
Evaluate whether the entire website (not just the example web pages shown in Google Search Console Page Indexing reports) effectively addresses your users’ needs and consider enhancing the quality of the website entirely. Consider fixing other Page indexing errors (if any) such as Page with redirect or Excluded by ‘noindex’ tag or any error shown in Search Console reports.
Important Questions to Ask for Fixing Crawled Currently Not Indexed Status
- Is the content informative and comprehensive according to the search query?
- Does it answer the user’s search query? So that people don’t go back and search Google again.
- Is it unique and not just duplicated from other sources?
- Does other external websites (e.g. competitor) have the same or similar content? If so, you have to ensure your website loads faster, HTML is semantically created, mobile usability is better.
- Is it expertly created, compared to other websites in the same or similar field does your website provide greater user experience?
- Is it well-structured and easy to read? HTML Structure of a website affects Google rankings as well.
Review robots.txt Rules & NOINDEX Meta Tags
Although this particular issue related to Google systems isn’t related to crawling or noindex tags, you still should check and confirm other URLs of the website so that there are no instructions preventing Google indexing other URLs. Look out for <meta name=”robots” content=”noindex”> tags or directives ‘disallow’ in the robots.txt file that might prevent Googlebot’s crawl process.
Review Web Page Title Element Meta Description
Inspect the HTML title and meta description on the page, these are important HTML elements to ensure they are informative and follow google guidelines.
Check and Verify URL Canonicalization
Ensure that there are no issues with canonical URL. The canonical tag should point to the preferred version of the page. Using link canonical of the URL a website will avoid duplicate content problems.
<link rel="canonical" href="https://www.rankya.com/google-search-console/page-indexing/howtofix-crawled-currentlynotindexed/" />
Check for Web Server or Website Theme Issues
Website files reside on a web server, any server issues (i.e. 500 Internal Server Error) problems can hinder Google’s ability to index a webpage. Monitor server logs for any connectivity issues that might be affecting the accessibility of the page. You can also check server error log files to see if there are any website theme issues which may be visible to Google search crawler but not you as website owner.
DO NOT Request Indexing via Google Search Console
If you’ve done everything correctly, Google will update its index database. You can NOT force Google to index a web page by using “Request Indexing” feature in Google Search Console.
Updated Content and Video Tutorials (Covers All Page Indexing Issues)
Technical part of SE. Crawled, but currently not index, is a very common issue. I was searching for how to solve it properly and finally got it.
This is not only an article, but rather a step-by-step strategy with a great image of specification.
yes right its ok
Thanks for the in-depth walk through guide. I faced this “Crawled currently not indexed” issue for my website. It was quite shocking at first, since I didn’t know what does crawled currently not index means at first place. Reading blogs and going through some technical details I realized that this is more about the “page rank algorithm”, which you rightly pointed out.
All I did was, interlinked that particular page from other internal pages, in order to increase its authority. And the URL was live in google. I really appreciate your time and the effort you’ve done, explaining all possible issues causing the error. Thanks man!
Well done Mehdi, this is exactly one of the optimization techniques you can do to get a particular URL indexed. Another things you can do is optimization of the URL that is crawled but not yet indexed to improve their relevancy.