Why is My Website Not Showing in Google Search - SEO 101
Your website actually does appear
We’ll get the easy one out of the way first. Just because your website doesn’t appear on the first, second, third, or eighth page of Google’s search results doesn’t mean Google hasn’t indexed it. It may very well be on page ten or higher and simply isn’t ranking very well.
There’s a quick method of determining if Google is aware of your website’s existence in the first place: The “site:” search operator.
The “site:” operator instructs Google to perform a search only within the context of a specific website. For example, if you search “site:Microsoft.com apples”, Google will only return pages hosted on Microsoft’s website that have to do with apples.
To check if your site is indexed by Google in the first place, just perform a “site:<yourdomain>” search. If your site is somewhere within Google’s index of websites, you’ll see your pages pop up in the search results, and you’ll see a number at the top listing however many results Google has for your website.
If this is the case and your website is indexed but not ranking well, then it’s time to work on your SEO!
On the other hand, if you see a message claiming something along the lines of “Your search did not match any documents,” then your website is unfortunately not currently indexed by Google.
If you’re sure your website is not in Google’s index, here are a few possible explanations and solutions.
Your website is new
If your website is very new, it’s possible that Google simply hasn’t discovered it yet. While it often seems like Google operates instantaneously, that’s not quite true. It continually crawls the web looking for websites to index, links to follow, and so on, but it doesn’t happen instantly.
If you’re dealing with a new website that Google hasn’t noticed yet, you can speed the process up with Google’s free Search Console service.
Google Search Console is essentially a free website monitoring tool that shows you general information about your site. Once you set up an account and verify that you own the website in question, there are two quick ways to get your site indexed by Google.
First, you can manually submit a sitemap for Google to crawl and index. A sitemap is an XML file that you host on your website’s server, and it essentially serves as an outline for your entire site.
It has a long list of all pages, posts, and images that comprise your site, and makes it much easier for Google to crawl through your website and discover all of your content. Submitting a sitemap in the first place is a great way of saying to Google, “Hey, here’s a new website you should check out!”
The second way is to use the “fetch as Google” functionality which is also available for free through the Search Console service. Basically, you can ask Google to fetch and parse a specific URL so that you can see how Google’s GoogleBot “sees” your website.
While this is a fantastic way of ensuring your website is structured correctly in general, it’s also a great method of getting your site quickly indexed by Google. After Google fetches and renders your page, you can click a button to have the URL in question submitted to Google for indexing.
Both of these methods usually still take a little bit of time to get your site indexed by Google, but it can also happen almost instantly if you’re lucky.
You can then verify that your site is indexed by Google and appearing somewhere in the SERPs by performing another “site:<yourdomain>” query.
The “noindex” tag
The “noindex” meta tag is a page-level directive that instructs Google’s crawlers not to index certain portions of your website. This may seem strange, but there are certain cases where you may not want Google to index certain pages on your site (administrative backend pages, PPC landing pages with no content, etc).
So, if one or several of your pages aren’t getting picked up by Google, it’s always good to check the HTML and see if a <meta name="robots" content="noindex"> tag is present in the page head.
A robots.txt directive
The robots.txt file is a plain text file that instructs web bots like Google’s crawlers how to behave when they visit your site. You can prevent web crawlers from visiting entire portions of your site with the robots.txt file.
Again, there are many legitimate reasons for preventing Google and other bots from crawling certain sections of your website. The issue is when you accidentally institute this “disallow” directive on pages that you do want indexed, or, even worse, if you enact a site-wide crawler disallow rule.
The site-wide exclusion directive typically looks like this:
The backslash (“/”) in that directive refers to the root of your website, and tells all web robots that they’re not allowed to crawl any of your website.
If you find that your robots.txt file is to blame for Google not indexing your website, all you have to do is edit the “Disallow” line so that Google can crawl your website the next time its GoogleBot crawler stops by.
Your website is penalized
This is the worst-case scenario. In rare situations, Google may temporarily or permanently deindex your entire website. This typically occurs if you’ve been found in excessive violation of Google Webmaster Guidelines, and needs to be dealt with on a case-by-case basis.
If you wake up one day to find your website penalized or deindexed, more information is typically available in a message from Google that’s displayed within your Search Console dashboard.