Why is My Site Not in the Index?
When Bing crawls and indexes pages on the Internet, it has to make decisions about which pages will most likely resonate with searchers. Given the virtually infinite size of the Web, not all pages will end up being selected. As a result, your pages may not currently be present in the Bing index. This article describes how to verify whether a page from your site is currently in the index and the most common reasons why a page has not made it into the index and some tips about what you can do to improve your chances.
There are a number of reasons why your website may not be in the Bing index, but first you should verify for your site if that is really the case. To do so, you can query for the specific URL of your website using the url: search operator followed by your site URL. For example the following shows that the URL is in the index, since it is returned as a result:If it turns out that no result is returned and your site or page is not in the index, it could be due to any or a combination of the following:
- Your site is new and we haven’t discovered and crawled it yet: With new websites it can take Bing time to find links and crawl through to your website. Even if you submit sitemaps or submit pages directly to us via your Webmaster account as described in Submit URLs to Bing, we still need to crawl and assess it before it appears in the index.
- We are running into issues crawling the site: If we consistently cannot crawl your website, we may not select it to be indexed. You can use Fetch as Bingbot to see if Bingbot, our main crawler, is able to crawl your homepage and other pages.
- There are no links pointing to your website: A common problem for new websites is a lack of links pointing to it from other places on the web. Not only do links help us discover your content, they also tell us a bit about the popularity of your page. Keep in mind we would rather see links from quality, authoritative websites pointing at you than see hundreds of new links from random site suddenly appear. The bottom line, however, is that there needs to be, at the very least, one link pointing to your website for us to find it. More quality links are obviously better, but concentrate on getting even a few quality links pointed at your new website by creating compelling, link-worthy content.
- Your robots.txt file is preventing us from crawling your site: If you have incorrectly set up your robots.txt file, you may be inadvertently telling Bingbot to not crawl your website. This could potentially keep you from being seen in our index. Check out How to Create a Robots.txt File to learn more about how to properly set up a basic robots.txt file.
- You have a <meta name="robots" content="NOINDEX"> on your pages: If your pages contain <meta name="robots" content="NOINDEX"> in the page's source code we will not add them to the index.
- You have blocked URLs inside your Bing Webmaster Tools: If you have an active Webmaster account with Bing, you may have blocked URLs form showing in our results using the Block URLs tool. Check your account to ensure the URLs you expect to see in the index are not blocked here.
- Your site does not meet the quality threshold required by Bing: Bing likes unique, quality content. Websites which replicate content, redirect users quickly, or provide little depth often don’t fare well in our results. Prolonged poor performance can lead us to delist websites to make room for quality websites.
- Your website has been assessed a penalty and has been removed from the index: In rare case where we have found your web site in violation of quality guidelines or we have discovered malware on your site your site may not appear in search results because it has been penalized. Bing communicates directly with website owners via the Webmaster tools to alert them to any malware we detect on a website. Should malware be detected on your website, we will either remove the site from our search results entirely or add a malware warning to the listing to protect our searchers. If you've been hit by malware you should follow the Malware Re-Evaluation process.