Getting Indexed By Google.

We have launched a new version of our website and our rankings are rising and falling daily. How will we know when Google has fully indexed the new site? How can we tell how much progress Google has made in indexing the new site?

The phrase “indexed” is rather complicated.  Technically, you could say that Google “indexes” a page when it finds a link to the page*, or you could say the page is indexed when it is visited by GoogleBot, the Google spider.  But that does not mean that the page will be included in the SERPs (search engine results pages) or that it will get the full credit in the SERPs that the page deserves.

What does “Index” mean for a single page of your site?

(1) Google has found a link to your page from another page and has predicted the contents of your page based on the text in the link and the context in which the link appeared.  You may see your pages in the SERPs with no text description or title.

(2) Google’s spider Googlebot has visited the page. You can track this by looking for the presence of Googlebot in your web logs.  Just because the Googlebot has visited your page, don’t expect to see your page in the SERPs.

(3) Google has indexed the contents of your page and has pulled data about the page into its database.  Google is aware of the page and may show the page in search results when you search for all pages from your site using the “site:www.yoursite.com” search query.  In this case, you should expect to see your new title in the SERPs and a snippet pulled from the actual text or meta description tag of your page.

(4) Google is showing the page in the SERPs for general searches – i.e. Google has fully evaluated the page and its relationship to all other pages on the Web.

(5) Google has assigned a PageRank value to your page.

What does “index” mean for your full site?

It can take several months for Google to fully index your new site.  In the process, Google will find pages that are new and will discover pages that were part of the old version of your site that are no longer there.  As Google maps out your site, you will see the PageRank of individual pages changing over time.  It depends on the extent to which your footprint has changed.  If the change is dramatic, you can expect to see dramatic changes in your rankings until the new footprint is fully appreciated.

How can we tell if Google has indexed the new version of a page or if it still has the old version in the index?

If you changed the title of the page, you can search for the page in Google by using a unique snippet of text on the page and look at the title in the SERPs.  If the title is your new title, Google has indexed the new version of that page.

You can also explore the tools and tips available through Google Search Central (formerly Google Webmasters).

In the end, the only way to know when Google has fully indexed your new site is by watching the rankings.  Eventually your site’s rankings will reach a stable state and the daily bouncing will cease.  At that point, you can start thinking again about what you can do to individual pages to improve their rankings.

* Google has a patent for the process of indexing a web page without visiting that page.  Basically Google decides what the page is about by analyzing the links to the page.

See also Will changing our URL affect our search engine rankings? How can we change urls without losing the traffic we’re already getting? and We are planning to relaunch our website with a new design at the same domain name. How do we get the search engines to reassess us for the new site?