We have launched a new version of our website and our rankings are rising and falling daily. How will we know when Google has fully indexed the new site? How can we tell how much progress Google has made in indexing the new site?
The phrase “indexed” is rather complicated. Technically, you could say that Google “indexes” a page when it finds a link to the page*, or you could say the page is indexed when it is visited by GoogleBot, the Google spider. But that does not mean that the page will be included in the SERPs (search engine results pages) or that it will get the full credit in the SERPs that the page deserves.
What does “Index” mean for a single page of your site?
(1) Google has found a link to your page from another page and has predicted the contents of your page based on the text in the link and the context in which the link appeared. You may see your pages in the SERPs with no text description or title.
(2) Google’s spider Googlebot has visited the page. You can track this by looking for the presence of Googlebot in your web logs. Just because the Googlebot has visited your page, don’t expect to see your page in the SERPs.
(3) Google has indexed the contents of your page and has pulled data about the page into its database. Google is aware of the page and may show the page in search results when you search for all pages from your site using the “site:www.yoursite.com” search querry. In this case, you should expect to see your new title in the SERPs and a snippet pulled from the actual text or meta description tag of your page.
(4) Google is showing the page in the SERPs for general searches – i.e. Google has fully evaluated the page and it’s relationship to all other pages on the Web.
(5) Google has assigned a PageRank value to your page.
What does “index” mean for your full site?
It can take several months for Google to fully index your new site. In the process, Google will find pages that are new and will discover pages that were part of the old version of your site that are no longer there. As Google maps out your site, you will see the PageRank of individual pages changing over time. It depends on the extent to which your footprint has changed. If the change is dramatic, you can expect to see dramatic changes in your rankings until the new footprint is fully appreciated. Install the Google Toolbar and look at the PageRank of your individual pages. Your new pages are likely to have no or zero PageRank. Over time, as Google comes to more fully understand your site, PageRank will start to appear on all pages.
How can we tell if Google has indexed the new version of a page or if it still has the old version in the index?
If you changed the title of the page, you can search for the page in Google by using a unique snippet of text on the page and look at the title in the SERPs. If the title is your new title, Google has indexed the new version of that page.
Another way to determine whether Google has “indexed” the new page is to go to the page click on the PageRank drop-down menu in the Google toolbar and select “Cached Snapshot of Page.” If the cached version matches the new version of the site, then the page has been indexed by Google.
You could sign up for a Google Webmaster Tools ( www.google.com—tools ) account and access information there. You can see all of the errors Google finds when it crawls your site. You can also see Crawl stats through the Statistics >Crawl stats report.
In the end, the only way to know when Google has fully indexed your new site is by watching the rankings. Eventually your site’s rankings will reach a stable state and the daily bouncing will cease. At that point, you can start thinking again about what you can do to individual pages to improve their rankings.
* Google has a patent for the process of indexing a web page without visiting that page. Basically Google decides what the page is about by analyzing the links to the page.
See also Will changing our URL affect our search engine rankings? How can we change urls without losing the traffic we’re already getting? and We are planning to relaunch our website with a new design at the same domain name. How do we get the search engines to reassess us for the new site?