Frames and Search Engines

Is it true that using frames makes it harder for search engines to properly index my site?

Yes. Using frames in your website design makes it harder for search engines to properly index your site. In fact, in the early days of search, Inktomi and AltaVista simply wouldn’t index your framed site at all.

A typical framed Web page is made up of three or more files instead of the standard one file that makes up non-framed Web pages. In a typical framed Web page, there are one or more files for the main body content, one or more files for the navigation, and one main file to stitch the other two or more files together into a Web page (the assembler frame). The illustration below shows a simple framed site, demonstrating how frames work together to create separate parts of a web page, which are then sewn together by the main assembler frame page. Put your mouse over the different parts of the image for a detailed explanation.

In the picture above the Main Frame (or assembler) is one HTML file, the Navigation Frame is a second HTML file, and the Content Frame is a third HTML file (these need not be HTML files). When a someone visits your site, they are presented with the Main Frame (or assembler) which they download into their Web browser. Their browser reads the code in that file and then sends a second and third request back to your server for the two additional files. Once the additional files are received, the visitor’s browser assembles the page.

When a search engine spider visits your framed Web page, it also sees the Main Frame (assembler) file, which is used to stitch the frames together. Unfortunately, the links from the main frame page (the stitching page) to the included frames pages (the body and navigation pages) are created using SRC links instead of HREF links. So all search engines will see is the code you use to assemble your frame pages, not your content or navigation. There is a workaround involving the use of the <NOFRAMES> tag, but then you have to maintain two versions of your website, one version spread out throughout your Content Files and one version included in your Main Frame files. That means editing your content twice for each change you make to any text on your site.

Google will follow your assembler page SRC links, but that’s not good news for you. Because your Content Frames don’t have any of your navigation in them (you pulled it all out to place in your Navigation Frame), the people who visit your page will see your content but without your site navigation. That’s a usability nightmare. There is a workaround for this issue as well, but it adds even more usability problems for your site, as it involves redirecting your visitor to multiple assembler pages when they land on your content pages from the search engines.

What the search engines say about frames and websites: A look back at search engine history (kept here for posterity)

The Inktomi spider, known as “Slurp, ” cannot follow SRC links in framed sites. The AltaVista spider also cannot follow SRC links in framed sites.

From: Inktomi – Slurp.

“Following links: Slurp follows HREF links. It does not follow SRC links. This means that Slurp does not retrieve or index individual frames referred to by SRC links.”

From: AltaVista – Help – FAQ – Adding & Removing URLs

“12. What types of Web pages does AltaVista not index?
AltaVista’s crawler does not index the following types of content: …
Information inside frames; AltaVista indexes the outside of the frame as a distinct page, so you should have non-frames as well as frames versions of your Web pages, and submit the non-frames versions via our Basic Submit or Express Inclusion services.”

Solutions for Framed websites

Don’t use frames in Web design.

Jakob Nielsen summed up many of the best reasons not to use frames way back in 1996. You can read why frames constitute on of the top ten mistakes in Web design on his site.

The <NOFRAMES> Tag Workaround.

One way around the problem is to use the <NOFRAMES> tag. This tag shows the search engines what they missed by refusing to follow the SRC links. Basically, you copy the content from your included body and navigation frames into the main frame page, including them between <NOFRAMES> and </NOFRAMES>. By putting your navigation and your home page content into the <NOFRAMES> part of your Main Frame, you are enabling the spiders to see what the visitor would see when visiting your site. You are also enabling the spiders to follow your links to the other content pages on your site. Unfortunately, your content pages don’t contain your navigation because you’ve put that in your Navigation File. So, your visitors will see your content, but they won’t see any of your main navigation links, leaving them stranded at a dead end or forced to follow the links you’ve included in your content. Another issue is the fact that many people who use framed sites put their corporate identity material in one frame to keep it at the top of the page at all times. Search engine visitors who visit your content pages will not be able to see your corporate identity material (site name, header, and logo) if you do this.

There is a workaround for this issue as well, but it involves putting code on each of your content frame pages that redirects visitors to your main frame page when they visit one of your content frames directly from a search engine. If you have just one Main Frame page, then all your search engine visitors will automatically be redirected to your home page. They won’t like that. You will have to build a unique Main Frame page for every page of your website if you want your visitors to be redirected to the page they intended to visit. Then you can use a Java redirect script that is unique on each content page. The script will check to make sure that the content page was loaded by a Main Frame page, and if not, will reload the browser using a Main Frame page so that the visitor sees the navigation and content.

In the end, you will improve your user’s experience with your site and cut down on your maintenance time by simply removing the frames outright so that the search engines can see your content as your visitors are intended to see it.