Indexing of dynamically generated pages in Search Engines

Reading Time: 3 minutes

Dynamically generated web pages sometimes cause problems with search engines. If your web pages are dynamically generated, it’s likely that many search engines have had difficulty indexing them. Dynamically generated pages are great, simply because of how fast they load and mainly because no one has to create them. They are more often than not generated by a php script. But do search engines have problems indexing dynamically generated content?

Dynamically generated Web Pages Can Be Indexed

Many search engines don’t like dynamically generated pages because they may get into an infinite loop if they follow some dynamically generated pages. Google had an official statement about this in its webmaster guidelines:

“If you decide to use dynamic pages (i.e., the URL contains a ? character), be aware that not every search engine spider crawls dynamic pages as well as static pages. It helps to keep the parameters short and the number of them few. Don’t use &id= as a parameter in your URLs, as we don’t include these pages in our index.”

That was many years ago but more recently Google changed its position on that subject. The entry has been removed from Google’s guidelines but here’s the official statement from Google’s blog:
“Google now indexes URLs that contain the &id= parameter. So if your site uses a dynamic structure that generates it, don’t worry about rewriting it — we’ll accept it just fine as is.

Keep in mind, however, that dynamic URLs with a large number of parameters may still be problematic for search engine crawlers in general, so rewriting dynamic URLs into user-friendly versions is always a good practice when that option is available to you. If you can, keeping the number of URL parameters to one or two may make it more likely that search engines will crawl your dynamic URLs.”

What Does This Mean For Your Website?

If you have a website that generates dynamically generated web pages make sure that it uses only one or two parameters to create those pages, then it’s likely that Google can index your web pages without problems. But there are other issues with dynamically generated web pages, like duplicate content.

In some cases PHP scripts can generate the same content with different URL strings. If this is the case, then Google’s spider will find the URLs but each one will have identical content and this is a big problem, which can lead to penalties.

Fixing Problems With Dynamic Pages

In cases as above an easy fix is to assign one url as the canonical url. This tells Search Engines that, that link is the most important version.

More Problems
You should also be aware that search engine spiders cannot follow links from submit buttons. So content that is generated from a submit button will in most cases never be found by search engines. But if it is found by directly going to the page itself, it will most likely not have a title or description or may be broken.

This too is easy to fix but this time it requires that you make that page noindex nofollow. It is as simple as adding that parameter to your meta robots tag. That tells search spiders not to index the generated page.

Summary

So it is all good right? Should you go out and make your website a dynamically generated one? Let us not get too hasty and foolish. Sure dynamically generated pages are good but at the same time old school tactics still work best when it comes to SEO. Building a website the right way is a better approach and adding dynamic pages is just fine.

45 total views, 1 views today

Animesh Niraj

Creative Head at Amazing Workz Studios - The expert at anything was once a beginner, I am that beginner. Reach us at info@amazingworkz.com for any Website, Software, Design, SEO/SEM projects.