Using Static vs Dynamic URLs
The static and dynamic URLs are as much alternative as they are complementary approaches to looking at web pages, with the latter being significantly more difficult to handle, for SEO purposes or as the technical skills are concerned. Although dynamic URLs are often favoured by web developers, they may have implications for both SEO and the user experience, explaining the presence of such a vast amount of tactics and methods that are used to tackle such implications that come with them.
A Static URL is used to store the address of an HTML page on a server and deliver it to the users over and over again, exactly as it was stored. Its distinguishing quality is that it remains the same over multiple user requests of that page unless it is manually changed.
A dynamic URL is the address of a dynamic web page that contains server-side code which generates unique content each time it is loaded, as opposed to it remaining the same over multiple user requests. A dynamic URL’s distinguishing quality is that it may change depending on the page results from the search of a database-driven website.
The use of dynamic URLs is especially common on large e-commerce websites that contain a high number of products, with categories that require to be updated virtually in real-time. It’s also especially relevant where there’s an increased number of product specifications that are used in categorising, cross-filtering and re-ordering of website content. This concept is known as faceted navigation and allows users to choose a selection of products based on selected pre-defined filters. In such cases writing optimised static URLs by hand may seem too time-consuming, often unnecessarily sophisticated or even impossible, arguing the value of URL parameters and the wider use of dynamic URLs.
The server-side code of dynamic URLs is triggered by the so-called URL parameters, which are also known by the aliases of query strings and URL variables. URL parameters are the segments of the URL that follow a question mark (?). They are comprised of a key and a value pair, separated by an equal sign (=), and can be used in multiple numbers in a single URL with the help of an ampersand (&). URL parameters may serve a number of purposes, such as web tracking, re-ordering and filtering of content, identifying content based on certain characteristics, paginating, searching, and translating.
The major search engines have expressed that dynamic URLs are crawled and indexed in the same manner as the static URLs and are therefore not an issue as far as the crawl-ability and index-ability of content is concerned. However, there are several aspects that make this practice challenging from an SEO perspective.
- The most widespread issue associated with the use of dynamic URLs is duplication, often caused by a change in URL parameters that essentially leads two or more different URLs to show pages with highly similar or virtually identical content. This is the case for URLs using web-tracking parameters, as they change the URLs at hand even though their content remains the same, thus resulting in duplicate pages. This is also true for URL parameters used to re-order the content of a page. Because, although in a different order, the content of the two or more pages is essentially the same, and as a result, the pages represented by all their URL versions will be treated as duplicates. This may also occur with URL parameters that are used for identifying information, as these options are often represented by stand-alone category pages on the website. Occasionally, this can reflect over the filter parameters when a pre-defined filter option matches or is very similar to a website category. When URLs feature pagination parameters spreading content over multiple pages while having a “view all page”, it becomes problematic as each page of the sequence will use content also present in the “View all page”. Although not always, URL parameters used for on-site searches may coincide with website categories, thus yet again leading to duplicate issues. Also, when using URL parameters to show different languages on the website, particularly in cases when navigation elements are translated into a different language while the page content remains the same, the page variations will be treated as duplicates. Lastly, having URLs with not one but two or more URL parameter pairs may lead to duplicate versions of web pages caused by the order of URL parameters in the URL.
- Thin content issues caused by URL parameters are generally related to duplicated content but are different in the fact that website pages are not as much alike as they are empty or incomplete. This can be the case for identifying, filtering, searching and pagination parameters when they lead to pages that have little to no actual content or results.
- Search Engines tend to crawl and index websites at impressive speeds, but even they have limitations. This is felt particularly in the case of large websites that are updated regularly and need to be re-crawled. Thus, the notion of crawl budget comes in to define the capacity of search engines to crawl and index websites over time. The more variations of web pages are generated as a result of varying URL Parameters, the more of the crawl budget is drained without valuable contribution to the website being fully crawled, indexed and frequently updated.
- Another issue that comes hand in hand with duplication is the split of organic ranking signals. This happens when the authority signals coming from inbound links from other websites to different versions of the same page are not consolidated but split among these pages. In simple terms when a search bot crawls several similar versions of a page, these pages end up competing for the same keywords, leaving each of them in a weaker position against competition if compared to one singular version.
- Lastly, URL parameters make URLs less readable to the user that are less likely to rely on them to make a click-through decision. Not only does this have a negative effect on the click-through rate to the website from search results, but more recently this has also been associated with negative brand engagement as these URLs are seen in SERPs, reach social media websites, or even emails and messaging apps.
All these issues are the product of parameter-based URLs that once changed, are treated as new pages, becoming multiple variations of the same page, targeting the same keywords, flooding the SERPs, offering limited value to users and draining the crawl budget. It’s worth noting that the most problematic aspect of using dynamic URLs isn’t so much the issues caused by individual URL parameters, which can be easily tamed, as it is the sheer variety and volume of issues that may overlap and often become difficult to predict or control.
While this is unlikely to result in a penalty or entirely remove a website from organic search results, it will often decrease the overall quality of the website and each web page in particular in the eyes of search engines.
Parameter-based URLs are by far the most unpleasant aspect to deal with in optimising URLs for search engines, making it the very aspect that, given the right implementation, can give a website its’ competitive edge. There are a set of considerations to take into account when moving forward with taming the issues caused by URL parameters which should be implemented in a specific order, a manner that allows the process to be structural and manageable over time.
This order will depend on the specific URL parameters that are used on the site, the priority of issues depending on the specifics of the website, and some of the options can work together while others are not compatible.
Before moving forward with tackling the URL-parameter issues, one must ensure to use of static URLs wherever possible, leaving parameter-based URLs only for sections that truly require it.
It then begins with the correct implementation of pagination, as it may reflect on all other parameters when used as part of the same URL. The second task is, then, limiting the actual number of parameter-based URLs, where possible. For tracking, identifying and reordering parameters, the use of canonical tags can help with the consolidation of ranking signals for duplicate pages, giving some degree of control over the URL versions to be indexed and displayed in organic search.
For even greater control over all parameters, one may further investigate the potential issues and configure the way the parameters are handled directly in the search console or webmaster platforms for each search engine in particular. This is especially relevant for filtering parameters, used for faceted navigation, as it may lead to a large number of pages with overlapping content that are neither duplicates nor unique, defining the importance of these pages to be understood by search engines.
The only downside here is that if the website targets multiple search engines, one must use a different platform for each one to configure the handling of parameters. Lastly, especially in situations where the crawl budget is of particular importance, one can use robots meta tags or a robots.txt file, to control the manner in which all parameter-generated pages are crawled and indexed by all search engines.
Implementing pagination
When a website page needs to display a list that ranges in tens, hundreds or thousands of items, it becomes complicated as far as the UX part of a website is concerned. Thus, it is usually split into a number of sequential pages, each with its own URL, using pagination. These sequential pages are irrelevant for organic search in the respect that hardly any user searches for the second or any of the consecutive pages of a list, but they are still crawled and indexed by search engines.
This justifies the necessity to explain the nature of these pages and the relationship between their URLs, which are part of the same sequence, to the search engines. As it happens, the search engines are currently capable of distinguishing these types of pages on their own, but some, however small, additional value may still be obtained by not leaving the manner in which a website is crawled and indexed fully in the hands of a search engine bot.
It begins with the use of rel=”next” and rel=”prev” link attributes as part of the head section of the page only, to indicate the following and the preceding pages of a sequence, respectively. Thus, page 1 (also known as the root page) of the sequence will only have a rel=”next” link attribute, while the last page of the sequence will only have a rel=”prev” link attribute, and all pages in between will have both.
If the URL uses other parameters in addition to pagination, it’s only normal that one must include them in the rel=”next” and/or rel=”prev” link tags to accommodate this. Search engines treat pagination markup only as a strong hint as opposed to a directive, which is expected to consolidate the ranking signals of a sequence of pages to the root page (page 1), making it the page that shows up in search results most of the time.
Search engines may still sometimes rank numbered pages in search results, when other ranking factors overpower the importance of this relationship, but all other things remaining equal between the numbered pages, search engines will very likely favour the first page.
Pagination has been around for a long time during which the industry managed to produce a number of implementation techniques that have become outdated, misunderstood or are simply plain wrong. These techniques may actually be of great value in determining potential issues around pagination for existing websites and justifying an SEO intervention.
Limiting the number of parameter-based URLs
The second task is, then, limiting the actual number of parameter-based URLs by eliminating unnecessary parameters, preventing URLs that use empty values as part of parameters, allowing the use of multiple values with a single key, and defining an order for how URL parameters should be presented in URLs.
This can be achieved firstly, through the elimination of unnecessary parameters, or in other words, parameters which serve no actual function. This is often referred to as parameters caused by “technical debt”, which historically served a function but have been subsequently replaced. The most common example of this is session IDs, used for web-tracking in the past but being replaced by cookies.
Secondly, preventing the URLs with parameters that have empty values. Thus omitting URLs with URL parameter keys that have their parameter values blank will further reduce the number of unwanted URLs.
Thirdly, when using a parameter key with multiple values, make sure the parameter values are used together after the single key, as opposed to each value having its own key. This prevents potential duplication issues further down the line when depending on the order of key and value pairs, you may have multiple URLs for essentially the same page. Not to mention, this helps with shortening the URL which beneficially reflects on click-through rates and search engine rankings.
Lastly, ordering URL parameters by using a script to place them in a consistent order, regardless of how the users select them will further ensure that no additional duplicate URLs will be generated as a result of a different order of parameters within it. Although having multiple versions of URLs generated by a different order of parameters will lead to duplicated pages, this issue can be easily tackled by most search engines assuming they understand the parameters. Nevertheless, this remains a concern because the additional number of URLs will put a strain on the crawl budget and potentially lead to split ranking signals.
This contributes to tackling all 5 major issues caused by URL parameters, addressing the duplication of content, improving the efficiency of the crawl budget, consolidating the ranking signals into fewer pages, and in some cases even making the URLs shorter, with beneficial effects on the click-through rates.
After all, the best way of fixing the issues caused by URL parameters is to prevent them from happening, with which this strategy should help. However, this is unlikely to fix all issues caused by all URL parameters, as different parameters have different functions, causing issues that may differ in their nature and therefore require independent attention.
Using canonical tags
The canonical relationship between URLs predisposes that website users will continue to have access to multiple URLs for web pages that use these parameters while instructing search engines when any of these pages have one or multiple identical or highly similar alternative pages as well as which page among them is the preferred version. It’s worth noting that apart from canonical tags, search engines use a number of additional factors in determining which page should be treated as canonical among a number of highly similar or identical pages.
To interpret this just right, it means search engines understand the website’s preference as to which of these page URLs the consolidation of ranking signals should go into, and thus which version of the URL is preferred to be shown in search results, but they will only treat it as a contributing, often decisive, factor to this decision and not a directive.
As a consequence, search engines may and do end up using other than canonical URL versions to consolidate ranking signals, showing them in search results as opposed to those which the canonical tag points to, but all other things being equal between the two or more pages, this is more of an exception than a rule.
- Because the implementation of pagination comes first in taming URL parameter issues, when implementing canonical tags, one must begin with the pages that use pagination parameters. This predisposes the use of self-referencing canonical link attributes, to supplement the already existing rel=”next” and rel=”prev” link attributes, defining that each of these pages that are part of a sequence are, in fact, unique pages. A very important point here is that when the URLs have other parameters in addition to pagination, they must be included in the above-mentioned rel=”next” and rel=”prev” link tags, but never in the rel=”canonical” link tags, as this practice would otherwise cause search engines to treat these pages as duplicates. As an alternative, the practice of using canonical link attributes to point from all sequenced pages to the root page (as opposed to that of using self-referencing canonical link attributes) may lead search engines to not acknowledge the content on any of the following sequenced pages. This, in its own turn, would reflect poorly on the consolidation of ranking signals, as search engines might ultimately end up seeing significantly less content. Additionally, the use of a View All page that lists all items from all paginated pages poses the question of how to handle it. Although, at some point, an option recommended by search engines, the choice of using canonicals on all sequenced pages pointing to the View all page, thus favouring it to show in search results instead of the root page (page 1) is far from ideal. The main reason behind it is that this can not be applied universally, being particularly problematic for very large lists, putting a strain on page loading times, thus negatively affecting SEO. Not to mention that it goes against the very function of canonical tags, which are meant to deal with pages that show virtually identical pages and not pages that have content which overlaps only partially.
- It continues with the building of a canonical link attribute relationship between the URLs that are duplicated based on tracking, identifying and reordering parameters. In this case, the canonicals are used for URLs with these three parameters because they are likely to generate pages that are identical or highly similar at worst, as opposed to pages that share some duplicate and some unique content. Canonical tags are, therefore, not suitable for URLs using searching, translating or some of the filtering parameters, as their page content wouldn’t be similar enough to the canonical as such.
Lastly, although it partially takes the strain off the crawl budget, as non-canonical pages will be crawled less frequently, supplementing canonical tags with meta robots tags or a robots.txt file will take it one step further to solve this issue to its fullest extent.
Configuring parameters in Search Console and Webmaster Platforms
Lastly, websites, where the crawl budget is of particular importance, may also ease the strain put on pagination URL parameters by setting up the pagination parameters in Search Console or Webmaster platforms for easier control over their crawling and indexing over time.
Using robots meta tags or a robots.txt file
There are a number of robots meta tags used to instruct search engines how to behave in regards to the pages they are used on. However there are 2 meta tags that address the crawl budget at the page level in particular, the nofollow tag and the noindex tag. Using a canonical tag for a set of URLs showing similar or identical pages ensures the duplicate versions of the page will be crawled less often, but canonicals are not applied universally across all parameter-using URLs.
In order to take the strain of the crawl budget completely one can use a meta nofollow tag, which gives a directive to search engine bots to not crawl the page. This often comes hand in hand with a meta no-index tag, which gives the directive to search engine bots to not index the page at hand, thus keeping it out of SERPs.
When it comes to pagination in particular, the use of a no-index tag is by no way a substitute or supplementation to the use of rel=”next” and rel=”prev” attributes, because it would prevent the consolidation of the raking signals (to the root page or otherwise). In the long run, the use of a noindex tag on paginated URLs may also lead search engines on a path of eventually nofollow the content behind them, which explains the next point.
Paginated URLs shouldn’t have a place for a nofollow tag either, as the content shown in all the paginated pages following the root page, may end up remaining undiscovered by search engine bots or dropped from SERPs as a result. Either of these tags will most often be used for the purposes of dropping paginated URLs from SERPs because they are of limited value or because the pages following the root page show up first, but this ends up being done at the expense of properly consolidating ranking signals to the root page.
When confronted with an issue of other than the root page ranking in SERPs, one must investigate the real causes behind it, as opposed to relying on noindex / no follow tags to fix it.
Disguising dynamic URLs as static
Dynamic URLs may seem somewhat unrepresentative of the landing pages they hold, particularly because of the use of so-called URL parameters. These can sometimes unravel to great lengths making the content of the URL more intelligible to machines but less understandable to humans. Thus these can be replaced by custom URLs through redirects. This way static seeming URLs hold a more descriptive function and are more useful to the customers.
It is, however, quite hard to correctly create and maintain rewrites that change dynamic URLs to static-looking URLs. It is also much safer to serve the original dynamic URL and let the search engine handle the problem of detecting and avoiding problematic parameters. The only recommendation for simplifying the URL is to strip it of unnecessary parameters while maintaining the overall structure of a dynamic URL.
If you transform your dynamic URL to make it look static you should be aware that search engines might not be able to interpret the information correctly in all cases. Thus, if you want to serve a static equivalent of your site, you might want to consider transforming the underlying content by serving a replacement which is truly static.