Implementing Clean URL's in ASP.Net to Boost SEO

This tutorial will show you how to implement truly clean URL's in ASP.Net and C# to benefit SEO and without using any commercial products.

ASP.Net doesn't seem to handle clean URL structures are well as Apache, and most of the tutorials I have seen do not give you what I would call a clean URL.

What Are Clean URL's in ASP.Net?

Clean URLs (also known as URL rewriting, static URLs and search engine friendly URLs) are designed to "clean up" the URL of a web page, remove variable names, increase readability and easier to remember. Consider the URL:

http://www.myshop.com/catalogue.aspx?prodID=JN451&prodTitle=suiting&level=0

This isn't a clean URL, it is impossible to remember, difficult to key in, looks horrible and lacks any form of SEO optimisation. Would this not be a better solution?

http://www.myshop.com/catalogue.aspx/suiting/0/JN451/

Easier to remember looks better, keywords are closer to the domain and enhances usability and maintainability. We can go one step further as well and remove the .aspx extension from the URL giving:

http://www.myshop.com/catalogue/suiting/0/JN451/

Where /0/ is the page number and JN451 is the internal product code for this particular application. The URL can be further cleaned by changing the product code to a keyword or two, but that isn't the focus of this tutorial.

Why are Clean URL's Important?

Aside from the points above, clean URLs can improve search engine rankings. Why?

1. Because you can remove unnecessary words and query string variable names, leaving behind a shorter and keyword-richer URL.

2. Consider these two URLs:

http://www.myshop.com/catalogue.aspx?prodID=JN451&prodTitle=jackets&level=0

http://www.myshop.com/catalogue.aspx?prodID=JN451&prodTitle=suiting&level=0

As far as a search engine is concerned, these are one page with multiple input variables. Input variables mean dynamic content and are less likely to be indexed because the content is likely to change. Search engines are known to index the content of dynamic pages much more slowly than that of static pages.

Converting to clean URLs creates static pages which will be treated as two separate pages.

Methods for URL Rewriting

For this tutorial, we will simplify the above URL to just

http://www.myshop.com/catalogue.aspx?prodTitle=suiting

.

Using Request.PathInfo

This technique will convert the URL to

http://www.myshop.com/catalogue.aspx/suiting/

removing the query string. Using Request.PathInfo does not use URL rewriting, but an object provided by .Net.

The Request.PathInfo property will return everything after the .aspx extension, so in this example, it would return "/suiting/". If you have more than one item at the end (i.e. /suiting/jackets/polyester/) it will return "/suiting/jackets/polyester" as one string. You can then process this string to retrieve the category and subcategories.

string fullPath = Request.PathInfo;

if (fullPath != "")
{
  string[] categoryList;
  categoryList = fullPath.Substring(1).TrimEnd('/').Split('/');

  Response.Write("Full path is: " + fullPath + "<br/>");

  foreach (string cat in categoryList)
    Response.Write("Found Category: " + cat + "<br/>");
}

Using URL Rewriting

An alternative method to PathInfo is the use of a URL rewriter. This will read the URL, examine the contents and modify it behind the scenes before the page is loaded.

Using this technique you can convert the URL to

http://www.myshop.com/catalogue/suiting.aspx

format (or you can modify the code to suit your requirements/preference).

You will need to create or modify your Global.asax to include the following code.

void Application_BeginRequest(object sender, EventArgs e)
{
  string fullUrl = Request.Url.ToString();

  if (fullUrl.Contains("/products/suiting.aspx"))
  {
    Context.RewritePath("/products.aspx?category=suiting");
  }
  else if (fullUrl.Contains("/products/jackets.aspx"))
  {
    Context.RewritePath("/products.aspx?category=jackets");
  }
}

This code will look for a particular string in the URL and if found, rewrite the URL to your existing products.aspx page. You can modify this technique to remove the aspx extension as well.

The downside of this technique is that you will need to maintain mappings for every category, if you are constantly adding, or you have many categories this will be a maintenance issue. As a solution, you could create a Hashtable (key and value pair) which can be serialised and saved as an XML document. This could be loaded and processed dynamically. There are also commercial products available that will manage rewrites for you.

URL Mapping with RegEx Support

Our final solution utilises the powerful Regular Expression (RegEx) syntax to create a powerful and scalable URL mapping solution which works on the same principles as above, except that rewrite rules are specified in the web.config and no code is required in the Global.asax.

You will need to head over to ASP.NET 2.0: URL Mapping with RegEx Support for the source, it is also included in the sample website project below.

Three Visual Basic scripts should be placed in the App_Code folder, and you need to make a few changes to your web.config.

The first change declares a custom 'RegExUrlMapping' section and handler that we will use further down.

<sectionGroup name="system.web">
  <section name="RegExUrlMapping" type="RegExUrlMapping_HTTPModule.RegExUrlMappingConfigHandler"/>
</sectionGroup>

Next, we need to tell ASP.Net to use the RegEx URL Mapping Module:

<httpModules>
  <add type="RegExUrlMapping_HTTPModule.RegExUrlMappingModule" name="RegExUrlMappingModule"/>
</httpModules>

And finally, we define the RegEx URL Mapping parser and rules, which are processed in sequential order.

<RegExUrlMapping enabled="true">
  <add url="~/catalogue/(.*).aspx" mappedUrl="~/RegExRewriteDemo.aspx?category=$1"/>
  <add url="~/About.aspx" mappedUrl="~/RegExRewriteDemo.aspx?page=about"/>
  <add url="~/ProductInfo_(.*).aspx" mappedUrl="~/RegExRewriteDemo.aspx?product=$1"/>
</RegExUrlMapping>

With each rule, the URL attribute specifies the request URL (the one the user entered in the browser). You can use any RegEx to match, in these examples (.*) will match anything, so when a user navigates to /catalogue/abc.aspx, the first rule is triggered and ABC is passed into the category parameter of the mappedUrl attribute using $1. You can have multiple matches substituting for $2, $3 and so on, for example:

/PostArchive/2008/05/01/

can be mapped using

<add url="~/PostArchive/(.*)/(.*)/(.*)/" mappedUrl="~/posts.aspx?year=$1&amp;amp;month=$2&amp;amp;day=$3"/>

The advantage of this method is that you can create multiple dynamic pages (i.e. one for each product), but they are all mapped to a single page which will reduce maintenance drastically. Because the rules are generated through RegEx they don't need to be modified when you add new products/categories. The downside is that you must validate every parameter in the code to ensure that valid data is being passed in (but you are doing that already aren't you?)

So, going back to our original ugly URL:

http://www.myshop.com/catalogue.aspx?prodID=JN451&prodTitle=suiting&level=0

We can use a RegEx rewrite rule to tidy up this URL without changing the existing catalogue.aspx page.

New URL:

http://www.myshop.com/products/suiting/0/JN451/

<add url="~/products/(.*)/(.*)/(.*)/" mappedUrl="~/catalogue.aspx?prodID=$3&amp;prodTitle=$1&amp;level=$2"/>

URL Rewriting with URLRewriter.Net

Another method uses some open-source software called URLRewriter.Net  that allows rules to be set up for your rewrites. I haven't had a chance to use this method in a project yet, but DotNetGuts has written a simple guide to use and implementing URL rewriting with URLRewriter.Net.

Which Solution is best for Clean URL's in ASP.Net?

Each solution has its advantages and disadvantages, so its use should depend on your requirements.

PathInfo

Pros:

• Quick and easy to implement
• Little server overhead

Cons:

• Aspx extension is still required
• Directory structure after aspx extension looks messy

URL Rewrite

Pros:

• Faster and more efficient that RegEx rewrite
• More flexible than PathInfo
• Does away with aspx extension
• No requirement to change existing pages

Cons:

• Only matches exact strings, not wildcards so each product will need a condition
• Code can get unmanageable with many conditions

RegEx Rewrite

Pros:

• Most flexible and fewer rules required
• Does away with aspx extension
• No requirement to change existing pages
• No code changes in global.asax or code behind

Cons:

• Code can get unmanageable with many conditions
• Each rewrite rule is processed sequentially, even if a match is found, so can slow the server down if there are lots of rules.

Sample Project

Available to download below is a sample project which illustrates the use of all three URL rewrite techniques shown above. When you navigate to Default.aspx you will be presented with a list of clean URLs which map to PathInfoDemo.aspx, URLRewriteDemo.aspx or RegExRewriteDemo.aspx. The code behind is kept as simple as possible, without any checking or validation and a simple Response.Write to keep the solution as simple as possible. These samples will need customising to suit your requirements but will work out of the box.

Clean URL's in ASP.Net Sample Project