Analytics Talk

This is part 1 in a two part series about tracking on site search with Google Analytics. In this post I outline the setup and configuration you need to perform to create accurate, usable data in the new reports. If you’d like to get a look at the reports, you can skip to part 2, where I talk about the reports and how to use them.

How to Set It Up

The new search reports are based on a query string parameter that contains the search term that the visitor entered. All you need to do is specify the name of the parameter in the profile settings. As GA processes the data it identifies the search term and builds the reports.

It is important to note that on site search processing happens BEFORE filters are applied. This means that we have no way to modify the way that search data appears in our reports. I personally believe that this is a mistake and it limits our ability as practitioners to generate the data that we need for analysis. I’m hoping that Google can somehow change this.

Basic Setup

The first thing you need to do is turn on site search reporting. This is easy, it’s an option in the profile settings (please see the image below).

Next, you need to tell Google Analytics which query string parameter contains your search term. The best way to identify the query string parameter is to do a search on your site. If I do a search on my site for ‘world series’, and the search results page URL looks like this:

http://www.website.com/search_results.php?q=world+series

then I would add ‘q’ to the search term field. Pretty simple, huh? The query string parameter goes in the ‘Query Parameter’ field (again, located in the profile settings).

You’ll notice that I’ve added two query string parameters in my example. The epikone.com website has two different search engines. You can define up to 5 parameters. This gives you a lot of flexibility if you are using multiple on site search engines.

Category Setup

Many search engines allow users to refine their searches using categories. For example, if I search for ‘lavender’ the search engine might let me focus my search in the ‘soaps’ category or ‘tea’ category. This configuration drives a specific report that show the search terms for each category.

Setting up the categories is the same as setting up the search terms. All you need to do is identify the query string parameter that identifies the category. In the following example I would enter ‘cat’ in the Category Parameter field.

http://www.website.com/search_results.php?q=lavendar&cat=soap

Other Options

There is one final setting that is very important. You may have noticed the radio buttons for strip query parameters out of URL. This setting will remove the on site search query string parameter from the URL after on site search processing is complete. This means that the query string parameter, and the search term, will be removed from ALL other reports. It’s very much like the exclude query string parameters setting in the profile settings.

If you choose yes, GA will consolidate pageviews for the search results page. This can be good and bad.

Here’s an example. If you do a search on my blog the URL will look like this:

http://www.cutroni.com/blog/?s=world+series

Now, if I choose to remove the query string parameters GA will process the site search and then merge the resulting URI with all other instances of

/blog/

As you can see this is going to INCREASE the pageviews for /blog/. Is that good or bad? In the case of my blog it’s a bad thing because /blog/ is also the homepage. Removing the query string parameter will artificially inflate pageviews for the homepage.

However, if your on site search engine has a specific results page, like:

http://www.site.com/seach-results.php?q=world+series

Then removing the query string parameters is a good idea. The reason is you’ll have a single line item for your search results page in your reports and you’ll still be able to analyze on site search data using the On Site Search reports.

Common Problems

What if your on site search results page does not contain the search term in the URL? This is usually the case if your search form is using the POST method.

The workaround is to alter the way the search results pageview is created. This means manually creating a pageview for the search results page, that contains the search term. This is done by passing the tracking code on the search results page a value. Make sure that value includes a query string parameter containing the search term. Then specify the query string parameter in the on site search setup.

Here’s how you can modify the tracking code on the search results page to resolve this issue:

Old urchin.js implementation:
urchinTracker('/blog/search?q=<term>');

New ga.js implementation:
pageTracker._trackPageview('/blog/search?q=<term>');

Where <term> is the search term.

NOTE: If you haven’t heard, there is a new GA tracking code. You can read more about it in this post.

Optimizing the Setup

One thing that I’ve noticed is that the reports do not massage the data in any way. They report the raw keywords that the visitor entered. It’s great that we can see raw data, but it’s very easy to have duplicate data based on capitalization or spelling. Here’s an example. The following terms would all appear as different line items in the search reports:

Red Sox
red sox
redsox
red socks

From an analysis standpoint, the above terms are all the same. Visitors are searching for information about ‘red sox’. A certain amount of normalization would help, but we don’t want to eliminate too much data. Unfortunately the site search processing happens BEFORE filters, which means that we can not use filters to modify the data. I had initially thought we could use filters, but that is not the case.

Any data normalization MUST be programatic, it can not be done with filters! So, if you want to force all of your search terms to lowercase you must modify the page tag on your search results page. Specifically, you need to change how the pageview is created in GA. Here’s a sample piece of code that will force the query string parameters to lower case. You’ll need to update the tracking code on your search results page.

New ga.js code:
pageTracker.trackPageview(document.location.pathname + document.location.search.toLowerCase());

Old urchin.js code:
urchinTracker(document.location.pathname + document.location.search.toLowerCase());

This code change will eliminate all duplicates caused by capitalization.

Unfortunately there is no easy way to remove duplicates caused by misspelling. If we could only use filters… :)

Other Uses

What I find interesting is that this framework can be used to track other things. For example, let’s say you have an online ticketing service and the search feature passes certain search characteristics via the query string. I perform a search for tickets to see the Red Sox play at Jacob’s field. I specified the price range of the tickets and the date of the game. The URL might look like this:

http://www.tickets.com/search?price=100-200&date=20071014-20071016

What if I configure the new search reports to use price as the search field and data as a category? The reports should show me some really cool information. I haven’t tried this yet, but it could be very useful.