UPDATED: Integrating Google Analytics with a CRM
March 18, 2009 by Justin CutroniA while back I wrote a post about how you can integrate Google Analytics with a CRM system. That posts referenced the old urchin.js code and is in need of an update. Plus a lot of people have been asking me about different integration options and I thought it would be a good time to revisit the topic.
So here it is, an update. I’m not going to re-do the entire post, just a few key points.
Why Do This?
This entire hack configuration is based on one simple idea: to build a more robust view of customers/prospects/leads/whatever-you-call-them. The more information we know about people that we sell to the more we can adjust our marketing. This technique let’s us combine where the visitor came from (referral, campaign, direct, search engine) and how often they’ve been to the site with CRM data.
How It Works
We’re going to extract data from the GA tracking cookies and add it to a lead generation form. The GA data will be stored in hidden form elements that the visitor can not see. When the visitor submits the form the GA will be connected to the other information that the visitor entered into the form (usually name, contact information, etc.).
Here’s the data that we’re going to pull from the GA cookies:
* Current referral information (i.e. where the visitor came from)
* Custom segment value (if it exists)
* How many times the visitor has been to the site
You’re probably wondering why we’re not extracting the data via the GA API. More on that in a minute.
The Code
Here it is, the updated code:
The new code differs from the original code in three ways:
First, the inclusion of ga.js. No explanation needed there.
The second change is the addition of the _uGC() function. This function was in the original urchin.js, and because I don’t include that file I need to include the code in the script.
The function is used to parse pieces of data and return certain parts. For example, I use to to parse the cookies (which are just a string) and return the value of the campaign cookie (__utmz), the custom segment cookie (_utmv) and the visitor identification cookie (__utma).
Finally, I added some new functionality to include this visitor’s number of visits. This data is pulled from the __utma cookie (which identifies the visitor). The last integer in __utma is actually a visit counter. I think it is interesting to understand behavior based on frequency so I added this nugget of data.
If you want to see the code in action try this:
In my previous post I included a reference section that documents the format of the campaign cookie and the _uGC() function. Check it out if you’re looking for more technical information.
An Alternate Approach
Remember, all the data we’re pulling is in first party cookies. This means that you could extract the info at the server level rather than the browser level (first party cookies are available to server side code).
You don’t need to do this in JavaScript, as I did. You could use ColdFusion, PHP, .NET or some other server side language.
I’m just trying to provide some inspiration here :)
What About the Google Analytics API?
I’m sure there are a lot of people out there wondering about the GA API and why I don’t just use it to pull data out from GA. The reason is the data in GA is all anonymous, aggregated data. There’s no way to reach into GA and say, “tell me where John Doe came from.” That’s why we pull data from the cookie, because we can combine it with a visitor’s info when they supply it.
Sure, it is possible to pull out GA data and place it in a CRM (like traffic from an email campaign, bounce rate from paid search, etc.) but it’s tough to identify individuals in that data.
Is it impossible? No, not from a technical standpoint. You would need to place a unique identifier in GA to link the data.
But we’re getting into a risky area here.
Remember, it is against the GA terms of service to capture data in GA that identifies an individual. This could be an email address or database ID number. Not my rule, Google’s, and they will crack down.
I’d be interested to hear people’s thoughts on this. Feel free to comment!
30 Responses to “UPDATED: Integrating Google Analytics with a CRM”
Thanks for the updated code Justin. We’ll be sure to use it when we switch to the new tracking code. (We use the old version with great sucess right now!).
By Boris on Mar 19, 2009
Closing the sales loop on marketing is the holy grail of results-driven marketing. Great, informative post.
By Adriel Michaud on Mar 19, 2009
Hello,
Thanks a lot for this post! One of my clients had some concerns about implementing this kind of script on their website, but I would love them to do so because it would enable me to link leads ans actual sales, providing an ROI for every marketing initiative.
Their concern is:
As a publicly traded company, they are worried about privacy issues and they have raised some questions about the fact that we would be collecting data about a user without him/her knowing it. Every Analytics tool does it, but as you mentionned in this article, they do not relate keywords or sources to specific individuals. This script goes a step further.
My question is: are large organizations using that level of precision when tracking online forms? If so, is there any privacy issue, or should it be mentionned somewhere that this kind of data can be collected by the website?
Since I would really like to convince them to use this script, any argument “in favor” or examples of large organizations using this kind of method would be very appreciated!
Thanks again,
Antoine
By Antoine Laurier on Mar 19, 2009
Thanks for the feedback everyone, glad you like the post and thanks for reading!
Antoine, to address your concern. This script only executes and collects data when a visitor submits a form. By submitting the form they’re sharing data with you, this form just collects a bit more.
As for examples, I can’t talk about specific clients and what they do or don’t do :)
Thanks again everyone.
Justin
By Justin Cutroni on Mar 19, 2009
Hi Justin
Thanks for all the great posts.
If I knew that a customer bought a certain product on a certain day, I could use that product ID to link the web analytics data to back end data, often to individual level. So in many cases the capacity is there already to link to PII. I personally do not have strong feelings about companies tracking my behaviour but I acknowledge and abide by the fact that many people are bothered by this. However I find the GA privacy policy vague and contradictory, even without storing down personal details with my analytics tool I often have the capacity to link behaviour to back end demographic data at individual level anyways.
Best wishes
Niamh
By Niamh Phelan on Mar 23, 2009
Thanks. Great info.
Was looking for a solution to track the quality of leads the sales team is receiving via online channels.
Your solutions is exactly doing that.
By Floris on Mar 23, 2009
Justin, great article. Love reading all these techniques/hacks.
Would really be neat to see more writing on CRM and Google Analytics. Maybe a list of some really popular CRM solutions and how the matching is being done. I assume it’s based on logged-in/registered users so it’s first name/last name or a unique key identified to them.
Would also be neat to see a table showing the pros’s and cons of server-side extraction vs. browser level in-terms of accuracy, how complicated, etc.
Finally, would like to see some articles showing how the Google Analytics data is being fed into the CRM for metrics such as engagement, etc.
By Brad Warthan on Mar 23, 2009
Great updated post, Justin! And it was a pleasure meeting you, if ever so briefly, at the Google offices.
By Hugo at Zeta on Mar 24, 2009
Justin as always your posts rock man and I am going to totally suck for doing this but I need some help on this one and cant find the expert advices elsewhere.
Google Website Optimizer Technical Question
Situation is this:
Have a site with product pages that dont see enough traffic to each individual product page for a multivariate test (Google website optimizer) without waiting months for results. For example lets say I have 100 product pages each seeing about 200 views a week. I want to setup a simple multivariate test with different checkout buttons and locations. For this example lets use total of 10 combination’s. If I run that multivariate test on even my most popular product page with only 200 views it will take forever to get solid results and to me Google was just never have enough solid data to calculate. Now if i could aggregate all the product pages into a test 100×200 so 20k views a week that is the kind of data i need GWO to see. So my questions is this:
Does the Javascript really care about the Test Page URL or even Conversion Page URL for that matter as long as the scripts are referenced. In other words I want to make 100 product pages look like one test page they will all have the same scripts on top of page and at bottom of page and then the checkout will always have the same conversion script. Has anyone tried something like this or any expert in JavaScript out there know if this will work? Any insight would be much appreciated.
-parky
By parky on Mar 25, 2009
I would love to see a comment from Google here re their T&Cs and how to work around it for a CRM.
By Joakim Nilsson on Apr 3, 2009
Hi Justin and Everyone,
My 2 cents for GA API ! If you put unique identifier , lets say the value of a cookie (or part of it), does it uniquely identify individuals ? Also, fromt a privacy standpoint,with cookie or unique value , isnt it just hard to trace down individuals (unless they did a transaction on site?
does it really uniquely identify human attached to that cookie ??!!
On second thought, a good revenue stream for adwords is their API,as every server call makes money for google. If Google Analytics to go that path, it would be great for Google to make money out of the API server calls from Analytics ! However to make that happen this privacy thing really needs a great clarification i believe.
Great work on code as expected !!
By Ravi Pathak on Apr 5, 2009
Hi Justin,
This is a great article. I thought it was interesting because with that integration, you’ll be able to tie number of visits to desirable action and other info attached to it.
This definitely helps CRM to segment various communications based to user traffic behavior to a certain level.
Great stuff!
By K Irizawa on Apr 17, 2009
Thanks K. I was thinking the same thing about the number of visits. Glad you liked this integration and thanks for reading the blog.
Justin
By Justin Cutroni on Apr 17, 2009
Thanks for the great blog! Just a comment about the GA TOS. I would say the reason GA has that TOS is that it is against the privacy law (at least here in the Netherlands but I assume in most countries) to exchange data that identifies an individual between two organizations, without the expliciet consent from the end user. As the statistics data from GA is owned by Google, not by the original site, this means this is an exchange of privacy data, hence not allowed, and will never be. So it’s not Google’s wish to have the TOS like that, they really have no choice.
I think to (legally) have real identifiable data (detailed profiles of users for example), the only way to do that is to have software in your own datacenter, not in a SaaS/hosted environment where you don’t own the data.
I would think that if you DO create(from a technological viewpoint) identifiable data in GA, you could get in trouble not only from Google itself (violating the TOS), but also from visitors of your website (if they find out), as you would be tresspassing the privacy law of exchange privacy data with a third party without explicit consent of the enduser (visitor).
I am no legal expert by any means, just my interpretation of how privacy legislation works.
By Martijn van Berkum on Apr 21, 2009
Thanks Martijn for the thoughtful analysis! I’m not sure, but I believe that the TOS for various geographic regions is different and the European TOS is more strict. I could not agree more that adding personal visitor data to GA is a bad idea. Plus, in my opinion, it’s just not actionable. That’s why I like this method a bit better.
I would assume that submitting a form, that has a disclaimer, would constitute “explicit consent”.
Thanks again for the comment and reading the blog.
Justin
By Justin Cutroni on Apr 21, 2009
The long term issue from my point of view is that the industry (at least the industry I work in: Web Content Management) is working towards a complete customer profile, to build up as much knowledge about the visitor as possible. Only then you can provide the visitor the content/products/information he/she needs, or in the case of CRM, provide sales with the best information about their customers. But with the current IT trend to ‘outsource’ parts of your applications to SaaS solutions like Google Analytics, you cannot legally put identifiable data to these solutions. I am curious how the IT world will solve this the coming years. Asking for expliciet permission is one way to solve it.
But this is all a bit offtopic for this blog entry of course ;)
By Martijn van Berkum - GX on Apr 22, 2009
Hi Justin,
Just read your article. Why don’t you just pull the cookie data out of ga.js with _getLinkerUrl(), and pass back through the utm.gif parameters, so the CRM can just make image requests to GA?
That way you can send back pageviews (conversions) as well as transactions and products.
At least that the way I’d do it :)
By Nick on May 12, 2009
Hey Nick,
That’s another way to do it. I’d say it’s about the same complexity :) That’s the thing I love about GA, there’s always more than one way to get something done.
Justin
By Justin Cutroni on May 18, 2009
Justin,
Thanks for the great “how to”. I have converted part of the script to asp.net vb. Hopefully someone can benefit. Code snippets are below.
Ethan
Dim strCookieValue As String = “”
Dim strKeyword As String = “”
Dim strStart As String = “utmctr=”
Dim strEnd As String = “|”
Try
strCookieValue = ReadCookie(“__utmz”)
strKeyword = getValue(strCookieValue, strStart, strEnd)
Catch ex As Exception
‘just swallow exception and move on
strKeyword = “”
End Try
Private Function getValue(ByVal strSearch As String, ByVal strStart As String, ByVal strEnd As String) As String
Dim strRtn As String = “”
Dim strWorking As String = “”
Dim intStart1 As Int32 = 0
Dim intStart2 As Int32 = 0
intStart1 = strSearch.IndexOf(strStart) + Len(strStart) + 1
strWorking = Mid(strSearch, intStart1)
intStart2 = strWorking.IndexOf(strEnd)
‘if no end found take the whole length
If intStart2 = -1 Then intStart2 = Len(strWorking)
strWorking = Left(strWorking, intStart2)
‘replace HTML encoded spaces
strRtn = Replace(strWorking, “%20″, ” “)
Return strRtn
End Function
Private Function ReadCookie(ByVal strName As String) As String
Dim strCookieValue As String = “”
Dim ga_cookie As HttpCookie = Request.Cookies(strName)
If Not ga_cookie Is Nothing Then
strCookieValue = ga_cookie.Value
End If
Return strCookieValue
End Function
By Ethan Johnson on May 29, 2009
Wow, thanks Ethan! I appreciate it!
Justin
By Justin Cutroni on May 30, 2009
Hey everyone!
Has this script been configured to PHP yet? I don’t have the skillset to do it myself, but I am told that’s how I need to have it by a friend of mine.
Thanks!
By Adrian on Oct 16, 2009
Hey Adrian,
Unfortunately no. But if you know a PHP developer they should be able to figure out he to translate the functionality out of JavaScript and into PHP.
Thanks for the question,
Justin
By Justin Cutroni on Oct 20, 2009