Search engines drive the major traffic to the most web-sites. To promote your web-site it's not enough to just analyze your content, link structure, etc. Knowing site's visibility in search engines is quite important too. You can Google for site:allforpromotion.com to see how many pages where indexed – but it's far away of being all picture. Here we comes to analyzing SERP (search engine results page). You must have set the main keywords for your site – and checking that each day (isn't that truth, ah :) ). But you can't check all the key phrases – especially that has low frequency words. But millions of Internet users can.

So here we come to very important thing – keeping track of queries which were used in search engines to reach your site.

How to figure query?

When you do search in each search engine (e.g. “what is search engine") – that query will be added to URL of the page. It will become like that: http://www.google.com/search?hl=en&q=what+is+search+engine. So to original address Google (let's take Google for example), added parameter q= in which the query was written (white spaces where replaced with "+"). Other search engines doing the same way (but each of that has it’s own parameter name, like text= for Yahoo). See the List of all major search engines on our site.

So when guys do searching and click on link (hopefully yours) nothing special happens – he just goes to your site. But the previous URL is stored in the referrers of current http request, and you can take it from there.

Implementation of the algorithm in ASP.NET

EFEX Consulting uses ASP.NET platform for almost all of the projects. Why? Read "ASP.NET conquers the world" – it’s really clear and interesting.

Anyway, here’s how it should be done in ASP.NET 2.0:

Step by step implementation.

Step 1. Figuring the search engine.

Which search engine it is – we need to get the domain information. Here’s peace of code.

string domain = GetDomain(Request.UrlReferrer.AbsoluteUri);

public string GetDomain(string url)
{
	if (url.IndexOf("//") >= 0)
	{
		url = url.Substring(url.IndexOf("//") + 2);
		if(url.IndexOf('/')>=0)
			url=url.Substring(0,url.IndexOf('/'));

		if(url.StartsWith("www."))
			url=url.Substring(4);
		return url;
	}
    return "";
}

Step 2. if the domain is search engine.

We check out our list – if this very domain is search engine, if yes select from our list name of parameter, which is used by search engine to store query (you should use database for that of course – for flexibility)

Step 3. Retrieving query from referrer string.

string word=GetQueryParam(Request.UrlReferrer.AbsoluteUri,wordParam);

public string GetQueryParam(string url, string param)
{
	int pos;
	pos=url.IndexOf('?'+param+'=');

	if(pos<0)
		pos=url.IndexOf('&'+param+'=');

	if(pos<0)
		return null;

	int end;
	end = url.IndexOf('&',pos+1);
	
	string searchWord ="";
	if (end < 0)
		searchWord = url.Substring(pos + param.Length, url.Length - pos - param.Length);
	else searchWord = url.Substring(pos + param.Length, end - pos - param.Length);
	
	return searchWord;
}

With English queries it’s all, but if you deal with Cyrillic texts – you will have some more troubles. I faced that while setting up our Handy Statistic on few Russian web-sites. I will tell you more about that in next blog, and also will explain few ideas – how statistic system can be done.

If you have any questions / suggestions – feel free to send me an email to ravemans@yandex.ru

Anton Raichuk aka RAVEman
Chief System Architect
Efex Consulting