Thursday, December 01, 2005

Montitoring Search Results

This begins a series of postings on methods for monitoring how well you are doing in the search engine results pages (SERPS). I’ll first be looking at direct measurement of search engine results, then I’ll discuss measuring how well marketing objectives are being achieved. I’ll be focusing on methods that can be used by anyone, large companies, small firms or individuals, and which require a minimal number of software tools.


I manually collect data about my client’s web sites every month. This means I perform a series of searches using the targeted keywords, and record the results in a spreadsheet. By collecting data manually I do not abuse the search engine’s limits on data mining, but more importantly it gives me an opportunity to get an overall view of the search results for each key phrase. I find that I notice things about my web sites, and about competitors web sites, that are not necessarily revealed by the data.

In the past I would collect data on ten search engines. Since the consolidation of search engine ownership, I now usually monitor just five search engines:

Ask (B to B) or AOL (consumer)

In some cases I add one or two other search engines, including some meta search engines, based on the client’s target market, and their customers search behavior.

The number of key phrases I use varies by client from five, up to seventeen.

As I search for each key phrase I record each instance of a web site that benefits the client in a program I've created. You can also use a spreadsheet as shown below. The standard practice is to look at the top 30 search results, but since most searchers don't even go past the first page, I only look at the top 20 search results for each key phrase. I record the position of the web page in the search results and the root URL of the web site.

Monitoring search results

What is a web site that benefits the client?

Any web site that benefits the client. This, of course, includes their web site. But it also includes web sites that have one of their press releases or an article about the client or one of their products. It may include dealers who carry the client’s products, if the dealer site does not steer customers to a competitive product. It could include blog posts, discussion forms, trade show sites and supplier web sites. Any web page that truly delivers value to my client is counted.

What about ads?

Whether or not to include ads as a part of the search results is a judgment call. If I think the search result page is designed such that a typical visitor is likely to consider the ads as a part of the search results, I count the ads. Thus if a search result page that has five paid ads at the top and ten organic search results, I may count it as having 15 search results. When I go to the next page I’ll only look at the top five “search” results on that page—which may all be paid ads.

Next – Determining Keyword Visibility


At Thursday, December 01, 2005 9:52:00 AM , Anonymous Anonymous said...

This seems like an arduous way to collect data about search engine positions. How long does it take you to collect the data when you are checking 17 key phrases? What method(s) do you use to collect the data? I know you say you do it manually, but what isthe actual process?

At Thursday, December 01, 2005 4:49:00 PM , Blogger SteveH said...

The time varies depending on how many web sites I look at, but 4-5 hours is a typical duration.

The method is to manually search for each key phrase in each search engine and then record the results.

I started this topic because today (the first of the month) is the day I research the 17 key phrases. Today I noticed a compeditor has brought a bunch (at least 6-9) mini-web sites online. This is something they do on a regular basis and it usually takes Google 2-3 months to eliminate the new web sites. Another compeditor has started using free classifieds to promote products to my client's customers. I also found a blog and two web site articles about this client's products.


Post a Comment

Subscribe to Post Comments [Atom]

<< Home