March 10th, 2005
The latest hoax in Search Engine Marketing
Commentary — Search Engine Watch is mentioning a research conducted by the Search Engine Marketer Enquiro on the way users look at a screen on Google.
Expectedly, users mostly look at the top results in the normal search result list, and the sponsored result list (color-coded image). It forgets to mention several things, however.
- Are these eyeviews turned into straightforward clicks, or are they only used to validate the search query, i.e. "Are my results relevant enough to go through the list, or should I rephrase the query?"
- Do the clicks on the top results produce more sales, or, as Sam rightly suggests, do the clicks on the last results produce more sales (more on the reason for this another time)?
Filed under Blog by on Mar 10th, 2005. 
Comments on The latest hoax in Search Engine Marketing
I'm not sure why you're referring to this study as a hoax. Are you saying they didn't actually do the research described, or that it's otherwise fraudulent? Just because you don't agree that the results are relevant to marketing and sales… and I don't know enough about marketing to comment on that one way or another… doesn't mean that it's a hoax. That term really ought to be reserved for situations where deliberate fraud is involved.
Er… We've a positively irrelevant methodology that leads to the presentation of a proof (as in the Cmabrigde uinervtisy rscheearch), and the meaningless proof is then served to customers as a sales argument. Isn't this a little bit of a fraud?
To be more precise:
Stopping to read upon seeing the top results doesn't mean you found what you're looking for; more likely, you quickly scan the first results, conclude on the relevance of your query, sometimes click on the first just in case, and search again.
But the Enquiro research merely emphasizes that users click on the first result — a reasonably obvious result don't you think? — and do not explore why users do so.
Moreover, the Enquiro research does not mention the conversion rate. Whereas, this is very much the only thing that counts if you're a webmaster.
Now, it is worth noting here that given 10 powerpoint slides, you tend to remember the first few and the last few. It is a perfectly normal pattern: It comes from 10 being greater than 7, plus or minus 2 — the maximum number of items a typical individual can manipulate.
Likewise, if you feed 10 search results to a visitor, expect him to remember the first few (stored for comparing with the next result), the last few (the last seen), and one or two in the middle (the ones that left him least indifferent).
Likewise, when it comes to choosing which product to buy, expect a few visitors to go for one of the first results. But as the visitor compares more products, he is also more likely to act consistently and buy. Thus, expect many more to go for the last result, or for the last result that left him least indifferent.
Didn't I just say that I'm not arguing with your conclusion about the relevance of the results to marketing?
What I'm saying is that if they actually did the study and got the results reported it may be an inappropriately applied study, it may be a study that they're abusing, but it is NOT and should NOT be referred to as a "hoax". That word has a specific meaning, and it doesn't mean what you're using it to mean.
Christ, you're in marketing, you know how important using the right word is. Don't tell me you can't see that this one isn't the right bloody word?
Yes, I'm quite aware of the importance of words. Which is why I call this a fraud.
The thing that troubles me here is that:
- These people are search engine marketers
- This research benefited from an obscene amount of press coverage by uncritical journalists
The net effect is this study is now more or less authoritative on the topic, and Enquiro is feeding their conclusions as a sales argument: "You need to buy our services _because_ our results show blah blah".
Now, imagine that the same gullible journalists report my self-proclaimed "proof" that attending my seminars on brainwashing is an absolute necessity to their business. You'd likely call it a fraud.
Yet trust me: My seminars are _a lot more_ interesting than their report's conclusions. ;)
"Yes, I’m quite aware of the importance of words. Which is why I call this a fraud."
Fraud and hoax are different words. Hoax implies the whole thing is made up or faked, all the way down to the eyetrack pictures. If you don't mean that, then don't call it a hoax. Because if you do, people will abuse (and ARE abusing) your words to discredit the real and useful (if less than interesting from a marketing viewpoint, if you're correct) results that are not, so far as I can tell, faked.
Tell me, do you have any evidence that they are? If not, please stop referring to it as a "hoax". because it isn't.
And, by the way, I can point to two far more egregious examples of fraud in the past two weeks alone. I don't need to "imagine" them.
This whole argument is a bit ludicruous. The research hasn't even been released yet, as you'd realize if you took the time to read the press release. This was one slide that was released as part of a presentation. We said we weren't suprised by the results. We expected them. Again, this was stated in the release. And we never used it as a sales argument to promote our services.
Perhaps before you rush to judge, you should take a few minutes to read the source material. And perhaps you should be more careful about your wording. Finally, perhaps you should take some of the obviously huge amount of the time you have on your hands and help contribute to understanding search rather than ranting on your online soapbox.
You are so right. Only preliminary results have been published. As well as a few notes on the methodology. Obviously, I must have explored the topic a bit more than you suggest before ranting about it.
Now let's take a closer look at your results, which indeed you expected as much as I did.
> The key location on Google for visibility as determined by the eye activity in the study is a triangle (…). This key area was looked at by 100 percent of the participants. (…)
> There seems to be a “F” shaped scan pattern
I concede. You have a breakthrough here. You've proved that journalists acknowledge the behavioral patterns of 50 participants are trustworthy statistics. And you've discovered that:
1. On a screen where users know the interesting stuff is on the top, they go for the top
2. About half of the users look at the sponsored results, which is consistent with previous research that suggests many users trust sponsored results as much as unsponsored results
3. Users read from left to right and don't finish reading a line if it doesn't look interesting at first glance
4. Users read from top to bottom and don't finish reading a page if it doesn't look interesting at first glance
> "We see a marked difference in how people say they search and what they actually do. Previous research had indicated that people were considered searchers and spent some time before choosing a link. (…)"
I do not recall reading anything credible that suggested what you state. On the contrary, I recall reading common sense remarks by Jakob Nielsen on the way users scan web pages that suggest users are everything _but_ searchers. Moreover, these common sense remarks are perfectly consistent with decades old research that shows the average Joe such as you or I is much more mindless than mindful.
> We never used it as a sales argument to promote our services.
Oh come on… It's the featured item on your web site's front page. And you're just as aware as I am that this research plus the attention it got works towards increasing your credibility.
In any case, the amount of attention your research got does prove two things: You _are_ an excellent marketer and you can be trusted to promote a web site or a meme.
> While interesting, the study’s main findings are still to come and will required detailed analysis of individual behavior patterns.
Be my guest and send me a free copy when it's available. I might decide to make an educated comment.
> You should take some of the obviously huge amount of the time you have on your hands and help contribute to understanding search.
If you had bothered wondering why I mentioned your research, it may have occurred to you that this online soapbox has no other purpose.