Monday, February 07, 2005

Lycos beats big guys in search derby


Lycos emerged as the most efficient place to search for information in my side-by-side comparison of the principal search engines.

Second place was shared by Dogpile and Yahoo. The other contenders, respectively ranked from best to worst in performance, were Alta Vista, Search Scout, Google, Ask Jeeves, Amazon's A9, Yahoo's Y!Q and Search.MSN.

Lycos and Dogpile ought to do better, because they synthesize searches from a number of sources. Accordingly, they handily beat such big names as Google, Amazonand MSN at what, presumably, ought to be their own game.

Lycos is an amalgam of HotBot, Raging Bull, Tripod, Wired and other search engines, according to its web site. Dogpile, the first runner-up, sifts through the results of Google, Yahoo! and Ask Jeeves.

My search bakeoff was inspired by the hubbub in the last few weeks surrounding the new stuff being rushed to the market by Microsoft, Yahoo and A9.

MSN just debuted its search engine, which, as will be demonstrated shortly, is not ready for prime time. Yahoo introduced Y!Q, a "first-of-its-kind contextual search technology," whatever that means. And A9 lately has added all sorts of embellishments, including a semi-illustrated Yellow Pages that lets you see an exterior photo (or not) of some of the businesses listed in it. (See post below.)

When I installed the new Y!Q search tool in my browser, I was instructed to test the new functionality by asking: "What is the gas mileage of my Toyota Sienna?" Not so amazingly, the rapid response provided perfectly relevant answers in a logical, efficient order.

When I asked Y!Q for the gas mileage of the gigantic new CXT pickup truck from Navistar, I found no useful links in two pages of results. "Could Yahoo be cooking the results?" I wondered.

Thus, I decided to ask each of several other search engines the following questions:

§ What is the gas mileage of my Toyota Sienna? (Answer: 18 city, 24 highway.)

§ What is the gas mileage of the CXT? (Answer: 6-10 mpg, with a tailwind.)

§ How many gallons needed to paint a room? (Answer: A gallon covers between 360 and 400 square feet.)

§ What number president was U.S. Grant? (Answer: 18.)

§ Value of euro? (Answer: It costs about $1.30 to buy one euro.)

§ What is length of Brooklyn Bridge? (Answer: 1.3 miles.)

§ Names of the Ashcan artists? (Answer: Robert Henri, Thomas Eakins, Thomas Anshutz, George Wesley Bellows, William Glackens, George Luks, Everitt Shinn and John Sloan.)

§ How many calories in a fried-chicken leg? (Answer Between 250-300. Not sure why the experts can't agree, but I am going for the lower number.)

Here's how performance was scored:

If the correct answer was in the top position in search results, the site got a 1. If it was in the second position, it got a 2, and so on. If the correct answer was nowhere to be found in the first two screens of results, the site got a 20. Therefore, the lowest score is the winner -- like in golf.

The results of the site searches were averaged for each search engine, resulting in the composite score in the table below. As you can see, Lycos, Dogpile and plain-Jane Yahoo (not Y!Q) outperformed the others by a significant margin and MSN came in a distant last.

The performances of About and LookSmart were so embarrassing that I left them out.


Blogger Peter said...

Cute, but I would question the quality of this testing. You shouldn't be typing natural language sentences into a search engine to get the best results. For example, Google answers your question on the first result if you search for.. << "toyota sienna" mpg city highway >> Actually, it answers it if you just search for << "toyota sienna" mpg >>. You get the answer for question 2 in the first result if you Google for << CXT highway mpg >>. For the Ulysses S. Grant question searching for << "Ulysses S. Grant" "was the * president" >> turns up trumps.

You could argue that this isn't what the majority of users would do, but this is also why the majority of users aren't professional researchers.

1:04 PM  

Post a Comment

<< Home