We are now including Search Engine News from a variety of different sources. Below are a few articles written within a few weeks of the release of this newsletter.

ResearchBuzz

AlltheWeb's Search Engine Offers New Options

Monday, 16 September, 2002

I always like to see competition in the search engine world. If Google's bothering to look back over their shoulders for anybody, it better be AlltheWeb.

AlltheWeb continues to offer fun search options and some cool tech. Check out their new advanced search options at http://www.alltheweb.com/advanced . You can now specify that a returned result page should or should not contain a variety of content types, including images, audio, video, Real content, Flash, Java applets, JavaScript, and VBScript.

And check out the further restrictions section. You can restrict results by date updated, document size, and DEPTH! What is depth? Depth is how many subdirectories AlltheWeb will go down searching for results. For example, using AlltheWeb you can do a search and specify that your search results come only from the top level. So a search might pull results from www.cnn.com, but not www.cnn.com/technology/ .

You can also specify that you search results be *below* the top level, meaning that results would not come from www.cnn.com, but might come from www.cnn.com/technology/ . The AlltheWeb interface allows you to specify that results must be above, below, or exactly as many as ten levels deep. You can also click a checkbox that specifies result pages must be personal (that is, have a tilde, a ~, in the URL.)

Fortune

Grading the Search Engines

Think your search engine is the best one around? You may be in for some surprises. FSB Browser columnist Maggie Overfelt offers her take on the best and the worst sites where you can troll for information.

Fortune Small Business

Thursday, 29 August, 2002 - By Maggie Overfelt

If there's one thing web surfers are always looking for -- aside from a naked newscaster they can trust -- it's the perfect search engine. So it's hardly a surprise that Google and its rivals each claim to have one kind of superiority or another. But how's a casual surfer to know?

That's where we come in. To test the top search engines, we ran dozens of queries in a multitude of subjects on each search engine at the same time (i.e., seven windows). We used a variety of techniques to reveal how each engine works and where it stumbles: We misspelled, hyphenated, and used multiple words; we asked questions (Why is it called Doppler radar?) and used words with multiple meanings ("vet"). We evaluated each engine by analyzing the quality of the top ten results for each search, and clicked through to see just how relevant the pages actually were. We checked for any links that were outdated, dead, or duplicated. We also considered any additional material on search return pages, such as paid-for listings (so-called sponsored links). Our findings? Read on.

AllTheWeb.com

Quality of results: Most relevant listings turn up in the top ten, but usually aren't the first or second item (No. 1 is always a sponsored link). Fewer duplicate and dead links than other engines. The site tries to anticipate searchers' needs in two ways: First it lists what it thinks are relevant categories (or Fast Topics) to match your query, but it weeds out unrelated pages only about half the time. The second feature, Narrow Your Search, lets a query for Yorkshire terriers, for instance, become more targeted by letting users choose "Yorkshire Terriers for Sale" or "Yorkshire Terriers UK" before getting results.

Organization: doesn't automatically divide results into categories (unlike Yahoo). Results come in more formats than from any other search engine: Web pages, news, pictures, videos, MP3s, or FTP files. Automatically rewrites queries to perform searches only with keywords ("history of the zoom lens" became "zoom lens"), which usually boosts results. Runs three sponsored sites before its own returns.

Advanced search: Six options, most of which are too technical to be of use (filtering by IP address or document size?).

Notable extras: Offensive Content Reduction Filter.

Overall grade: B

Google.com

Quality of results: Returns largest number of relevant results right up top. Returns too many duplicate links. Suggests new searches if it thinks query is misspelled (unlike MSN, which automatically corrects errors).

Organization: Most-relevant sites are first. Returns appear very unorganized because only very general searches "senators") yield matching categories ("Regional>North America>…>Government…>Senate"), and it doesn't give site descriptions for all results, making the best link harder to see. Relevant ads are marked as sponsored links on the right-hand side, and are equipped with an interest bar, which tells you approximately how many other people thought the link was relevant enough to click through.

Advanced search: Can narrow search by date, domain (e.g., fsb.com), file format (e.g., MP3), or language. Hitting "similar page" on a relevant page yields better results: A query of "technology terminology" resulted in only one computer-term dictionary, but hitting "similar page" yielded eight.

Notable extras: Can translate from English to five other languages; can search the Usenet public bulletin board system.

Overall grade: B+

MSN.com

Quality of results: Almost always lists relevant results on first attempt, though best results can be hard to find. Many duplicate results (though not as many as Google) make the list seem cluttered. If a query is somehow related to an MSN partner (a search for "CD players" suggests MSN's eShop first), results can be too numerous. Many dead links, but most do redirect to the site's home page.

Organization: Actual search engine results are buried beneath whatever links MSN deems relevant ("top ten most popular sites," "featured sites," "sponsor sites," and "news clips"). At bottom, "shopping results" links directly to a site to buy, if pertinent.

Advanced search "Stemming" feature lets you broaden results to include both word roots and word derivations, returning "hacking" and "hackers" when you search for "hacker." No place to add in extra search terms. Doesn't support quotation marks to specify an exact phrase inside the query.

Notable extras: Automatically corrects spelling.

Overall grade: B-

Overture.com

Quality of results: Results are highly relevant, but only if you're looking to buy a specific product or brand. Search a topic for which Overture doesn't have a sponsor and results can be as good as Google's.

Organization: Very clean; fewer results than any other search engine ("Ford Mustang" pulled up 79 responses; on Google, 372,000). Category links unrelated to queries: Click on "Computing" and get 40 links to 40 sites of paid advertisers (Westwood College's computer learning program came before Dell).

Advanced search: None.

Notable extras: Notes how much the advertiser paid to be listed.

Overall grade: C

Teoma.com

Quality of results: Frequently returns dead or irrelevant links. (This -- along with the site's other quirks -- will presumably be addressed soon when Teoma starts to use Google's listings, reflecting a deal the two recently signed.)

Organization: Divides results into three "R's:" Results (main list of uncategorized pages), Refine (related categories), and Resources (links to "experts"), the latter two being frequently wrong (a search for a technology-term dictionary pulled up a French-Dutch linguist).

Advanced search: Can search for an "exact phrase."

Notable extras: Specific "search tips" help narrow searches.

Overall grade: C-

WiseNut.com

Quality of results: Results for multiworded queries are unrelated -- as if it searched each word separately. Pulls up many outdated links. Supports only up to seven words per query.

Organization: Summaries don't accompany links, so users can't determine relevance quickly. The category list, or "WiseGuide," differentiates between terms that could obviously fall into the same category (e.g., "Vietnam vets" and "Vietnam veterans").

Advanced search: "WiseSearch" lets you expand the search by providing space to enter in up to seven extra search terms.

Notable extras: "Sneak-a-Peek" lets you view the site in an in-page window if you use Internet Explorer (very cool). But service failed, complaining of overload.

Overall grade: D-

Yahoo.com

Quality of results: Always dead-on, Yahoo divides its results into relevant categories and gives quick, pertinent descriptions. Always offers "related searches" for the overall query and "similar results" for individual listings to accurately refine a search on the fly. If it doesn't have what you need in its own directory, it helpfully uses Google's Web page listings.

Organization :Results are broken down into neat rows of related categories, making relevant links easy to locate. Website listings come with a precise, one-sentence summary. Search terms are highlighted in results, like Google. Ads are small and usually related.

Advanced search: Not needed, because Yahoo offers up all necessary search methods on the first pass. Otherwise, the "advanced search" is weak.

Notable extras: Click on "Research Documents" to pull up current news stories relating to the topic.

Overall grade: A

Do you agree with our assessments? Share your opinions by writing to editor@fsb.com.

PC World

Thursday, 12 September, 2002

China hijacks Google's domain name

"This is not what Tianwang Search hoped to see"

Sumner Lemon, Taipei
Try to access Google's search engine from inside China and there's a good chance you'll instead be sent to Tianwang Search, a search engine operated by China's prestigious Peking University.

Internet users looking to reach Google from inside China are being rerouted to Tianwang, and several other sites like it, after Internet service providers in China hijacked the domain name for the Mountain View, California, Internet search company.

The frequency with which Chinese users have been rerouted to other sites depends on the Internet service provider (ISP) and the location where the user is accessing the Internet, indicating that traffic to Google is not being rerouted at a national level, according to Duncan Clark, managing director at telecommunication market research company BDA China.

Domain names and URLs (Uniform Resource Locators) are matched to IP addresses using Domain Name System (DNS) software. When an Internet user types www.google.com, or any other URL, into a browser, a query is sent to the ISP's name server which returns an IP address for the site. ISPs in Beijing and Shanghai have apparently altered those addresses, redirecting traffic to Chinese search sites, Clark says.

"It's not possible for someone else to do this," he says.

The Chinese government has sought to block access to undesirable Web sites using IP (Internet Protocol) filters since commercial Internet access first became available here in 1995. Search engines Google and Altavista are the two latest Web sites to find themselves blocked in China. But this is the first time censors have hijacked a domain name and rerouted traffic to another Web site, Clark says.

Not everyone in China is happy that Internet traffic meant for Google has been rerouted elsewhere. "This is not what Tianwang Search hoped to see," the search engine said in a message posted on its Web site.

China frequently clamps down on foreign media in the run-up to politically sensitive dates and events. With Chinese President Jiang Zemin expected to hand power to a successor at the upcoming Communist Party congress, Internet censors may be trying to tighten control over information available on the Internet.

"It is in violation of the universal approach, changing the DNS system. When you type in a URL, from anywhere in the world, you expect to get to that address," says Bruce Tonkin, chief technology officer at Melbourne IT and chair of the Names Council of the Domain Name Supporting Organisation at the Internet Corporation for Assigned Names and Numbers (ICANN).

For Chinese users and Google alike, there may be little available recourse, however. "China has not signed any agreement (not to tinker with the DNS system inside China). No government has. There is no legislation, no mechanism to stop them," Tonkin says.

(David Legard, in Cairns, Australia, contributed to this report.)

More from Sumner Lemon