A vast amount of information on web sites doesn't return on conventional searches even though the main operators index up to 8 billion documents. This content is known as Deep Web and it's either buried deep within web sites or it's available only by subscription.

The search engine operators know this and, from the leaders Yahoo and Google to the small operators like Groxis and Mamma.com, they're attempting to make it easier to find. This isn't a new idea - Yahoo tried it out some years ago.

It's the way the premium content providers including the written press, research databases and traditional aggregators are reacting that's changing , suggest analysts and search engine executives. "I see this as part of an emerging need to go beyond what is available on the Web today," said Allen Weiner, a research director at Gartner Inc. "Search has plateaued." A couple of weeks ago, search leader Yahoo launched a beta service for subscription-based content from publisher sites and databases, including the Wall Street Journal Online, Consumer Reports, the IEEE (Institute of Electrical and Electronics Engineers Inc.) and Forrester Research Inc. Tuesday, Groxis launches a free pilot search site called Grokker Research which will offer content from Ebsco Information Services database and the IEEE. The trial will run for about six months and then charges for access to the premium content will apply explained R.J. Pittman, president and CEO of the company.

Last week there was talk about Google being in negotiations with content providers about extending its subscription-based search service - they didn't comment on the report despite a request. Mamma.com has ventured into "deep Web," with a health search site that returns detailed results with on-the-fly crawling of selected health sites. Called Mamma.com Health Search, it is the first of up to four specialized search sites the company has plans for, said Guy Faure, CEO of Canadian Mamma.com. Even six months ago, premium content providers had little interest in co-operating with Google or Yahoo. "I see this as a major, major deal," Weiner said. "To me the other shoe has dropped. Search companies knew that to increase quality of their results they needed access to this content, and, on the other hand, the premium content companies realized that the Yahoos and Googles of the world were not going away and that they would be an important channel for them." Weiner anticipates search operators going beyond acting as conduits for current subscribers to reach premium content. Yahoo Search Subscriptions, for example, requires users to be subscribers to publications to click through. Eventually, search sites will offer one-off paid access to premium content - for the content providers, the incentive would be the potential to gain subscribers. While Groxis has been working with publishers it has identified a change in the way people see search engines.

Groxis is targeting librarians and corporate researchers, and Pittman said that both publishers and researchers are growing more interested in working with search engine operators. "The world has evolved, and it will continue to evolve around search and research," he said. "People are finally understanding the value of going beyond the top 10 search results."

Reuben Dunn
Jump2Top.Com
Search Engine Optimization Company