Alex Wright, In search of the deep web, Salon, March 9, 2004. The next generation of deep-web search engines will give the public what it needs and deserves but has not been getting: free online access to government documents. Excerpt: " 'The U.S. Government Printing Office has the mandate of making the documents of the democracy available to everyone for free,' says Tim Bray, CTO of Antarctica Systems. 'But the poor guys have no control over the upstream data flow that lands in their laps.' The result: a sprawling pastiche of databases, unevenly tagged, independently owned and operated, with none of it searchable in a single authoritative place. If deep Web search engines can penetrate the sprawling mass of government output, they will give the electorate a powerful lens into the public record. And in a world where we can Google our Match.com dates, why shouldn't we expect that kind of visibility into our government?"
Deep-web search engines are also opening access to research literature. "For example, when gene researchers identify a new DNA sequence, they usually submit the sequence to the National Institutes of Health's GenBank -- a public deep Web resource -- before submitting it to journals for publication." Wright also gives a mini-picture of the OA movement apart from any connection to deep-web search engines.
Finally, on a very different front, search engine relevance algorithms provide a quality filter that challenges some of the traditional functions of peer review. "And as more scholarship finds its way onto the Web, page-ranking algorithms are also providing an alternative quality rating system to the traditional scholarly peer review that journals have always employed....While page ranking won't replace the scholarly review process anytime soon, the expansion of public Web search engines will put downward pressure on the premium that publishers can command. 'I don't think [page ranking] is more reliable,' says [Peter] Lyman [professor of Information Management and Systems at Berkeley], 'but I do think it's perceived as legitimate. The cost of creating formally quality-controlled information may drive people to consider lower-cost alternatives.' " (Thanks to Al Magary.)
Posted by
Peter Suber at 3/09/2004 08:45:00 AM.
The open access movement:
Putting peer-reviewed scientific and scholarly literature
on the internet. Making it available free of charge and
free of most copyright and licensing restrictions.
Removing the barriers to serious research.
