Where Research Will Happen: The Web

The Web needs no introduction. However, there are a few important facts about today's Web that every researcher must know. Notice the emphasis on the word today's. The warnings outlined below are for the Web we use now. Because of continual improvement, in five years these warnings may sound quaint.

The knowledge and information contained in websites is not categorized or organized in any significant fashion. This Person in maze comes as no surprise to anyone who has used search engines. How many times has your favorite search engine returned hits, and after looking at the first few pages you wondered, Why did it give me these sites? This is a serious defect of the Web and the single most important reason why doing serious research on the Web is still nearly impossible. For example, try finding useful material on a fairly complex topic like Hitler's treatment of the mentally retarded as shown in this , or the influence of eastern religions on Thomas Mann. You will expend significant effort and not find much of value. A few useful sites are often found, enticing the searcher to think more can be located.

Much of the Web cannot be "seen" using standard search engines like Google or Teoma. Even the biggest search Iceburg engines search less than 60% of all webpages. The remaining 40% lie hidden behind security barriers, are too deep in a website's hierarchy to be indexed, or require a password. There is even a larger "invisible Web," according to a study found on the Search Engine Watch site, that can be mined only by using individual database portals. In fact, the study determines that only 1/500th of the information on the Web is accessible through standard search engines! The rest lies buried in databases. The Making of America (MOA) website is an example of what lies buried in the "invisible Web." Through the MOA portal, a researcher can access the full text of 8,500 books and 50,000 journal articles, yet not a single MOA source will be found using a standard search engine!

There is a public, or "free," Web and a hidden, or "fee," Web with virtually no overlap. This is closely related to the "invisible Web" discussed in the previous point. The public Web contains the sites retrieved by standard search engines. The hidden Web contains huge databases of journal articles and books that are password protected. It's on the hidden side where you will find all the high-quality sources needed for a research assignment; but not a single one will be found using Google, Yahoo, or Teoma. These sites are created by companies that add value to information by organizing, cataloging, and packaging it. Access to these sites is then sold to organizations, such as libraries. When you think about it, it makes sense that there would be a hidden Web. Billions of dollars are spent annually to produce and sell books and journals. Publishers who let that material flow freely on the Web would quickly go bankrupt. Typically, an information provider licenses campus-wide database access to a library; then all computers on that campus can access that database. For example, all the Research Databases in the Hekman Digital Library that have the symbol reside on the hidden Web.

In praise of the public Web. Obviously, the public Web is useful—why else would 75% of frequent users say they find what they are looking for most of the time or all of the time? There are times during the research process when the resources found on the public Web will be invaluable. If you need specific pieces of information, the public Web is the tool to use. If you need a quote, the population of Columbia, the molecular structure of benzene, or a biography of Abraham Lincoln, the public Web is the place to go. There are also sites on the public Web that rival those found in the hidden Web. Many of these websites have been incorporated into the library's website. For example, the Hekman Digital Library at Calvin College provides links to websites that contain the full-text of thousands of books, U.S. government documents, and Web directories found in the public Web.

The bottom line: The Web does contain a wealth of information, but much of it can't be accessed using standard search engine. To access that wealth of information, you need to enter the Web through the library's website. For example, students at Calvin can enter through the Hekman Digital Library.

If you encounter technical errors, contact computing@calvin.edu.