About the OSU Search
Introduction
The OSU search tool has been updated to provide a better long-term web search solution for Oregon State University.
The purpose of this document is to examine the evolution of OSU's web search, and offer more detailed information about OSU's current search capabilities.
Background
History
OSU has seen three search solutions through the course of its web history.
- Inktomi - 1998-2002
- Inktomi was OSU's first search engine. Inktomi's base technology was initially developed at Berkeley, and during the mid-to-late 90's became the driving force behind the Yahoo and HotBot search engines. Inktomi has recently made a resurgence as a subsidiary of Yahoo.
Inktomi history »
- Google - 2002-2004
- Google originated at Stanford university as project BackRub, named for its weighting of backlinks in its search algorithm. In a few years, it developed into the most popular search engine in the world. As a natural expansion to the search engine, Google developed standalone search appliances aimed towards large organizations with a substantial web presence. Google Search Appliances provide a solution-in-a-box for searching large intranets and offering more specific content filtering than is possible with google.com's web interface. One of these appliances powered the OSU search for nearly 2 years.
Google history »
- Nutch - 2004-Present
- In August of 2004, at the end of the Google contract, Central Web Services evaluated Nutch as a replacement search service. Installed on OSU hardware, running software built, configured, and maintained through ardent cooperation between CWS and Nutch programmers, Nutch now powers the search.oregonstate.edu service.
Nutch organization »
Why Change?
The migration to Nutch was initiated to improve flexibility and extensibility. Having full control over the code means we can now adjust the search tool to best meet OSU's growing needs. Both previous search engines were limited to searching a fixed number of documents, which meant being selective about what information was included in their results.
Being able to run the application on commodity hardware also means improved reliability for searching. In the event of a hardware failure, OSU's search can make use of backup device, rather than waiting for a vendor to ship replacement parts.
Flexible and Extensible
Through multiple steps in the evolution of OSU's search technology, Central Web Services has labored to make these changes transparent to the OSU community. The OSU web has been able to handle drastic change due to the abstraction of search scripts. This means that the search boxes you see on many OSU pages do not depend on what technology our search uses. As a result, OSU may easily incorporate any future improvements in web search technology.
New to OSU is the ability to customize search features and capabilities. The implementation of an open source solution such as Nutch improves the extensibility of OSU's search technology. We now have the ability to implement custom features to meet OSU's evolving needs. In addition, any improvements we make are shared with other organizations that have deployed Nutch.
Nutch
About Nutch
Nutch is an emerging open source project aimed at creating a non-biased, optimized search solution.
Nutch Links
Features
- Flexible
- Nutch is open and customizable.
- Unbiased
- Nutch's searches are not weighted on anything but the relevance of information.
- Cost-Effective
- Nutch requires no licensing or vendor maintenance fees. OSU provides hardware, maintenance and programmer time.
- Cooperative
- As more organizations adopt Nutch, a larger pool of developers will be working to improve it and contribute those improvements.
- Software
- Nutch is a web application, so as our search database grows, so can the hardware that powers it. And it's simple to provide redundancy in case of a hardware failure.
Support
Contact Central Web Services with questions, comments or concerns.
Credits
- Nutch Team
- Special thanks to the Nutch Programmers for their dedication and excellent work.
- OSU Project Management
- Herb Vloedman -- Associate Director, Central Web Services, Oregon State University
- OSU Server Administration, Testing, Development, Maintenance
- Robert Hopson -- Web Systems Engineer, Central Web Services, Oregon State University
- Additional Programming, Documentation, User Interface
- Central Web Services, Oregon State University
