Search engines are usually thought of in the framework of the WWW but this type of service can be provided as a means to access just about any collection of data. The Open Archives Initiative is an organization dedicated to making digital libraries interoperable and has developed a standard way to access the data stored in a single archive. If a search engine could be tailored to work with this standard data interface (known as the OAI Data Provider interface) then that search engine could be used to search the contents of any of a growing number of dynamic collections on the web (without such a service, the digital libraries in question are not searched by standard web search engines since their content is dynamically-generated from databases).
For an example of such a service, see ODU's ARC.
The aim of this project is to go beyond ARC and develop a service with two additional capabilities:
It is not well-known which of the freely-available search engines are most appropriate for this task, so the first part will involve doing a survey of what is available. Then, one or more selections can be tailored to preprocess a stream of data and return results to queries through the standard OAI-like interface (a specification for this will be provided).