Project Suggestion: NSDL CS Crawling and Analysis

• Number of people: 2-4
• Goal: Pilot effort related to NSDL, helping proposal and CSTC
• Contact information: Professor Lillian (Boots) Cassel at Villanova (cassel@acm.org), Professor Lee Giles at Penn State (Giles@ist.psu.edu), Professor John Impagliazzo at Hofstra (cscjzi@Mail1.Hofstra.edu) and the instructor, along with local faculty Professors Lee (janlee@cs.vt.edu) and Perez (perez@cs.vt.edu)
• Required background: interest in education and computing
• Description: NSF proposes to build a digital library to help undergraduate students, NSDL (National Science (and Mathematics, Engineering and Technology Education) Digital Library - see references). There is need for a group to coordinate this effort for the field of computing and information technology. This project aims to help develop such a collections-oriented proposal, by undertaking pilot studies of what content is already on WWW.

  First, students should explore technology for Web crawlers and spiders and find the best available. Professor Giles (see above) has some tools and ideas on this, as does the instructor. See reference 7, for example.

  Second, students will run crawlers/spiders in a series of runs on the Web sites for the top 100 CS departments in USA. Professors Cassel and Lee can help find such a list. The idea is to create a local copy or at least an index of those sites so we can study them in depth.

  Third, students will prepare tables and figures from that analysis. These will answer a series of questions that evolve during the semester and support preparing a proposal to NSF (see previous project in this list). For example, we will determine (see Prof. Giles for heuristics to begin to employ):

  • How many pedagogically useful applets are there in each of the various topical areas of computing?
  • How many pedagogically useful PowerPoint lectures are there in each of the various topical areas of computing?

  References:

  (1) NSF, "National Science, Mathematics, Engineering, and Technology Education Digital Library (NSDL) - Program Solicitation," National Science Foundation NSF 00-44, 2000. http://www.nsf.gov/cgi-bin/getpub?nsf0044. Note: This is superceded by the version mentioned below as reference 7.
  (2) NSF, "National Science, Mathematics, Engineerin6, and Technology Education Digital Library (NSDL) - Program Information". National Science Foundation, 2000. http://www.ehr.nsf.gov/EHR/DUE/programs/nsdl/
  (3) SMETE.ORG. Website for one of pilot projects related to the NSDL, pointing to many of the other projects, http://www.smete.org
  (4) D. Knox, S. Grissom, E. A. Fox, R. Heller, and D. Watkins, "CSTC: Computer Science Teaching Center", 2000. http://www.cstc.org
  (5) Lillian Cassel and Edward A. Fox, co-editors, ACM Journal of Educational Resources in Computing (JERIC, http://purl.org/net/JERIC/)
  (6) NSF, "National Science, Mathematics, Engineering, and Technology Education Digital Library (NSDL) Program Solicitation", National Science Foundation NSF 01-55, 2001, http://www.nsf.gov/cgi-bin/getpub?nsf0155
  (7) Allan Heydon and Marc Najork, "Mercator: A Scalable, Extensible Web Crawler", Compaq Systems Research Center, June 26, 1999, http://research.compaq.com/SRC/mercator/papers/www/paper.html

  
  

• Honors students involved during Fall 2000:
  • Kate McDevitt, kmcdevit@vt.edu] (continuing, working on independent study in Spring 2001)
  • Brian Hobbs, brhobbs1@vt.edu (graduated in Dec. 2000)