RIT Grace Scraper

This utility scrapes through the contents of all user folders on Grace (the RIT web server) looking for files. As opposed to attempting to visit the web site over the web and spidering the pages, navigating the directories directly proved to be much more useful and powerful.

The application need to be run on Grace itself, and could probably be quickly configured to work on other web servers too.

The first version of this application is also provided – it scraped the RIT web server’s public traffic logs looking for images and inserted them into a database.


Posted: April 3rd, 2004

Subscribe for email updates

Comments are closed.
Comments are automatically turned off two weeks after the original post. If you have a question concerning the content of this post, please feel free to contact me.

Secrets of the JavaScript Ninja

Secrets of the JS Ninja

Secret techniques of top JavaScript programmers. Published by Manning.

John Resig Twitter Updates

@jeresig / Mastodon

Infrequent, short, updates and links.