by Joshua Brauer on February 12, 2006
Without realizing it was new I spent some time with Google's robots.txt tool. All and all a very nice new addition to the Google suite. I also got around to getting the sitemap module for Drupal so now it might show something in their cache that is newer than last July.
There is also a good comment that discusses why Google is not looking to support the crawl-delay directive that some other search engines support.