Url-Spider.com

Web Scraping Software
Advanced data scraping services at an affordable price!
www.infosquire.com/

Crawlers
Everything to do with Crawlers items.
yahoo.com/

Robots.txt Generator
Create and maintain your robots.txt files in seconds!
www.basisoft.com/

Google Mini
Put the power of Google to work searching your servers - Learn more
google.com/mini/

Online Website Crawler
SiteQuery - Scan your website for search engine account codes - Free
www.rapnikasconsulting.com/seotools/