Search for software
Crawl a website with this free ActiveX spidering component. Advanced features include caching, "avoid" patterns, and robots.txt compliance.