Description
Web targets can be crawled systematically to discover pages, links, and application surface area.
It is useful for authorized security testing, inventory, and web application analysis. Crawling can stress servers and collect sensitive URLs, so use it only on systems you are allowed to test and respect rate limits.