patricksaw / fetchurls Goto Github PK
View Code? Open in Web Editor NEWThis project forked from adamdehaven/fetchurls
A bash script to spider a site, follow links, and fetch urls (with built-in filtering) into a generated text file.
Home Page: https://www.adamdehaven.com/blog/easily-crawl-a-website-and-fetch-all-urls-with-a-shell-script/
License: MIT License