jstdoit / commoncrawldocumentdownload Goto Github PK
View Code? Open in Web Editor NEWThis project forked from centic9/commoncrawldocumentdownload
A small tool which uses the CommonCrawl URL Index to download documents with certain file types or mime-types. This is used for mass-testing of frameworks like Apache POI and Apache Tika
License: BSD 2-Clause "Simplified" License