This web crawler(crawl.py) has been written in Python using "Python 3" in the development environment. The library used for Crawling the pages is "BeautifulSoup".
- For executing the crawler without the keyword, execute it as follows:
python3 <file-name> <url>
E.g: python3 crawl.py http://en.wikipedia.org/wiki/GitHub
- For executing the crawler with the keyword, execute it as follows:
python 3 <file-name> <url> <keyword>
python3 crawl.py http://en.wikipedia.org/wiki/GitHub developer