Software: Python3.6
Required Packages:
BeautifulSoup4:
If you’re using a recent version of Debian or Ubuntu Linux, you can install Beautiful Soup with the system package manager:
$ apt-get install python-bs4
Beautiful Soup 4 is published through PyPi, so if you can’t install it with the system packager, you can install it with easy_install or pip. The package name is beautifulsoup4.
$ easy_install beautifulsoup4
$ pip install beautifulsoup4
Execution Instructions:
Max number of pages N/limit can be changed at the bottom of Spider.py, under
Spider(url, limit).
Run Spider.py in Terminal:
$ python3 Spider.py