krishna-priyaaa / web-crawler Goto Github PK

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

STEPS TO RUN TASK 1:

1. Intsall requests using the following command: pip install requests

2. Install BeautifulSoup using the following command: pip install BeautifuSoup4

3. Navigate to the directory in which the files are there and run the following command python hw1_task_1.py

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

STEPS TO RUN TASK 2:

1. Navigate to the directory in which the files are there and run the following command python hw1_task_2.py

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

FILES PRESENT IN HERE:

1. hw1_task_1.py - for TASK 1
2. hw1_task_2.py - for TASK 2
3. Task 1-E - list for UrLs(result of task1) - Maximum depth reached is 3
4. Task 2 - list for UrLs(result of task2) - maximum depth reached is 6
5. task2Brief - Explaination for TASK 2

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

REFERENCES:

1. https://en.wikipedia.org/wiki/Tropical_cyclone
2. http://www.pythonforbeginners.com/python-on-the-web/beautifulsoup-4-python/
3. https://www.crummy.com/software/BeautifulSoup/bs4/doc/
4. https://docs.python.org/2/library/re.html
5. https://pypi.python.org/pypi/stemming/1.0
6. https://en.wikipedia.org/wiki/Breadth-first_search
7. https://en.wikipedia.org/wiki/Depth-first_search
8. https://www.hackerearth.com/practice/algorithms/graphs/breadth-first-search/tutorial/