This script scrapes famous quotes from the website quotes.toscrape.com and saves them to a JSON file for offline usage.
- Scrapes quotes and their authors from the first three pages of the website.
- Stores the scraped data in a structured JSON format.
- Uses politeness policy by waiting 2 seconds between requests to not overload the server.
- Python: The script is written in Python.
- Requests: A simple HTTP library for Python used to make requests to the website.
- BeautifulSoup: A library for pulling data out of HTML and XML files, used for web scraping tasks.
- JSON: Used for storing the scraped data in a structured format for later usage.
-
Make sure you have Python installed on your machine.
-
Install the required Python libraries:
pip install requests beautifulsoup4
-
Run the script:
python main.py
-
After the script has run, you'll find a
quotes.json
file in the same directory. This file contains the scraped quotes.
- This script is intended for educational purposes. Before scraping any website, always refer to the website's
robots.txt
file and terms of service to ensure you're allowed to scrape it. - Avoid making rapid, consecutive requests to websites. Always implement delays and respect the website's rate limits.
This project is open-sourced under the MIT License.