This is a Web Scraping Project Architecture. This will help to setup and start a web scraping project, using Python and DRF.
pip install -r requirements.txt
After editing code, go back at the root of the project and run black .
This will format the code.
Go at the root of the project.
Run cd ScrapingProject/
Run python manage.py migrate
Run python manage.py createsuperuser
Run python manage.py runserver
Address to use : http://localhost:8000
It's easy, there is just 3 folders.
config
/ Scraping
/ ScrapyScraper
config
is the folder where all the configuration for the django project are.
Scraping
is the folder where all the files to setup and adjust the web interface to manage data are.
ScrapyScraper
is the folder where all the files to setup the scraper (Programs that are collecting data on website) are.