omar-elmaria / python_scrapy_airflow_pipeline Goto Github PK
View Code? Open in Web Editor NEWThis repo contains a full-fledged Python-based script that scrapes a JavaScript-rendered website, cleans the data, and pushes the results to a cloud-based database. The workflow is orchestrated on Airflow to run automatically