The scraper is used to scrape the study programs and related courses from the Faculty of Computer Science and Engineering at the Ss. Cyril and Methodius University in Skopje. which can be found at the following url.

The scraper will save the scraped data in three different files:

• study_programs.csv: contains the details of the study programs
• curriculum.csv: contains the details of the study programs and related courses
• courses.csv: contains the details of the courses

• Python 3.9 or later

Before running the scraper, make sure to set the following environment variables:

• OUTPUT_DIRECTORY_PATH: the path to the directory where the output files will be saved
• MAX_WORKERS: the maximum number of workers that will be used to scrape the website
• EXECUTOR_TYPE: the type of executor that will be used to scrape the website. The possible values are "THREAD" and "PROCESS"

1. Clone the repository

   git clone <repository_url>

2. Install the required packages

   pip install -r requirements.txt

3. Run the scraper

   python main.py

Make sure to replace <repository_url> with the actual URL of the repository.