hasselfrau / indeed_web_scraper Goto Github PK

CLI. Update README.MD
(add information about the ability input arguments via CLI)

Implement Grequests

2020-07-26 21:48:30,198 - scraper_log - INFO - 15 new job positions has been added to the database
2020-07-26 21:48:30,198 - scraper_log - INFO - Current number of positions: 12

DB.
Is that necessary to store it in the job postings table as well?

Add logging.
(https://docs.python.org/3/library/logging.html)

In this exercise you will enrich the data you scraped by accessing a public API of your choice,
querying it, and storing the response in the database.

1. Find a free public API, that have some relevance to your data. You can start here for a
   list of some public APIs.
2. Add to your scraper the ability to query this API. Store the responses in your database.
   There should be some link between your scraped data, and the new data you retrieved
   from the API.
3. Use this checkpoint to complete unfinished features and fix bugs. The next two
   checkpoints will not involve writing additional code.
   Good luck!