Giter VIP home page Giter VIP logo

bcv_scraper's Introduction

BCV Exchange Scraper

Table of Contents

About

This is a simple web scraper built in Python with FastAPI. It scrapes data from the oficial website Banco Central de Venezuela and returns it in JSON format.

Getting Started

These instructions will get you a copy of the project up and running on your local machine for development and testing purposes. See deployment for notes on how to deploy the project on a live system.

Requirements

  • Python 3.12 or higher
  • FastAPI
  • Requests
  • BeautifulSoup4
  • Docker

Installation

  1. Clone the repository.
  2. Install the required packages using pip install -r requirements.txt.
  3. Run the application using python run.py.

Usage

  1. Send a GET request to http://localhost:8000/get-data.
  2. The response will be in JSON format.

Docker

To run the application in a Docker container, follow these steps:

  1. Build the Docker image using docker build -t bcv-scraper ..
  2. Run the Docker container using docker run -p 8000:8000 bcv-scraper.

Deployment

Keep in mind that every call to the endpoint executes a scraping to the website. You need to consider storage solutions to avoid problems with the institution and yourself.

If you have the means, I recommend crawlab. You can create a scrapy spider with the selectors in scraper.py, set the cron task and forget about it (Well, mostly).

Contributing

Contributions are welcome! Please feel free to submit a pull request.

I hope this helps! Let me know if you have any questions.

bcv_scraper's People

Watchers

 avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.