Giter VIP home page Giter VIP logo

howtoquant-backend's Introduction

How To Quant (backend)

This is a proof of concept hobby project aimed at exploration of finance topics and their technical implementation. While the functionality and architecture bares some resemblance to real-world systems and some of the data reflects true facts, no part of this project is intended as financial advice and is not to be considered as such.

The deployed API can be accessed on root https://howtoquant.pythonanywhere.com/ <appname> with available endpoints documented here.

The front-end is deployed to howtoquant.com.

The front-end repository is available here.

Table of contents

  1. Project structure
  2. Data model
  3. Running locally
  4. Past sprints
  5. Roadmap

Project structure

The project is built with Django framework, providing a set of JSON API endpoints for retrieving and inserting data into MySQL database. The functionality is divided into apps (modules) as follows:

  • Classifiers app - stores standartised re-usable data units representing sets of objects shared by other apps. The datasets are provided in SQL seed script for this app and canot be changed by the user.
  • Staticdata app - stores information that is expected to rarely change and have infrequent additions once a mature dataset is established. By convention "static data" in finance is used in reference to instrument details/contract setup. This app also includes organizations that can be referenced by other entities and apps, such as funds, prime brokers, issuers and 3rd parties. This app provides functionality for requesting instrument data (equity only) from 3rd party API (Yahoo Finanace) and manual instrument setup.
  • Marketdata app - stores data that is produced as a byproduct of market activity/transactions, i.e. prices, fxrates, interest rates, and other numerical data (colloquially referred to as "analytics"). This app provides functionality for price data download from 3rd party API (polygon.io) and saving the data for permanent storage.
  • Accounting app - stores data pertaining to organization portfolio and trading activity, including book, strategy and account setup, trade log, cash and non-cash asset ladders.

Data model

Data is organised in Django ORM as per ER diagrams below. Each image features only the models contained within the app, relationships between apps are inferred by field names matching table names in another app.

Classifiers app

Classifiers ER diagram

Staticdata app

Static data ER diagram

Marketdata app

Market data ER diagram

Accounting app

Accounting ER diagram

Running locally

Requirements and dependencies

Hardware requirements:

  • Network card and internet connection.
  • Processor and RAM requirements depend on the browser/API client of your choice, and whether MySQL database is hosted locally or remotelly. Requirements for MySQL can be found here.
  • Storage space of at least 360 MB (with local database), although 500 MB is recommended.

Software dependencies:

  • Git.
  • Python and pip - the project was developed using Python 3.11, earlier versions may work, but have not been tested.
  • MySQL database either locally or remotely hosted - aiven hosting service was used for development.
  • Browser or API client of your choice.

Third party dependencies:

  • API key to use polygon.io data

Setting up the project

Follow these steps to set up the project on a local machine:

  1. Create a directory where the project will be contained and git clone this repository.
  2. In console navigate to the root directory of the project (where manage.py file is located). Alternativelly, you may want to create a virtual environment.
  3. Run pip install -r requirements.txt to install all pythonic dependencies (this will take several minutes to complete).
  4. In project folder howtoquant add ./.envs/.env_dev file with environment variables like this:
ENVIRONMENT=development

DB_ENGINE=django.db.backends.mysql
DB_NAME=YOUR_DATABASE_NAME_HERE
DB_USER=YOUR_DATABASE_USERNAME_HERE
DB_PASSWORD=YOUR_DATABASE_PASSWORD_HERE
DB_HOST=YOUR_DATABASE_HOST_HERE
DB_PORT=14300

POLYGON_API_KEY=YOUR_POLYGON_API_KEY_HERE

SECRET_KEY=ANY_RANDOM_SEQUENCE_OF_CHARACTERS
  1. From the command line run command python manage.py migrate - this will create tables in the database.

  2. Run in sql script located in project folder howtoquant/db/reseed_db.sql. The exact procedure of executing this script will vary depending on your databse hosting solution and the database manager of your choice. If you are hosting the database locally and accessing it via MySQL Workbench, use File-->Run SQL Scripts command, choose your db schema name and utf8 as the default character set.

  3. You should be ready to start the server

Starting the server and accessing endpoints

To start the server execute this command python manage.py runserver. If successful, you should see a line Starting development server at http://127.0.0.1:8000/ amongst the output.

The website will now be available on the localhost. Input the following url in the browser/APi client to access documentation listing all endpoint: http://127.0.0.1:8000/api. Other endpoints will be accessible on http://127.0.0.1:8000/<appname>/<endpoint>

Monkey fixes

If you encounter below error when starting the server:

File "C:\Python311\Lib\site-packages\corsheaders\__init__.py", line 1, in <module>
    from .checks import check_settings  # noqa: F401
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Python311\Lib\site-packages\corsheaders\checks.py", line 2, in <module>
from collections import Sequence
ImportError: cannot import name 'Sequence' from 'collections' (C:\Python311\Lib\collections\__init__.py)

you will need to manually edit the file C:\Python311\Lib\site-packages\corsheaders\checks.py

Change the following line:

from collections import Sequence

to

from collections.abc import Sequence

If you encounter below error when starting the server:

File "C:\Python311\Lib\site-packages\corsheaders\middleware.py", line 10, in <module>
from .signals import check_request_enabled
File "C:\Python311\Lib\site-packages\corsheaders\signals.py", line 6, in <module>
check_request_enabled = Signal(providing_args=['request'])
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: Signal.__init__() got an unexpected keyword argument 'providing_args'

you will need to manually edit the file C:\Python311\Lib\site-packages\corsheaders\signals.pyy

Change the following line:

check_request_enabled = Signal(providing_args=['request'])

to

check_request_enabled = Signal('request')

Past sprints

Code and release notes of each of the past releases can be found in the respective branch of this repo, as per table below.

Sprint No. Relase notes
#1 v.0.1.0-alpha
#2 v.0.2.0

Roadmap

The trajectory of this project is bound to change in accordance to time constraints, priorities and interests of the author. A rough plan for near future sprints is as follows:

Sprint No. Tasks
#3
  • Add POST and GET endpoints for trade data
  • Establish new trade data flow through accounting model tables (cumulative qty only)
  • Add roll-forward functionality
#4
  • Add portfolio data seed
  • Add MV calculation to trade processing
  • Add PnL calculation to trade processing
#5
  • Add message queue for process control and coordination
  • Implement unused market data download truncation
  • Add test suite in pytest
#6
  • Port least efficient ladder recalculation functionality to Rust
  • Add user management
#7
  • Add VaR functionality
...
    ...

License

The source code and all artifacts are available under CC BY-NC-SA 4.0 DEED terms.

howtoquant-backend's People

Contributors

viksil avatar

Watchers

 avatar

howtoquant-backend's Issues

Use generic DF to DB function for marketdata download

The marketdata app uses save_download_data function from marketdata.utils_download module to save downloaded data into the database, download_data table.
A more generic function save_df_to_db is now available in howtoquant.utils module. The old function should be replaced where it is called. A new option to save into download_data table will need to be added into the generic function.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.