Most existing reinforcement learning (RL)-based portfolio management models do not take into account the market conditions, which limits their performance in risk-return balancing. In this paper, we propose Deep-Trader, a deep RL method to optimize the investment policy. In particular, to tackle the risk-return balancing problem, our model embeds macro market conditions as an indicator to dynamically adjust the proportion between long and short funds, to lower the risk of market fluctuations, with the negative maximum drawdown as the reward function. Additionally, the model involves a unit to evaluate individual assets, which learns dynamic patterns from historical data with the price rising rate as the reward function. Both temporal and spatial de- pendencies between assets are captured hierarchically by a specific type of graph structure. Particularly, we find that the estimated causal structure best captures the interrelationships between assets, compared to industry classification and correlation. The two units are complementary and integrated to generate a suitable portfolio which fits the market trend well and strikes a balance between return and risk effectively. Experiments on three well-known stock indexes demonstrate the superiority of DeepTrader in terms of risk-gain criteria.

This repository holds the Python implementation of the method described in the paper published in AAAI 2021.

Zhicheng Wang, Biwei Huang, Shikui Tu*, Kun Zhang, and Lei Xu*, “DeepTrader: A Deep Reinforcement Learning Approach to Risk-Return Balanced Portfolio Management with Market Conditions Embedding,” in Proceedings of the 35th AAAI Conference on Artificial Intelligence, AAAI-21, 2021, Feb.02-09

1. Requirements
2. Data Preparing
3. Training
4. Acknowledgement

• Python 3.6 or higher.
• Pytorch == 1.3.1.
• Pandas >= 0.25.1
• Numpy >= 1.18.1
• TensorFlow >= 1.14.0 (For you can easyly use TensorBoard)
• ...

According to the data usage policies of WRDS and WIND, we have no right to provide you with a copy of the data except for industry_classification.npy file :). If you have access to the WRDS or WIND database, please obtain and process the corresponding data yourself based on our paper.

The following files are needed:

These files should be placed in the ./data/INDEX_NAME folder, e.g. ./data/DJIA/stocks_data.npy

As an example, after putting data source file to the data folder, you can simply run:

python run.py -c hyper.json

Some of the available arguments are:

This project would not have been finished without using the codes or files from the following open source projects:

• Environment.py is inspired by PGPortfolio
• README.md is inspired by HPSG-Neural-Parser

Please cite our work if you find our code/paper is useful to your work.

@article{Wang_2021, 
title={DeepTrader: A Deep Reinforcement Learning Approach for Risk-Return Balanced Portfolio Management with Market Conditions Embedding}, 
author={Wang, Zhicheng and Huang, Biwei and Tu, Shikui and Zhang, Kun and Xu, Lei}, 
journal={Proceedings of the AAAI Conference on Artificial Intelligence}, 
volume={35}, 
number={1}, 
year={2021}, 
month={May}, 
pages={643-650} 
}