notadamking / stock-trading-visualization Goto Github PK

Nice project mate,

After making some tests i'm having a problem, I think you are updating only the first and the 3th values on line

I'm trying to understand what's happening here, i'm looking forward for you answer, thanks

Hello bro,
After doing some changes I could make the bot beat the market by a small margin:

There is a lot of work to do but the main changes are:

• adding Ray so we can change the algorithms and the hyperparms
• Adding technical indicators to the csv
• changing the observable space (we do not need the past 5 days of data since with the indicators much more information is already been passed)
• changing the reward function

To train the agent in this images I used PPO with learning rate 5e-5 trained over 1.2 million timesteps with ourly data from 2019-04-19 16:00:00+00:00 to 2019-04-28 05:00:00+00:00 and then to evaluate I used data from 2019-04-28 06:00:00+00:00 to 2019-05-06 20:00:00+00:00.

You can see the whole code in this fork

Nice project mate,
Thanks

Wanted to give a heads up to anyone trying to run this. You will need specifically tensorflow==1.14.0 because stablebaselines only supports up until this version (aka breaking module name changes).

For anyone trying to run on Colab, you'll need to run:

pip uninstall tensorflow
pip install tensorflow==1.14.0
pip install stable-baselines[mpi]==2.8.0

hill-a/stable-baselines#520

If you encounter the following error:
from mpi4py import MPI
ImportError: DLL load failed: The specified module could not be found.

I could get rid of the issue by running the following command:
conda install mpi4py

The following NEW packages will be INSTALLED:

mpi4py intel/win-64::mpi4py-3.0.0-py37_6

Proceed ([y]/n)?

Issue is resolved for me thereafter.

I've opened an issue in Stock-Trading-Enviroment (link) that I belive also affects this repo [same class, more specifically, same method].

However, I'm opening this new issue, because I think the changes in StockTradingEnv class (present only in this repo) created a new problem.

  def _next_observation(self):
        frame = np.zeros((5, LOOKBACK_WINDOW_SIZE + 1))

        # Get the stock data points for the last 5 days and scale to between 0-1
        np.put(frame, [0, 4], [
            self.df.loc[self.current_step: self.current_step +
                        LOOKBACK_WINDOW_SIZE, 'Open'].values / MAX_SHARE_PRICE,
            self.df.loc[self.current_step: self.current_step +
                        LOOKBACK_WINDOW_SIZE, 'High'].values / MAX_SHARE_PRICE,
            self.df.loc[self.current_step: self.current_step +
                        LOOKBACK_WINDOW_SIZE, 'Low'].values / MAX_SHARE_PRICE,
            self.df.loc[self.current_step: self.current_step +
                        LOOKBACK_WINDOW_SIZE, 'Close'].values / MAX_SHARE_PRICE,
            self.df.loc[self.current_step: self.current_step +
                        LOOKBACK_WINDOW_SIZE, 'Volume'].values / MAX_NUM_SHARES,
        ])

I'm not sure what was the reason to use np.zeros and np.put instead of directly using np.array like in Stock-Trading-Enviroment repo, but the way it is being used here, doesn't seem to achieve its purpose.

From what I gathered so far, and I might be wrong here, the purpose of the frame is to aggregate all past candle data (OHLCV) up to a 41 days, assuming LOOKBACK_WINDOW_SIZE is kept at 40. Ignoring the issue I already mentioned in the top of the post, the use of np.put here is causing the frame to only receive two data points, while the rest is kept at zero.

Return of the frame after the np.put:

[[0.002726 0.       0.       0.       0.0033   0.       0.       0.        0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.      ]
 [0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.      ]
 [0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.      ]
 [0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.      ]
 [0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.       0.      ]]

As you can see, the only values that were changed were the ones passed in the parameters as indices:
np.put(frame, [0, 4], [
All the rest is kept at zero.

I am having this issue...
Updated to stable_baseline3 and mplfinance and changed the imports according to the new package versions.

Stock-Trading-Visualization/env/StockTradingEnv.py:97: RuntimeWarning: invalid value encountered in double_scalars
self.cost_basis = (

| time/ | |
| fps | 400 |
| iterations | 1 |
| time_elapsed | 5 |
| total_timesteps | 2048 |