llsourcell / make_money_with_tensorflow_2.0 Goto Github PK

This is the code for "Make Money with Tensorflow 2.0" by Siraj Raval

Jupyter Notebook 46.46% Dockerfile 1.09% Makefile 0.52% Python 31.07% CSS 0.83% HTML 20.04%

make_money_with_tensorflow_2.0's Introduction

Make_Money_with_Tensorflow_2.0

Overview

This is the code for this video on Youtube by Siraj Raval on Making Money with Tensorflow 2.0. In the video, i demonstrated an app called NeuralFund that uses deep learning to make investment decisions.

Pull requests

I encourage pull requests that make this code better

Dependencies

Tensorflow 2.0
flask
Tensorflow serving

Instructions

NeuralFund is a combination of 2 github repositories. This is a work in progress.

First, I used this tensorflow serving web app skeleton code as my base project. In that app, the author integrates TF Servng with Flask to create a structure that allows for a continous training pipeline. Download that code and run it locally.

Second, I used the flask boilerplate code from my last video for the user authentication + MySQL database integration it had implemented. Thats the code in the folder in this repository.

TODO: Step 1 - Merge the two repositories by starting with the simple TF serving demo. Copy and paste the user auth + SQL code from the boilerplate demo into the simple TF serving demo.

Step 2 - In the 'train.py' file in the simple TF serving demo, under main(): add this code snippet to pull real-time stock data from the web. It will do that dynamically as per the continous training pipeline.

Step 3 - Add this trading view widget anywhere on the front end for a nice stock visualization.

Step 4 - The model will be able to make time series predictions, but what if it could also predict which stock to buy? Have 3 seperate models train on 3 different stock prices simulatenously. When done training, have them perform inference to predict the next price. Use the prediction that offers the highest increase from the previous price.

Step 5 - Have 3 more models train on 3 news datasets via the google news API for each of the stocks. perform sentiment analysis using a pretrained model like BERT to do this. Pick the stock that has the highest sentiment and price prediction.

Step 6 - Figure out a way to implement Deep Reinforcement Learning in tensorflow serving, i haven't yet seen an example of this done on GitHub. I might just do this in my next video. Treat the market as a markov decision process, the agents actions are buy sell or hold.

Credits

toebit3hub, tensorflow team, cedrickchee, my parents, my Wizards, all humans who came before me, thank you i am but a temporary vessel of knowledge

make_money_with_tensorflow_2.0's People

Contributors

Stargazers

Watchers

Forkers

jshuadvd tomarraj008 mistertoby flamato abhishek-ag2000 asirem16 lionaruc lnsongxf iamvazu jbdatascience nkrth singh-ap hedgefair victor8733 alexpriftuli zwytop allenlzcoder muharremokutan farmisen gil2abir daytradecentral vinayvm althetinkerer pruthvishetty drroad decollibus10 iamsile abdelpakey hellochengzhezhixing l4ivi ajinkyapuar desh-woes damianphung stjordanis hjain4 cfirmo33 jasonltt terrywalker866 tienduyvo merlinvn johangenis dcallega ericohenrique athansopoulos benpbolton damilolaalao crosstuck leobistek puzzlebird zalegzl microstepf momor666 asr-engineering-consulting atifsari banyip robmarkcole unosonu hamissionimpossible gwthompson mangotalk snowsky paolominguzzi kelvinson dangutdavid nanaakwasiabayieboateng damonclifford bhardwajrahul sgwd djpain africa-ai-engineers silas-asamoah vvkim20 pynchmeister gary7 synchronicity89 wheng3 edorexy danorul9 dp167 dongheekang sunbc0120 roaminggamer goseign ivorobyev charlottesean jkginfinite parthnagarkar875 eltoro13 halixness hercules261188 agracian saiyerniakhil chenmoshushi chandc 4thbosz vmuthuk2 katkamarun abdaladiasse mohamed-elkahef bellinsd

make_money_with_tensorflow_2.0's Issues

How doo i run this

How do i run this theres no python file i see to make this run

Broken link found in readme

This is the code for this video on Youtube by Siraj Raval on Making Money with Tensorflow 2.0.

This line has broken link

K is not defined while trying to run build_model function.

"TypeError: can't pickle _thread.RLock objects" when saving the model calling "multi_head.save(filepath)"

The problem comes when calling "model.get_config()". Any idea why this could happen?

ValueError Traceback (most recent call last)

======DDPG Validation from: 2021-10-04 to 2022-01-03======Best Model Retraining from: 2010-01-01 to 2022-01-03======Trading from: 2022-01-03 to 2022-04-04

ValueError Traceback (most recent call last)
in <cell line: 1>()----> 1 df_summary = ensemble_agent.run_ensemble_strategy(A2C_model_kwargs, 2 PPO_model_kwargs, 3 DDPG_model_kwargs, 4 timesteps_dict)

5 frames
/usr/local/lib/python3.10/site-packages/pandas/core/internals/construction.py in _extract_index(data) 643 644 if not indexes and not raw_lengths:--> 645 raise ValueError("If using all scalar values, you must pass an index") 646 647 if have_series:
ValueError: If using all scalar values, you must pass an index
Sent message. ======DDPG Validation from: 2021-10-04 to 2022-01-03======Best Model Retraining from: 2010-01-01 to 2022-01-03======Trading from: 2022-01-03 to 2022-04-04 --------------------------------------------------------------------------- ValueError Traceback (most recent call last) in <cell line: 1>()----> 1 df_summary = ensemble_agent.run_ensemble_strategy(A2C_model_kwargs, 2 PPO_model_kwargs, 3 DDPG_model_kwargs, 4 timesteps_dict) 5 frames /usr/local/lib/python3.10/site-packages/pandas/core/internals/construction.py in _extract_index(data) 643 644 if not indexes and not raw_lengths:--> 645 raise ValueError("If using all scalar values, you must pass an index") 646 647 if have_series: ValueError: If using all scalar values, you must pass an index

I got this error, after just running it. No modified have been make , So any thing to fix it ,, Thank you event Bard, OpenAI did not have way to fix it .

ValueError: Found array with 0 sample(s) (shape=(0, 1)) while a minimum of 1 is required by MinMaxScaler.

Trying to train the scalar with training data and smooth data:

smoothing_window_size = 2500
for di in range(0,10000,smoothing_window_size):
scaler.fit(train_data[di:di+smoothing_window_size,:])
train_data[di:di+smoothing_window_size,:] = scaler.transform(train_data[di:di+smoothing_window_size,:])

I get this error:
ValueError Traceback (most recent call last)
in
2 smoothing_window_size = 2500
3 for di in range(0,10000,smoothing_window_size):
----> 4 scaler.fit(train_data[di:di+smoothing_window_size,:])
5 train_data[di:di+smoothing_window_size,:] = scaler.transform(train_data[di:di+smoothing_window_size,:])

~\Anaconda3\lib\site-packages\sklearn\preprocessing\data.py in fit(self, X, y)
306 # Reset internal state before fitting
307 self._reset()
--> 308 return self.partial_fit(X, y)
309
310 def partial_fit(self, X, y=None):

~\Anaconda3\lib\site-packages\sklearn\preprocessing\data.py in partial_fit(self, X, y)
332
333 X = check_array(X, copy=self.copy, warn_on_dtype=True,
--> 334 estimator=self, dtype=FLOAT_DTYPES)
335
336 data_min = np.min(X, axis=0)

~\Anaconda3\lib\site-packages\sklearn\utils\validation.py in check_array(array, accept_sparse, dtype, order, copy, force_all_finite, ensure_2d, allow_nd, ensure_min_samples, ensure_min_features, warn_on_dtype, estimator)
460 " minimum of %d is required%s."
461 % (n_samples, shape_repr, ensure_min_samples,
--> 462 context))
463
464 if ensure_min_features > 0 and array.ndim == 2:

ValueError: Found array with 0 sample(s) (shape=(0, 1)) while a minimum of 1 is required by MinMaxScaler.

Apologies if I'm missing something glaringly obvious but I'm at a loss.

Normalize the data

Hello Siraj
I was Implementing the code and found out that the following section of code is not working for less data (2000 training points).

# Train the Scaler with training data and smooth data
smoothing_window_size = 2500
for di in range(0,10000,smoothing_window_size):
    scaler.fit(train_data[di:di+smoothing_window_size,:])
    train_data[di:di+smoothing_window_size,:] = scaler.transform(train_data[di:di+smoothing_window_size,:])

So i change it to following code:

# Train the Scaler with training data and smooth data
scaler = MinMaxScaler()
smoothing_window_size = 250   # Changed 
for di in range(0, len(train_data), smoothing_window_size):  # Changed 
     scaler.fit(train_data[di:di + smoothing_window_size, :])
     train_data[di:di + smoothing_window_size, :] = scaler.transform(
         train_data[di:di + smoothing_window_size, :])

By this part of code we dont have to write this portion:

# You normalize the last bit of remaining data
scaler.fit(train_data[di+smoothing_window_size:,:])
train_data[di+smoothing_window_size:,:] = scaler.transform(train_data[di+smoothing_window_size:,:])

Please let me know if I am doing something wrong.
Big fan of your work, I really appreciate it 👍

IndentationError: expected an indented block

for row in rows_noheader:
[date, value] = row[1:len(row)-1].split('\t')
stock_date.append(date)
stock_value.append((value))
stock_data.append((date, value))
for row in rows_noheader:
[date, value] = row[1:len(row)-1].split('\t')
stock_date.append(date)
stock_value.append((value))
stock_data.append((date, value))

#Numpy array of all closing values converted to floats and normalized against the maximum
stock_value = np.array(stock_value, dtype=np.float32)
normvalue = [i/max(stock_value) for i in stock_value]

#Number of closing values and days. Since there is one closing value for each, they both match and there are 4528 of them (each)
nclose_and_days = 0

How to merge the two repositories?

Hey,

Can someone please explain exactly what needs to happen in this step:

TODO: Step 1 - Merge the two repositories by starting with the simple TF serving demo. Copy and paste the user auth + SQL code from the boilerplate demo into the simple TF serving demo.

I've tried just copying all of the files in the .\flaskSaaS-master\app\templates folder into the ./simple_tensorflow_serving/templates folder but of course run into an issue when trying to run the serving server again. Siraj glances over that part in his video. Thanks.