Using multidimensional LSTM neural networks to create a forecast for Bitcoin price

Jupyter Notebook 97.35% Python 2.65%

multidimensional-lstm-bitcoin-time-series's People

Contributors

Stargazers

Watchers

Forkers

wavelets pustar hitum-dev leedaga tartaruszen yipeng5 ituco jianfly lefnire tonypius zfy1989lee rosdyana foxis tony-blake easyrider blesson3 leechang-soo miguelperalvo aimlnerd steelhand primekun raveeef mi5victim cuky88 bemot skyjiao sompalarya1 mohala562 daheechoi1208 xieguobin darwile jmwdpk charleswg heyuhere bwv988 hlo415 echo-ray westamine sadhumangal saisriharshas tony32769 elulue denethor1997 zandaya dgq2011 immmmmp afcarl jbdatascience deepblueparticle luciojp yueshuai8086 mazaixing llsourcell kritikalcoder doyley91 auserj github59987 zhanglc yyang88 xaviergoby tanshinepan christophergs zoonono akmalkhalid benikah gitouyou badobaron geraybos mahmud83 ritesh1991 tienbt93 satroan mfzhang sdhuangao nanaomonika alex-mishin libardo1 celinecheng1029 amir3411701 saadmahboob walobit ssgalitsky mustapha-wasseja ernestcr algorithmictradinglstm dvorson pysca stevengao1999 crackend battyone joe-nano kavymp ravipurohit1991 suvrajeet01 pratheek-kalva rahul5757 2017wxyzwxyz monkeyduck maybeee18 adesuyitolulope

multidimensional-lstm-bitcoin-time-series's Issues

I ran the program but what datasets from kaggle or quandl show it works well for time series data, any related to predictive mantanance?

how to understand your Standardise method - dataframe div first row

Dears,

I am confused about the standardise method in the code -

    def zero_base_standardise(self, data, abs_base=pd.DataFrame()):
        """Standardise dataframe to be zero based percentage returns from i=0"""
        if(abs_base.empty): abs_base = data.iloc[0]
        data_standardised = (data/abs_base)-1
        return (abs_base, data_standardised)

My understanding about the standarise is -

(x - mean) / standard_deviation
mean = sum(x) / count(x)
standard_deviation = sqrt( sum( (x - mean)^2 ) / count(x))

I think (data/abs_base)-1 should be far from standardised values, could you please kindly let me know if I misunderstood anything?
Thank you very much!

I coppied and pasted code from multidementional forcasting into a spreadsheet for the bitcoin multi dimentional forcasting software to run on 300 lines

I coppied and pasted code from multidementional forcasting into a spreadsheet for the bitcoin multi dimentional forcasting software to run on 300 lines.

I am not shure why the data line plotted only and the predicted line was behind it or invisible or not plotted.

I pasted it for all the columns the 2 columns of data.

here is where I got my data set from
https://www.itl.nist.gov/div898/handbook/pmc/section4/pmc451.htm

After training I do not get valid results...

Any suggestions?

TypeError: Cannot interpret feed_dict key as Tensor: Tensor Tensor("lstm_1_input:0", shape=(?, ?, 4), dtype=float32) is not an element of this graph.

Fresh clone, data/bitcoin.csv unzipped, Keras(2.0.6) TensorFlow(1.2.1) Python(3.6.2) (full pip freeze).

[Edit] Also tried on Python 2.7, same error. (full pip freeze)

Full error:

(btc3) lefnire@lefnire-ubuntu:~/Sites/btc/github/Multidimensional-LSTM-BitCoin-Time-Series$ python run.py 
Using TensorFlow backend.
> Creating x & y data files...
> Clean datasets created in file `data/clean_data.h5.h5`
> Generating clean data from: data/clean_data.h5 with batch_size: 100
> Clean data has 180610 data rows. Training on 144488 rows with 722 steps-per-epoch
> Compilation Time :  0.010142087936401367
> Testing model on 36122 data rows with 361 steps
2017-08-09 17:15:15.447882: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.1 instructions, but these are available on your machine and could speed up CPU computations.
2017-08-09 17:15:15.447905: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.2 instructions, but these are available on your machine and could speed up CPU computations.
2017-08-09 17:15:15.447909: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX instructions, but these are available on your machine and could speed up CPU computations.
2017-08-09 17:15:15.447913: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX2 instructions, but these are available on your machine and could speed up CPU computations.
2017-08-09 17:15:15.447917: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use FMA instructions, but these are available on your machine and could speed up CPU computations.
2017-08-09 17:15:15.563391: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:893] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2017-08-09 17:15:15.563705: I tensorflow/core/common_runtime/gpu/gpu_device.cc:940] Found device 0 with properties: 
name: GeForce GTX 1080 Ti
major: 6 minor: 1 memoryClockRate (GHz) 1.582
pciBusID 0000:01:00.0
Total memory: 10.90GiB
Free memory: 10.02GiB
2017-08-09 17:15:15.563716: I tensorflow/core/common_runtime/gpu/gpu_device.cc:961] DMA: 0 
2017-08-09 17:15:15.563719: I tensorflow/core/common_runtime/gpu/gpu_device.cc:971] 0:   Y 
2017-08-09 17:15:15.563724: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1030] Creating TensorFlow device (/gpu:0) -> (device: 0, name: GeForce GTX 1080 Ti, pci bus id: 0000:01:00.0)
> Compilation Time :  0.009964227676391602
Epoch 1/2
Exception in thread Thread-1:
Traceback (most recent call last):
  File "/home/lefnire/anaconda3/envs/btc3/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 942, in _run
    allow_operation=False)
  File "/home/lefnire/anaconda3/envs/btc3/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 2584, in as_graph_element
    return self._as_graph_element_locked(obj, allow_tensor, allow_operation)
  File "/home/lefnire/anaconda3/envs/btc3/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 2663, in _as_graph_element_locked
    raise ValueError("Tensor %s is not an element of this graph." % obj)
ValueError: Tensor Tensor("lstm_1_input:0", shape=(?, ?, 4), dtype=float32) is not an element of this graph.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/lefnire/anaconda3/envs/btc3/lib/python3.6/threading.py", line 916, in _bootstrap_inner
    self.run()
  File "/home/lefnire/anaconda3/envs/btc3/lib/python3.6/threading.py", line 864, in run
    self._target(*self._args, **self._kwargs)
  File "run.py", line 64, in fit_model_threaded
    epochs=configs['model']['epochs']
  File "/home/lefnire/anaconda3/envs/btc3/lib/python3.6/site-packages/keras/legacy/interfaces.py", line 87, in wrapper
    return func(*args, **kwargs)
  File "/home/lefnire/anaconda3/envs/btc3/lib/python3.6/site-packages/keras/models.py", line 1117, in fit_generator
    initial_epoch=initial_epoch)
  File "/home/lefnire/anaconda3/envs/btc3/lib/python3.6/site-packages/keras/legacy/interfaces.py", line 87, in wrapper
    return func(*args, **kwargs)
  File "/home/lefnire/anaconda3/envs/btc3/lib/python3.6/site-packages/keras/engine/training.py", line 1840, in fit_generator
    class_weight=class_weight)
  File "/home/lefnire/anaconda3/envs/btc3/lib/python3.6/site-packages/keras/engine/training.py", line 1565, in train_on_batch
    outputs = self.train_function(ins)
  File "/home/lefnire/anaconda3/envs/btc3/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py", line 2268, in __call__
    **self.session_kwargs)
  File "/home/lefnire/anaconda3/envs/btc3/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 789, in run
    run_metadata_ptr)
  File "/home/lefnire/anaconda3/envs/btc3/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 945, in _run
    + e.args[0])
TypeError: Cannot interpret feed_dict key as Tensor: Tensor Tensor("lstm_1_input:0", shape=(?, ?, 4), dtype=float32) is not an element of this graph.

I realize you're likely not keen on supporting a blog-post's code-demo, but just in case someone has top-of-the-dome.

When I return the values to original scale the result is very poor

As you can see I scaled back everything to the original. You can also notice the SP500 trend is usually upward rather the scaled graph predicted many down trends.

I wonder what to solve "Cannot interpret feed_dict key as Tensor is not an element of this graph." problem

When I predict the result, the kernel told me this warning, so I have no idea what's going wrong.

Question regarding predict_sequences_multiple function

I'm trying to understand your implementation of predict_sequences_multiple() and especially the way you insert your predictions to the current frame.

As I read your code, the prediction made from the current frame, gets inserted here replacing the last element of the curr_frame array, which makes sense since you are slowly using your previous predictions to make new predictions, and hereby create the ability to predict t+n, rather than t+1.

But I notice that you are inserting all 4 columns values with the predicted y-hat value (bitcoin prize)? (please correct me if I'm wrong about that).

This puzzles me, as I think it would confuse the model to mix the predicted bitcount prize with the 4 features of the dataset (Open, Close, Volume (BTC) and Volume (Currency)) - do you agree?

I've inspected the code several times with the debugger of pycharm, so I'm pretty sure that is what the code is doing, so I guess my question really is why that is the approach you have taken? I understand that the model.predict() method from Keras expects an array of the same shape as it was trained, and you achieve that by doing so, but is this approach "is the way to it", or am I wrong about something?

btw thanks for a great ressource about forecasting with t+n!

Jesper.

Tensor type error

Hi,

When I try to train the LSTM I get the following error:

TypeError: Cannot interpret feed_dict key as Tensor: Tensor Tensor("lstm_1_input:0", shape=(?, ?, 4), dtype=float32) is not an element of this graph.

Any suggestions?
Thanks!

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.

jaungiers / multidimensional-lstm-bitcoin-time-series Goto Github PK