Hi, my name is Luis Molina, and I am developing this open source project:

https://github.com/Superalgos/Superalgos

Some community members and me are thinking about using your experiment as a first ML implementation at our project. Maybe you like the idea and would like to help?

We are currently discussing just this at this Telegram group: https://t.me/superalgosmachinelearning ,maybe you would like to show up?

Superalgos is a free and open source software to automate crypto trading.

Hi this is a great guide and i've managed to use it to integrate a bit of forecasting in to a BI tool i was working on so thank you.

I was just wondering if it would be at all possible to feed the result of the prediction back into itself in order to increase the forecast window a week or so.

Hello,

First of all, thank you for great tutorial. And sorry for the long post.

Was reading through the code and there are a couple of things I can't understand. And maybe some potential improvement ideas.

Improvements: Data selection

You are using the "price" for your calculations which I believe isn't the best value, you usually want an "Adjusted" (aka Adj) value. When you look at the stock prices for Apple (AAPL) for example, they did a stock split about 1-2 months ago (and end of 2014) and you can see in the data that it has a huge drop in "price" because it was split (ex: 1 share now became 2 shares)...so the results won't be correct. I would advise to use Adj Close or similar. Tesla also recently did a split.

(FYI: I ended up using Yahoo finance to get data that was Adj in my model)

Maybe not correct: Calculation of the SMA

I was looking at your ComputeSMA

function ComputeSMA(data, window_size)
{
  let r_avgs = [], avg_prev = 0;
  for (let i = 0; i <= data.length - window_size; i++){
    let curr_avg = 0.00, t = i + window_size;
    for (let k = i; k < t && k <= data.length; k++){
      curr_avg += data[k]['price'] / window_size;
    }
    r_avgs.push({ set: data.slice(i, i + window_size), avg: curr_avg });
    avg_prev = curr_avg;
  }
  return r_avgs;
}

I might be reading the code wrong but in curr_avg += data[k]['price'] / window_size you are dividing the price by the window_size before adding it to the average. Ex:

const data = [1,2,3,4,5]
const window_size = data.length // => 5
// What it seems you are doing:
const avg = 1/5 + 2/5 + 3/5 + 4/5 + 5/5. // => 2.83
// How average is calculated
const avg2 = (1+2+3+4+5) / 5 // => 3 

Maybe issue: Where do you offset your Y results?

As I was reading your ComputeSMA, it seems like the set is the avg. Once again I might be wrong, but here are my thoughts. In your onClickTrainModel, you have

 let inputs = sma_vec.map(function(inp_f){
    return inp_f['set'].map(function(val) { return val['price']; })
  });
  let outputs = sma_vec.map(function(outp_f) { return outp_f['avg']; });

So it is taking the direct output from your ComputeSMA I believe. Now, when we look at your ComputeSMA again, your average is calculated like so:

 let curr_avg = 0.00, t = i + window_size;
    for (let k = i; k < t && k <= data.length; k++){
      curr_avg += data[k]['price'] / window_size;
    }

So it's taking the values from i to t which is i+window_size (so: i to i+window_size) to calculate the avg, and your set is also data.slice(i, i + window_size). So I'm not sure where you are offsetting your Y values for your model.

Question: Model

Any chance you can explain in more details how you build your model?
Some examples, are:

• How do you decide input_layer_neurons = 100? What does it change?
• Same for const rnn_input_layer_features = 10
• What does .div(tf.scalar(10)) do on your tensors? Does it normalize the data?

Any help much appreciated.

Can you submit a sample data format that will be feed in the model? Because I don't want to use API to fetch data.

Hello this is a very interesting project. I tried with the default graphics (MSFT) and I got a great result.

But now, I am trying to train a model for EURUSD asset from Forex (15 minutes timeframe)

But I am getting this result... and sometimes is just a rect line...

I am trying to understand this better, I am not experienced with neural networks before, but may you able to help me to understand?

Thank you!

Every time I run any test I am getting a flat line. so when I train on 80% data and validate on the 20% remaining, line 225 gives results:
(220) [51.777862548828125, 51.78431701660156, 51.84973907470703, 51.965545654296875, 52.03998947143555, 52.07977294921875, 52.12873458862305, 52.22467041015625, 52.245277404785156, 52.23814392089844, 52.30770492553711, 52.34142303466797, 52.36872100830078, 52.380374908447266, 52.41004180908203, 52.414215087890625, 52.41645431518555, 52.433067321777344, 52.436336517333984, 52.4455451965332, 52.45906066894531, 52.465614318847656, 52.461029052734375, 52.469947814941406, 52.47417449951172, 52.47554397583008, 52.48143005371094, 52.48564147949219, 52.48596954345703, 52.49354934692383, 52.50459671020508, 52.51314163208008, 52.52178192138672, 52.5322265625, 52.54399108886719, 52.551612854003906, 52.55399703979492, 52.56175231933594, 52.57003402709961, 52.564884185791016, 52.56238555908203, 52.568458557128906, 52.57207489013672, 52.576416015625, 52.585086822509766, 52.589359283447266, 52.58890151977539, 52.58769989013672, 52.589717864990234, 52.59192657470703, 52.597312927246094, 52.5996208190918, 52.60268783569336, 52.606224060058594, 52.60929489135742, 52.61307144165039, 52.622291564941406, 52.628456115722656, 52.63402557373047, 52.638301849365234, 52.640830993652344, 52.640846252441406, 52.640708923339844, 52.642784118652344, 52.6439208984375, 52.64525604248047, 52.646976470947266, 52.64796447753906, 52.649070739746094, 52.651641845703125, 52.65275955200195, 52.653404235839844, 52.654991149902344, 52.65419387817383, 52.65365982055664, 52.655967712402344, 52.656612396240234, 52.656105041503906, 52.65705108642578, 52.65494918823242, 52.653289794921875, 52.65436553955078, 52.655303955078125, 52.65666580200195, 52.65773010253906, 52.65778350830078, 52.658077239990234, 52.659000396728516, 52.65916442871094, 52.65989303588867, 52.66079330444336, 52.66077423095703, 52.660621643066406, 52.66080856323242, 52.660499572753906, 52.66173553466797, 52.6634407043457, 52.6645622253418, 52.66503143310547, 52.66551971435547, …]

How exactly are you predicting next day? Can you please refer me to the code which predicts 51st day as shown in the picture you have shared?

The last thread on the strange result was closed, i am not sure if that was resolved, but i am also facing the same issue, so want to continue the thread.

I simply click on this link https://jinglescode.github.io/time-series-forecasting-tensorflowjs/

1.) I clicked fetch Data
2.) Window size - click compute SMA and draw chart
3.) Train data - train dataset size 80%, Epochs 25, learning rate 0.01, hidden LSTM layers 4 - the loss finishes at 0.0008373598102480173
4.) Click validate and the result comes at a flat line

So I've taken the time to change the code to use Nodejs instead of the html/js. So when I test with the MSFT dataset I get the expected results, but when I'm testing with a Covid-19 dataset using a very similar method I get loss at around 30. The lowest loss I got was 30 at using 90% for the trainingsize. I also posted a question on StackOverflow to see if I could get some help from there but haven't gotten any help yet. I think I've probably messed something up but I just can't find what.

Stackoverflow

Would be nice to see a way to save and load a model in your demo. It is a bit frustrating to deal with the tfjs lib.

Почему шаг не i+=windowsize?