betomorrow / reimprovejs Goto Github PK

Hello! Really nice library, thank you!
I am bit confused, I've trained model with good results, saved it to file and now I would like to use with model.predict in "plain" tensorflow.js or even something like agent.predict.
How can I do that? I am new to tensorflow.js, tried multiple options, always getting errors that shape is different than expected on tensor2d contruction.

Thank you for your help.

I tried saving the model using tensorflow's model.save().

But when I tried using it after loading, I got an error:
"this.model.randomOutput is not a function"

Please guide on how we can save and load the model to continue training.

First of all I want to thank you for providing this library to the public!
What are your plans currently regarding finishing the implementation for convolutions? And is there like a Todo-List that others could look at to contribute?

cheers & keep up the good work!

Hello, thanks for putting all your work out there 💯
Is it possible to train on a recorded data set? Specifically from something like a csv.txt?

Hi,

I am still new to Reinforcement learning, I remember Reinforce JS did have this act() function, which takes as input the state variables and outputs an action. How can we act upon the output of the RL agent in ReImproveJS?

thanks in advance!

When using your example code I get an error at Academy.step:

I do the following:

// Animation loop, update loop, whatever loop you want
async function step(time) {
    let inputs = [1,2]; 
  await academy.step([               // Let the magic operate ...
        {teacherName: teacher, agentInputs: inputs}
    ]);
}

but I always get an Error no matter what the input is:

reimprove.js:1 Uncaught (in promise) TypeError: Cannot read property 'length' of undefined
   at Agent.forward (reimprove.js:1)
    at Agent.listen (reimprove.js:1)
    at reimprove.js:1
    at Set.forEach (<anonymous>)
    at Teacher.<anonymous> (reimprove.js:1)
    at step (reimprove.js:1)
    at Object.next (reimprove.js:1)
    at reimprove.js:1
    at new Promise (<anonymous>)
    at __awaiter (reimprove.js:1)

Hey, so I am just trying to get a basic example working that can tell if a number is even or odd. I have tried running multiple examples but can't seem to get them working. Just wanted to ask if someone here could help me out with a basic working example. Thanks!

If I wanted to train a feedforward network agent with ~100 inputs, 8 outputs, and a hidden layer of 512 or so, can I use a DQN from this library to do it and expect it to work out okay?

Does the DQN have any of the bells and whistles such as dueling DQN, prioritized experience replay, etc.?

On the MIT lecture e.g. they have an example of a DQN with convolutional layers.
https://youtu.be/i6Mi2_QM3rA?t=1531
Or here: https://towardsdatascience.com/welcome-to-deep-reinforcement-learning-part-1-dqn-c3cab4d41b6b

Would be interested if this is a problem in general or just with this implementation of DQN. I couldn't find anywhere that it is a problem.

Hi!

Great looking library! I have been able to produce some simple enough things using the 0.0.2 version of ReImproveJS, however a couple of issues that are preventing me from taking this any further;

NPM's latest version is 0.0.2, which doesn't have updated TS or TF and no helper for exporting the accumulated model.

0.2.5 release doesn't contain the compiled code, and when trying to build, I am met with the following error, meaning 0.2.5 is impossible for me to access.

node_modules/@tensorflow/tfjs-converter/dist/src/data/compiled_api.ts:20
export declare interface IAny {
^
ParseError: 'import' and 'export' may appear only with 'sourceType: module'

Any help would be appreciated.

Tom

I am getting this error when running ReimproveJs

Uncaught (in promise) Error: Error when checking : expected dense_Dense1_input to have shape [null,12] but got array with shape [1,14].
    at new ValueError (reimprove.js:1)
    at checkInputData (reimprove.js:1)
    at Model.predict (reimprove.js:1)
    at Sequential.predict (reimprove.js:1)
    at Model.predict (reimprove.js:1)
    at reimprove.js:1
    at Object.Tracking.tidy (reimprove.js:1)
    at Agent.createTrainingDataFromMemento (reimprove.js:1)
    at reimprove.js:1
    at Array.map (<anonymous>)

This error is not happening directly but after 5 seconds of it working. its called by academy.step
My part, only added inputs:

let inputs = [speed,turn,sensor1Dist,sensor2Dist,sensor3Dist];          // Need to give a number[] of your inputs for one teacher.
        let result = await academy.step([               // Let the magic operate ...
        {teacherName: teacher, agentsInput: inputs}
        ]);

I see a loadFromFile() method but I'm not sure how to save the model. Is there an easy way to do that?

https://repl.it/@NicholasFicara/Flappy-AI

This is the code that I am working on. I got the whole ai part down. I was wondering how I could implement the step method. Every way I have done it so far has given errors. The update() function is called every 1/60 of a second. I wanted it to run there to make the decision to flap or not. Right now I get this error.

Uncaught (in promise) TypeError: Cannot read property 'randomOutput' of undefined

Thanks

The Loss function in your example code does not exist anymore:

You should change this line:

// Finally compile the model, we also exactly use tfjs's optimizers and loss functions
// (So feel free to choose one among tfjs's)
model.compile({loss: 'crossEntropy', optimizer: 'sgd'})

to something like
model.compile({loss: 'meanSquaredError', optimizer: 'sgd'})

It seems that the model must goes inside the Agent config:
model.compile({loss: 'categoricalCrossentropy', optimizer: 'sgd'}) const agentConfig = { model: model, memorySize: 5000, batchSize: 128, temporalWindow: temporalWindow };

await academy.step([ {teacherName: teacher, inputs: inputs} ]);
The parameters in the interface AcademyStepInput are "teacherName" and "agentsInput".

Hi,
Thank you for your job.
I'm working for fun with your lib, but I don't clearly understand the concept of lesson (lessonsQuantity, lessonsLength, and lessonsWithRandom) and how to use lessons callback.

Imagine I'm trying to train 50 agents to play to Flappy bird. How to setup my lessons please ?

Thank you,
Jeremie.