benchmark datasets (for examples) about talos HOT 7 CLOSED

Ok sounds good. Thanks for clarifying.

Gonna store this here for safekeeping so I know where to look when I have time to work on this.

Reproducible results with Keras.

from talos.

I unfortunately do not. However, I know someone who might. Will get back to you 👍

from talos.

Unfortunately, no luck. I think this is a very complicated problem and finding the local minima of the hyperparameter space, so to speak, is not trivial.

On the other hand, there is a way to get reproducible results during development using Keras. What we could do is agree on a "good enough" result as a benchmark, initialize the same way every time to ensure the answer is the same.

from talos.

I think so. Regarding "good enough" I think that's quite important here >> the goal of an experiment is to find "good enough" model and once a given number of those models is found, the experiment can proceed to the steps we had discussed in #40. For example, somebody training a model for sentiment classification...good enough really can't be more than 90% as even human coders will disagree at about that rate (10% disagreement). So the first step would be to find "good enough" depending on the use-case (so user will input) and then the next step is to work on finding the best of many options at the good enough level where best now means generalization.

from talos.

For sure. And I think for the purposes of unit-checking Scan (I'm using the term "unit" quite flexibly here... Scan is many units!) I think this will be suitable.

I will consider this my next project here since I can do it more or less independently of Reporting. Are all the branches (dev, master, daily-dev) compatible right now and if not do you have a recommended branch to start working from?

Also I apologize for my lack of contributions as late. I've been completely slammed.

from talos.

@x94carbone Great :) I think we should merge dev with master at this point, to make sure all is in sync and then from master to daily-dev. I will create the pr.

from talos.

@x94carbone there was some issues in dev that had passed tests, but it's now fixed and dev and daily-dev are in sync and master has nothing new compared to dev.

from talos.