aiuiuc / pyaiutils Goto Github PK

https://www.tensorflow.org/versions/r0.12/api_docs/python/state_ops.html#deprecated-functions-removed-after-2017-03-02-please-don-t-use-them

https://www.tensorflow.org/versions/r0.12/api_docs/python/state_ops.html#all_variables

https://www.tensorflow.org/versions/r0.12/api_docs/python/state_ops.html#initialize_all_variables

https://www.tensorflow.org/versions/r0.12/api_docs/python/state_ops.html#initialize_local_variables

https://www.tensorflow.org/versions/r0.12/api_docs/python/state_ops.html#initialize_variables

In the layers.py we are initializing the weights by the following line:

w_init = tf.random_normal_initializer(stddev)

but the first argument should be mean (which is 0.0 in this case). so it should be updated to:

w_init = tf.random_normal_initializer(0.0, stddev)

I would do this myself, if I had write permission! :)

Currently, batchnorm doesn't work as expected with tf12. I think this is due to changes in how the EMA functions, since you have to pass in a value for placeholders (likely to initialize the EMA). I don't know how to fix this as of yet.

I think something like:

multi_rate_optimizer(optimizer, learning_rate_list, variables_list, loss)

That returns a properly canned opt which computes gradients and applies them could be useful.

Eg code: http://stackoverflow.com/questions/34945554/how-to-set-layer-wise-learning-rate-in-tensorflow

#41

Since we don't have our own guide lines for code review, I suggest we adopt some.

I found these:

• Effective code review
• Code review checklist

Of course nothing will be enforced, but just to set the tone. I do like the idea of getting in the habit of doing a little bit of code review though.

Hi @arunmallya @kevjshih @BigRedT,
It sounds like there is interest in continuous integration for this repo. And since we have a few tests, it seems like a good idea.

Someone suggested Travis CI. It seems like all you need to get travis ci working is:

• Give travis ci access to your repo.
• Make a config file called .travis.yml that says: which version of python to try, which dependencies need to be installed, and what testing package to use.
• Save the .travis.yml in your repo.

Blocks is a neural network library that used a lot of the python scientific computing stack and they use travis ci too! We can probably copy a lot of what they do. Here is their .travis.yml file: link

Who wants to try doing this? 😈

Per convo with @BigRedT