rdipietro / jupyter-notebooks Goto Github PK
View Code? Open in Web Editor NEWLicense: Apache License 2.0
License: Apache License 2.0
The code uses only one training example.
What should I add to perform a mini-batch training?
Coud I use tensorflow.scan
over batch_size?
there is an error in the first equation
Hi,
Great article!
I have a question, not sure if it's OK if I post this here as an issue. If it's not I'll delete it.
When talking about encoding elements of a distribution in order to minimise the number of bits, i your article says this:
It turns out that if you have access to the underlying distribution y, then to use the smallest number of bits on average, you should assign log(1/y_i) bits to the i-th symbol.
Why log(1/y_i)?
How can we prove that this is the minimum?
Hi Rob!
I enjoy reading "A Friendly Introduction to Croo-Entropy Loss", but I have a few questions about the notation used, because I don't think that I've come across of it before.
I think the notation is explained well enough in the post, but I'm wondering if it's a standard notation in some fields, and if it is, if you could point me towards a webpage/pdf providing an overview of the notation :)
It's the red parts which is new to me.
Hi, I am a beginner in TensorFlow and I run into your brilliant scan tutorial. I wonder whether you could share some knowledge of iterating across multiple tensors at once and/or returning tuples from fn
.
So far I've tried passing a list of tensors (possibly wrapped by tf.identity
) but it doesn't work.
I believe this would be valuable extension of your post :)
Petr
I am not sure if everyone can reproduce this issue, but in the Content section
for Cross Entropy and About Me
, I am not able to open the links
Maybe you can give links like:
...
* [Cross Entropy](https://rdipietro.github.io/friendly-intro-to-cross-entropy-loss/#Cross-Entropy)
* [About Me](https://rdipietro.github.io/friendly-intro-to-cross-entropy-loss/#About-Me)
...
and it should work like a charm, something like:
...
...
Actually I believe that you can try giving hyperlinks in markdown using the same structure, i.e, instead of using <url>/cross-entropy
, use <url>/Cross-Entropy
(usage of same letters as in text, and separating them with hyphens) and it should work for all links in Contents Section
First, thanks for the excellent tutorial! I've always wondered about the relationship between entropy and cross-entropy loss, and you explained it perfectly.
I did notice a possible typo in the "Predictive Power" section:
is just the first entry of y^(1)=(0.4,0.1,0.5)T , which is y1(1)=0.4 .
Shouldn't that last y1(1) be y^1(1)?
I am miles off an understanding here i assume and this is my first time raising an issue on git so please i beg everyone's indulgence
The post makes the point that
Because we usually assume that our samples are independent and identically distributed, the likelihood over all of our examples decomposes into a product over the likelihoods of individual examples:
L({y(n)},{y^(n)})=โnL(y(n),y^(n))
(sorry first issue on git, do not know how to math)
Here is what my issue is:
To keep going with this example, let's assume we have a total of four training images, with labels {landscape, something else, landscape, house}, giving us ground-truth distributions y(1)=(1.0,0.0,0.0)Ty(1)=(1.0,0.0,0.0)T , y(2)=(0.0,0.0,1.0)Ty(2)=(0.0,0.0,1.0)T , y(3)=(1.0,0.0,0.0)Ty(3)=(1.0,0.0,0.0)T , and y(4)=(0.0,1.0,0.0)Ty(4)=(0.0,1.0,0.0)T
so are not y(1), y(2), y(3), y(4) having different distributions?
I can never remember whether y_hat is supposed to be the target value or the prediction output, which makes skim reading your article quite difficult.
Two possible solutions...
p
for the prediction output, t
for the true value.A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.