Chapter 4 - How is weight_delta computed ? about grokking-deep-learning HOT 6 OPEN

I think the derivative is 2 * ((0.5weight) - 0.8) * 0.5, that is 2 * 0.5 * ((0.5weight) - 0.8) , so the result is 0.5*x - 0.8

from grokking-deep-learning.

the general forum for this : 2 * ((input*weight) - goal_pred) * input.
in nerualnetwork people may dont care about the exactly coefficient of derivative, so just omit 2 and leave the key part of the derivative

from grokking-deep-learning.

Hmm, if error = (input * weight - goal_pred) ** 2
should derivative not be 2 * (input * weight - goal_pred)?
Since in this example input = 2, it's the same but I'm also confused...

from grokking-deep-learning.

Hmm, if error = (input * weight - goal_pred) ** 2
should derivative not be 2 * (input * weight - goal_pred)?
Since in this example input = 2, it's the same but I'm also confused...

the derivative should be 2 * weight * (input * weight-goal_pred),that's the chain rule, you also should do derivative to input * weight

from grokking-deep-learning.

As far as I'm concerned, weights_delta in 4th Chapter are calculated via delta rule

Just to clarify:
The Delta rule is an update rule for single layer NN. It makes use of Gradient Descent.
Backpropagation is an update rule for multi layer NN based on Gradient Descent.

from grokking-deep-learning.

But if we are using direction_and_amount = (pred - goal_pred) * input * 2 (e.g., not omitting the two), the model converges much faster?

from grokking-deep-learning.