Comments (10)
Jasper, do you want the full Jacobian of a vector-to-vector function or do you just want its diagonal? If it's the Jacobian itself that you want, then you have to loop over the gradients of each output component with respect to the input vector, as Matt shows. If you want just the diagonal, and if the off-diagonal elements are zero (the conversations we've had makes me think that's what you're talking about) then you can just use the gradient of the sum of the output, and the calculation happens in a single pass. For example:
>>> import autograd.numpy as np
>>> from autograd import grad
>>> def jac_diag(fun):
... return grad(lambda x : np.sum(fun(x)))
...
>>> x = np.linspace(-3, 3, 5)
>>> jac_diag(np.sin)(x)
array([-0.9899925, 0.0707372, 1. , 0.0707372, -0.9899925])
>>> np.cos(x)
array([-0.9899925, 0.0707372, 1. , 0.0707372, -0.9899925])
I could add the wrapper functions jacobian
(Matt's D
) and jac_diag
if you think they'd be useful...
from autograd.
This is a bit messy but it's a quick wrapper:
import autograd.numpy as np
from autograd import grad
def D(f,outdim):
def f_i(i):
return lambda *args, **kwargs: f(*args,**kwargs)[i]
def deriv(*args,**kwargs):
return np.concatenate(
[grad(f_i(i))(*args,**kwargs)[None,...] for i in xrange(outdim)])
return deriv
It can be used like
A = np.random.randn(3,3)
def test(v):
return np.dot(A,v)
print D(test,3)(np.ones(3))
print
print A
I'm sure it can be improved but I think it reflects the best general strategy (for reverse mode). EDIT: it would be easy to change outdim
to outshape
, too.
from autograd.
Those wrappers would be useful!
from autograd.
That's really neat @mattjj. I already used your solution to quickly put together a Kayak module :-) I took the diagonal of the output of Matt's solution, but yes taking the sum is much simpler. Those wrappers are tremendously useful, but I'm not sure where exactly they'd fit in to autograd.
from autograd.
Ok, I'll put them in autograd.util for now
from autograd.
Sounds like there was side channel information about what Jasper really wanted!
Support for general derivatives (of maps from R^n to R^m), a.k.a. Jacobians, would be a nice feature even if it's not the main thrust of the library. Then autograd could be used for easy implementations of e.g. extended Kalman filters and smoothers (unless I'm missing something).
Maybe the jacobian
function could avoid the outdim
(or outshape
) argument if it ran a single forward pass the first time it was called and inspected (and cached) the shape of the result.
from autograd.
@dougalm I don't think that jac_diag
function returns the diagonal of the jacobian in general:
def test2(x):
return np.array([np.sum(x), np.sum(x**2), np.sum(x**3)])
print D(test2,3)(np.ones(3))
def jac_diag(fun):
return grad(lambda x: np.sum(fun(x)))
print jac_diag(test2)(np.ones(3))
# prints:
# [[ 1. 1. 1.]
# [ 2. 2. 2.]
# [ 3. 3. 3.]]
# [ 6. 6. 6.]
EDIT: oh you said "if the off-diagonal elements are zero" of course!
from autograd.
Exactly. It's a common case that people seem interested in. They have a scalar-to-scalar function and they want its gradient at a number of places. Mike Gelbart and Jon Malmaud were both cross that grad
doesn't automatically do this when you give it a vector-to-vector function.
But perhaps jac_grad
is a misleading name. Maybe elementwise_grad
? or map_grad
?
from autograd.
Or maybe diag_jac
. I'm probably parsing too much here, but that could be slightly more suggestive that it computes a diagonal Jacobian (represented by its diagonal elements) rather than the Jacobian's diagonal (in the general case). On the other hand jac_diag
probably conveys pretty much the same thing and the docstring can be used to spell out the constraints :).
from autograd.
Done.
from autograd.
Related Issues (20)
- support for Jax-like custom forward pass definition? HOT 1
- Gradient become Nan for 0 value test HOT 1
- Is it possible to see gradient function? HOT 2
- Four scipy tests are failing HOT 6
- Add np.float128,np.complex256 dtypes to vspaces and boxes registers HOT 2
- unsafe URL HOT 1
- Numpy 1.25 breaks a few linalg functions HOT 1
- `autograd` 1.6 breaks Apple M-series macOS and Windows builds (module `numpy` has no attribute `float128`) HOT 3
- Can I differentiate this function?
- Python 2 and dependency on future HOT 2
- `'ArrayBox' object has no attribute 'dot'` when differentiating function containing `x.dot(y)` HOT 1
- How do I create a scalar value that does not depend on the independent variables ?
- AttributeError: module 'autograd.numpy' has no attribute 'numpy_extra'
- Autograd for quantum circuits
- [BUG] Differentiating `autograd.numpy.linalg.norm` gives incorrect results
- Support for advanced library based on autograd HOT 4
- Release new version 1.6.3 on PyPI HOT 3
- autograd return nan with to norm function
- Saving `ArrayBox` to hdf5 file
- Incompatibility with numpy 2.0.0 HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from autograd.