Comments (2)
Pathfinder fits a sequence of multivariate normal approximations to the target distribution. When the optimization trajectory converges to a MAP estimate (not guaranteed or enforced), then the final distribution in this sequence will approximate the Laplace approximation, where the closeness depends on the accuracy of the estimate of the inverse Hessian constructed from the history of the optimization trajectory. Of course, not all distributions have a MAP, and even if the optimization converges to a mode, that doesn't mean it's the MAP. This is why returning the ELBO-maximizing distribution in the sequence is important and useful for more distributions.
In the next release, the entire sequence of distributions will be exposed to the user, so they are free to just grab the last one and use it like a Laplace if they like (#19). I don't think we want to support constructing the exact Laplace distribution (i.e. computing the exact Hessian). This can be done in ~2 lines of code without Pathfinder, so it doesn't seem to be worth the extra complexity.
But we may want to support different optimizations than maximizing the ELBO (#15), and if we have an interface for that, it would be easy to provide one that just always chooses the last distribution as the one to be returned. But this would be wasteful compared to just 1) performing MAP and 2) computing the Hessian (or an approximation to it)
from pathfinder.jl.
Great point, just having access to the intermediate computations would be enough, and this package can stay light weight. I'll close this issue, let me know or reopen if you think there's more to discuss.
from pathfinder.jl.
Related Issues (20)
- Supporting inputs implementing the LogDensityProblems interface
- Fixing type-noninferrability from Optimization.solve HOT 1
- Using Optimization.MaxSense HOT 1
- Multi-threaded multi-path Pathfinder broken with recent Transducers versions HOT 4
- Support alternative ways of choosing normal approximations HOT 2
- More-Thuente line search fails for posterior
- TagBot trigger issue HOT 53
- Return all intermediates in a custom struct HOT 1
- Drop StatsFuns as a dependency
- Turing integration HOT 1
- Load time is doubled on the latest Julia beta
- Switching to Hager-Zhang line search HOT 1
- Making Optimization (formerly GalacticOptim) optional HOT 5
- multipathfinder - no method matching iterate HOT 6
- WoodburyPDMat and unwhitening and square norms HOT 2
- Making subpackages for compatibility with HMC packages HOT 1
- Choosing AD backend in Turing integration HOT 7
- NaNs introduced with 3 posteriordb models HOT 9
- Benchmark with posteriordb models HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pathfinder.jl.