Comments (2)
Here's a design sketch for SMC2 implementation.
We make the following assumptions on the model to which SMC2 is applied:
- All calls to
param
are made at the beginning of the program, that is before any calls toassume
orobserve
. - All parameters are continuous.
- The dimension of the parameters vector is the same in all possible execution paths. We can later modify this assumption to allow infinite collections of parameters.
- The support of all parameters is the same in all possible execution paths.
- The number of
observe
s is the same in all possible execution paths.
Violating any of those conditions results in undefined behaviour, where we would produce an exception whenever possible.
We can implement SMC2 by introducing a new data structure called ThetaParticle
and using ParticleContainer{ThetaParticle}
. Each ThetaParticle
would start by creating a single execution thread that would go through the param
s, replaying supplied values if required. When the first probabilistic statement other than param
is reached, it would initialise a ParticleContainer{TaskC}
by copying its task n times. From that point on it would execute normal SMC, yielding control where required. I do not believe there are any problems with copying ParticleContainer{TaskC}
without replaying anything, so I don't think there would be any problems with copying ThetaParticle
. Therefore I hope we could implement SMC2 as a variant of SMC operating on ParticleContainer{ThetaParticle}
with custom rejuvenation step.
Let me know your thoughts. If it all seems sensible, I will write a more detailed implementation outline.
from turing.jl.
Making assumptions about param
explicit is a good idea. I did not find anything missing in this list : )
I thought about the implementation sometime ago, below are some notes:
Requirements:
- R1: Able to share theta's across x-particles belonging to the same theta-particle.
- R2: In the resampling step for theta particles, we need to duplicate theta-particles.
- R3: In the mutation/rejuvenations steps, we need to re-create theta's using external values (proposals).
- R4 (*): Able to support theta with varying dimensions.
(*): not urgent.
Two possible mechanisms:
- M1: Theta-particle replaying: the same as our existing SMC2 implementation, but with x-particle copying.
- M2: Theta-particle copying:
Pros of M2:
- Avoiding re-running executions generating thetas, thus computationally more efficient.
Cons of M2:
- There are some potential (future) risks when we want to support R4: in such cases, theta are indeed modified after creation, e.g. lazy arrays.
- R3 needs replaying external MH proposals to re-create theta's anyway.
NOTE: It first appears that we would need to use TArray
(or equivalents) everywhere since theta's are shared by multiple x-particles. But since theta values are not modified after they are created, the need for TArray
is no longer necessary.
from turing.jl.
Related Issues (20)
- Maths not displayed in variational inference writeup HOT 3
- Experimental Gibbs seems to have higher variance than "stable" Gibbs (maybe) HOT 3
- Better return type for mode estimation
- Undeterministic test failure HOT 2
- Tests to check that the correct AD method is actually used
- Setup continuous benchmarking for Tuirng/DynamicPPL
- Problem with using `predict` with vector valued random variables HOT 16
- Turing guaranteed to error (when compiled with GPUCompiler) HOT 7
- Style Guide and Style Checker HOT 13
- Dependent uniform variables not working correctly HOT 2
- Add `Tapir` to Turing's AD test suite
- Implementing `adtype` for closely linked "external" samplers / adding metric warm-up HOT 2
- Constrained or partitioned inference for Turing models.
- Compat issue HOT 13
- missing keyword arguments not property processed HOT 5
- Enable Aqua.test_ambiguities
- Progress reporting in parallel sampling HOT 3
- Using @distributed to accelerate but getting wrong results
- Mode estimation's support of Pathfinder integration HOT 5
- Docs on SGHMC / SGLD? HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from turing.jl.