Generative model snorkel vs Generative model metal about metal HOT 3 CLOSED

Hello @ajratner,
Thanks a lot for your quick answer. Are you planning in a next release to provide a method that calculates automatically the dependencies? (I saw this TODO in the the train_model method of the gen model training)

In the meantime, what would you suggest to use to calculate the deps that meets your requirements in order to work with metal ?

Thanks

from metal.

Hi @Wronskia there are a number of differences (see resp. papers online), but at a high level:

• Model: By default, MeTaL uses a slightly more expressive model, where conditional accuracy parameters are learned, i.e. an accuracy parameter for each (LF, class) pair; whereas the default model in Snorkel is just learning a single acc. param per LF
• Speed: MeTaL uses a new matrix completion-style approach which involves running e.g. SGD over a small matrix---which importantly does not scale wrt to the amount of unlabeled data---and thus is much faster; see recent AAAI-19 paper for details!

As to instability, not sure without seeing examples, but we'll look into it.

And regarding deps: In the current matrix completion-style approach, we require the set of dependencies to meet certain requirements (which guarantee that the resulting problem is uniquely solvable). My guess is that the deps from Snorkel's selection approach just don't necessarily uphold/check for this criterion; but we should at least create a better error message! @paroma @fredsala

from metal.

Hi @Wronskia — thanks for the question! I'll note that we've brought the best of MeTal to our v0.9 release of Snorkel — check it out!

We have Dependency Learning on our roadmap there; stay tuned for updates! (cc @paroma )

from metal.