Giter VIP home page Giter VIP logo

infer's Introduction

Infer.NET

Infer.NET is a framework for running Bayesian inference in graphical models. It can also be used for probabilistic programming.

One can use Infer.NET to solve many different kinds of machine learning problems - from standard problems like classification, recommendation or clustering through to customised solutions to domain-specific problems.

Infer.NET has been used in a wide variety of domains including information retrieval, bioinformatics, epidemiology, vision, and many others.

Contents

Build Status

Release
Windows Win Release
Linux Linux Release
macOS macOS Release

Installing pre-built binaries

Binaries for Infer.NET are located on nuget.org. These binaries are cross-platform and work anywhere that .NET is supported, so there is no need to select your platform. The core packages target .NET Standard 2.0, making them useable from any project that targets .NET framework version 4.6.2 or .NET Core 3.1, as explained at .NET implementation support. You do not need to clone the GitHub repository to use the pre-built binaries.

There currently are four maintained Infer.NET nuget packages:

  1. Microsoft.ML.Probabilistic contains classes and methods needed to execute the inference code.
  2. Microsoft.ML.Probabilistic.Compiler contains the Infer.NET Compiler, which takes model descriptions written using the Infer.NET API and converts them into inference code. It also contains utilities for the visualization of the generated code.
  3. Microsoft.ML.Probabilistic.Learners contains complete machine learning applications including a classifier and a recommender system.
  4. Microsoft.ML.Probabilistic.Visualizers.Windows contains an alternative .NET Framework and Windows specific set of visualization tools for exploring and analyzing models.

NuGet packages do not need to be manually downloaded. Instead, you add the package name to your project file, and the binaries are downloaded automatically when the project is compiled. Most code editors have an option to add a NuGet package reference to an existing project file. For example, in Visual Studio 2017 for Windows, you select Project -> Manage NuGet packages.

.NET Core 3.1 provides command-line tools for creating and editing project files. Using the command line, you can add a NuGet package reference to an existing project file with:

dotnet add package Microsoft.ML.Probabilistic
dotnet add package Microsoft.ML.Probabilistic.Compiler
dotnet add package Microsoft.ML.Probabilistic.Learners

Tutorials and Examples

There is a getting started guide on docs.microsoft.com.

More tutorials and examples can be found on the Infer.NET website.

Documentation

Documentation can be found on the Infer.NET website.

Structure of Repository

  • The Visual Studio solution Infer.sln in the root of the repository contains all Infer.NET components, unit tests and sample programs from the folders described below.

  • src/

    • Compiler contains the Infer.NET Compiler project which takes model descriptions written using the Infer.NET API, and converts them into inference code. The project also contains utility methods for visualization of the generated code.

    • Csoft is an experimental feature that allows to express probabilistic models in a subset of the C# language. You can find many unit tests of Csoft models in the Tests project marked with Category: CsoftModel trait.

    • Examples contains C# projects that illustrate how to use Infer.NET to solve a variety of different problems.

      • ClickThroughModel - a web search example of converting a sequence of clicks by the user into inferences about the relevance of documents.

      • ClinicalTrial - the clinical trial tutorial example with an interactive user interface.

      • InferNET101 - samples from Infer.NET 101 introduction to the basics of Microsoft Infer.NET programming.

      • ImageClassifier - an image search example of classifying tagged images.

      • LDA - this example provides Infer.NET implementations of the popular LDA model for topic modeling. The implementations pay special attention to scalability with respect to vocabulary size, and with respect to the number of documents. As such, they provide good examples for how to scale Infer.NET models in general.

      • MontyHall - an Infer.NET implementation of the Monty Hall problem, along with a graphical user interface.

      • MotifFinder - an Infer.NET implementation of a simple model for finding motifs in nucleotide sequences, which constitutes an important problem in bioinformatics.

    • FSharpWrapper is a wrapper project that hides some of the generic constructs in the Infer.NET API allowing simpler calls to the Infer.NET API from standard F#.

    • IronPythonWrapper contains wrapper for calling Infer.NET from the IronPython programming language and tests for the wrapper. Please refer to README.md for more information.

    • Learners folder contains Visual Studio projects for complete machine learning applications including classification and recommendation. You can read more about Learners here.

    • Runtime - is a C# project with classes and methods needed to execute the inference code.

    • Tutorials contains Examples Browser project with simple examples that provide a step-by-step introduction to Infer.NET.

    • Visualizers/Windows contains an alternative .NET Framework and Windows specific set of visualization tools for exploring and analyzing the code generated by the Compiler.

  • test/

    • TestApp contains C# console application for quick invocation and debugging of various Infer.NET components.

    • TestFSharp is an F# console project for smoke testing of Infer.NET F# wrapper.

    • TestPublic contains scenario tests for tutorial code. These tests are a part of the PR and nightly builds.

    • Tests - main unit test project containing thousands of tests. These tests are a part of the PR and nightly builds. The folder Tests\Vibes contains MATLab scripts that compare Infer.NET to the VIBES package. Running them requires Vibes2_0.jar (can be obtained on the VIBES website) to be present in the same folder.

    • Learners folder contains the unit tests and the test application for Learners (see above).

  • build folder contains the YAML definitions for the Continuous Integration builds and the specification files for the nuget packages.

  • docs folder contains the scripts for building API documentation and for updating https://dotnet.github.io/infer. Please refer to README.md for more details.

Building Infer.NET from its source code

Please, refer to our building guide.

Contributing

We welcome contributions! Please review our contribution guide.

When submitting pull request with changed or added factor, please make sure you updated factor documentation as described here.

License

Infer.NET is licensed under the MIT license.

.NET Foundation

Infer.NET is a .NET Foundation project. It's also a part of ML.NET machine learning framework.

There are many .NET related projects on GitHub.

  • .NET home repo - links to 100s of .NET projects, from Microsoft and the community.

infer's People

Contributors

0xflotus avatar a-vishar avatar anthonypuppo avatar bfdes avatar colcarroll avatar dependabot[bot] avatar dotnet-bot avatar johnguiver avatar jonathantims avatar jsboige avatar kant2002 avatar lazy avatar marodev avatar mikhailnikolaev avatar msdmkats avatar myshkov avatar philipjball avatar pocherninaelena avatar r-tinn avatar samwebster avatar sergey-b-berezin avatar terrajobst avatar thareshjose avatar thepiranha avatar tminka avatar yordanz avatar

Stargazers

avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar

Watchers

avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar

Forkers

edwardowusuadjei shyamalschandra colcarroll ifle jralexander rajeevyadav kalebka55 ademar danmoseley leispeed zeroyou dut3062796s awesomedotnetcore wangchengqun shaunstanislauslau h0kd33 awesomemachinelearning cgoss 520hacker kevinyzy aleksanderhan dl1109783 devhttps benzei wxjwz codeaudit ominux 0xflotus afcarl kenneychen rajpratim najielkotob briando2005 tolysz johnvonlzf lopage jangocheng lixin832500 jiaodaxiaozi gzcassan hanfeijp sathyaish globalhqw curtainwang ricklentz agbaezehenry sabahshariq shahilendra rlilojr nkhlken shewenhao woxihuanjia wangzhefeng2000 anke522 not-a-builder yubobo magically-solutions-india reloadbrain czcz1024 li3741 aisnnu qiuliliang thareshjose xhuvom steccami hbcbh1999 vishalbelsare zyril-8204 mbrukman 1ramkrishnan polytronicgr singhmousam kushalshm1 b-xiang mbyase alwaysyangzhijin sondro ukrcherry d3v3us tralivali1234 bosconi krishnand tobyxu mayase8 kartikeychandraa cephdon andy521 antonio-petricca maxprofs stjordanis jekpoint gongypgit sureshannapureddy narinkaew yasseraj cranzy a-sklyarov teegr wycharry b2220333

infer's Issues

Exception: Improper distribution during inference

Need some help.
InferNet_2019.zip

I converted the DifficultyAbility.cs code from C# to F#. But, I am getting the exception "Microsoft.ML.Probabilistic.Factors.ImproperMessageException: Improper distribution during inference (Gaussian(m/v=-0.01752, 1/v=0)). Cannot perform inference on this model"
The source code and executions results are attached.

Upgrade from .NET Core 2.0 to 2.1

.NET Core 2.0 went out of support Oct 1st. It probably makes sense to update to 2.1 (ie netcoreapp2.0->netcoreapp2.1 in the .csproj's and replace 2.0 with 2.1 in the README.md.

Everything should "just work" (assuming you install 2.1)

I can offer a PR if you like.

Initializing arrays of VectorGaussians

I've got a variable X which is an VariableArray of VectorGaussians. I'd like to initialize each element to a VectorGaussian with different (known) means. I've tried:

  1. Creating another variable which is a VariableArray of Vectors, observing it, and initializing each member of X accordingly using the range syntax (fails because VectorGaussian.FromMeanAndPrecision doesn't seem take a probabilistic mean argument).
  2. Using the ForEachBlock syntax to initialize using a C# array of Vectors (fails because the index is a Variable<int> and doesn't carry an observed value that can be used to index a normal array).
  3. Creating an Array of VectorGaussians with the desired means/covariances, and initializing X to it (fails because it seems as though you can't initialize a VariableArray with an Array).

Is there a simple way to do this? Thanks!

NuGet packages outdated

Hi guys,

the GaussianProcess examples I am working do not run with current NuGet build of infer (only run with source from GitHub). Please update the NuGet version, because it seems to be pretty outdated.

Best regards,
Christof

Matrix multiplication

Hello!

Is it possible to multiply variable of type Variable<PositiveDefiniteMatrix> with variable of type Variable<Vector>?
I can't find a way how to cast Variable<PositiveDefiniteMatrix> to Variable<Matrix> and use Variable.MatrixTimesVector.

Thanks!

Online learning with shared variables

The "Sharing variables between models" article in the user guide mentions that we could (should?) use Shared Variables in the common scenario of doing online inference when the data is not all available ahead of time.

Looking at the "Online Learning" article from the user guide, there's no mention of using Shared Variables at all. Instead of using Shared Variables, the sharing of the parameters of the model (called mean in this case) is done through a separate "message accumulator".

I'm confused about how to determine which approach to take. Generally, I can imagine 3 different ways of sharing parameters batch-to-batch:

  1. The naรฏve approach: define the model, infer the parameters (the shared ones as well) of the model and use the posterior of the parameters as the priors in the next batch. The "Online Learning" example does mention that this is not a valid approach to sharing parameters when the prior has a different type than the posterior.
    Why can't we infer the distribution of the variance variable or even the mean variable and use the posterior as the prior in the next batch in the case of the first example (Fig. 1) in the "Online Learning" article?

  2. Define the model with an "accumulator" (meanMessage) that stores the product of all messages sent to the connected variables, in this case mean.
    I'm confused about why we "update" the posterior (soon to be prior) of the mean variable by factoring in meanMessage into the posterior of mean with ConstrainEqualRandom? This seems like a roundabout way of doing the first option (inferring the posterior after the batch and using that as the prior in the next batch, with the difference that this stores the previous posterior temporarily in the accumulator until the end of the next batch)

  3. The way the first article describes it: define the parameters as Shared Variables, define the model using copies of the Shared Variables. The article does state that using this approach we need to iterate through the batches multiple times (passes), what makes this different than the above two approaches?

ArgumentException:โ€œvB is infiniteโ€

how did this exception happen?
The code is to infer skills of players.

            Range player = new Range(nPlayers);

            //2-D skill (player, game)
            //int[] playerGameNum = Util.ArrayInit(nPlayers, y => PlayerGameData[y].Count);
            var pPlayerGames = Variable.Observed(playerMatchCntData, player);
            Range playergame = new Range(pPlayerGames[player]);

            var priorSkill = Variable.Observed(BatchPlayerSkill, player);
            //var beforeSkill = Variable.Array<double>(player);
            var skill = Variable.Array(Variable.Array<double>(playergame), player);
            using (var playerBlock = Variable.ForEach(player))
            {
                var p = playerBlock.Index;
                //beforeSkill[player] = Variable<double>.Random(priorSkill[player]);
                using (var gameBlock = Variable.ForEach(playergame))
                {
                    var g = gameBlock.Index;
                    using (Variable.If(g == 0))
                    {
                        skill[player][playergame] = Variable<double>.Random(priorSkill[player]);
                    }
                    using (Variable.If(g > 0))
                    {
                        //skill[player][playergame] = Variable.GaussianFromMeanAndPrecision(skill[player][g - 1] + expOffset[g], skillChangeWithMatchPrecision);
                        skill[player][playergame] = Variable.GaussianFromMeanAndPrecision(skill[player][g - 1], skillChangeWithMatchPrecision);
                    }
                }
            }
            /*var beforeSkill = Variable.Array<double>(player);
            using (var playerBlock = Variable.ForEach(player))
            {
                var p = playerBlock.Index;
                beforeSkill[player] = Variable<double>.Random(priorSkill[player]);
                //beforeSkill[player].ObservedValue = BatchPlayerSkill[p.ObservedValue].GetMean();
            }*/

            Range game = new Range(nGames);
            int[] nMembersData = Util.ArrayInit(nGames, g => t1PlayerData[g].Length);
            var nMembers = Variable.Observed(nMembersData, game);
            Range member1 = new Range(nMembers[game]);

            var Team1Player = Variable.Observed(t1PlayerData, game, member1);
            var Team1Kill = Variable.Observed(t1KillData, game, member1);
            var Team1Death = Variable.Observed(t1DeathData, game, member1);
            var Team1GameIndex = Variable.Observed(t1GameIndexData, game, member1);
            var Team1Mvp = Variable.Observed(t1MvpData, game, member1);

            int[] nMembersData2 = Util.ArrayInit(nGames, g => t1PlayerData[g].Length);
            var nMembers2 = Variable.Observed(nMembersData2, game);
            Range member2 = new Range(nMembers2[game]);
            var Team2Player = Variable.Observed(t2PlayerData, game, member2);
            var Team2Kill = Variable.Observed(t2KillData, game, member2);
            var Team2Death = Variable.Observed(t2DeathData, game, member2);
            var Team2GameIndex = Variable.Observed(t2GameIndexData, game, member2);
            var Team2Mvp = Variable.Observed(t2MvpData, game, member2);

            var timePlayed = Variable.Observed(gameTimeData, game);
            var outcome = Variable.Observed(outcomeData, game);

            var Team1Perf = Variable.Array(Variable.Array<double>(member1), game);
            var Team2Perf = Variable.Array(Variable.Array<double>(member2), game);

            //var v_Team1Kill = Variable.Array(Variable.Array<double>(member1), game);
            //var v_Team2Kill = Variable.Array(Variable.Array<double>(member2), game);
            //var v_Team1Death = Variable.Array(Variable.Array<double>(member1), gm).Named("v_Team1Death");
            //var v_Team2Death = Variable.Array(Variable.Array<double>(member2), gm).Named("v_Team2Death");

            //var zeroIndex = Variable.New<int>();
            //zeroIndex.ObservedValue = 0;
            // Learn the skills from the data            
            using (Variable.ForEach(game))
            {
                using (var playerBlock = Variable.ForEach(member1))
                {
                    var PlayerIndex = Team1Player[game][member1];
                    var GameIndex = Team1GameIndex[game][member1];
                    //var Team1Perf = Variable.GaussianFromMeanAndPrecision(skill[PlayerIndex][GameIndex], performancePrecision);
                    var t1Perf = Variable.GaussianFromMeanAndPrecision(skill[PlayerIndex][GameIndex], performancePrecision);
                    Team1Perf[game][member1] = t1Perf;
                }

                Variable<double> Team1PerfSum = Variable.Sum(Team1Perf[game]);
                //Team1PredPerfSum[game] = Variable.Sum(Team1PredPerf[game]);

                using (var playerBlock = Variable.ForEach(member2))
                {
                    var PlayerIndex = Team2Player[game][member2];
                    var GameIndex = Team2GameIndex[game][member2];
                    //var Team2Perf = Variable.GaussianFromMeanAndPrecision(skill[PlayerIndex][GameIndex], performancePrecision);
                    var t2Perf = Variable.GaussianFromMeanAndPrecision(skill[PlayerIndex][GameIndex], performancePrecision);
                    Team2Perf[game][member2] = t2Perf;
                }
                Variable<double> Team2PerfSum = Variable.Sum(Team2Perf[game]);
                //Team2PredPerfSum[game] = Variable.Sum(Team2PredPerf[game]);

                using (var playerBlock = Variable.ForEach(member1))
                {
                    Team1Kill[game][member1] = Variable.Max(zeroVal, Variable.GaussianFromMeanAndPrecision((pKillW * Team1Perf[game][member1] + oKillW * (Team2PerfSum / squadSize)) * timePlayed[game], killV * timePlayed[game]));
                    Team1Death[game][member1] = Variable.Max(zeroVal, Variable.GaussianFromMeanAndPrecision((pDeathW * Team1Perf[game][member1] + oDeathW * (Team2PerfSum / squadSize)) * timePlayed[game], deathV * timePlayed[game]));
                    Team1Mvp[game][member1] = Variable.Max(zeroVal, Variable.GaussianFromMeanAndPrecision((pMvpW * Team1Perf[game][member1] + oMvpW * (Team2PerfSum / squadSize)) * timePlayed[game], mvpV * timePlayed[game]));
                }

                using (var playerBlock = Variable.ForEach(member2))
                {
                    Team2Kill[game][member2] = Variable.Max(zeroVal, Variable.GaussianFromMeanAndPrecision((pKillW * Team2Perf[game][member2] + oKillW * (Team1PerfSum / squadSize)) * timePlayed[game], killV * timePlayed[game]));
                    Team2Death[game][member2] = Variable.Max(zeroVal, Variable.GaussianFromMeanAndPrecision((pDeathW * Team2Perf[game][member2] + oDeathW * (Team1PerfSum / squadSize)) * timePlayed[game], deathV * timePlayed[game]));
                    Team2Mvp[game][member2] = Variable.Max(zeroVal, Variable.GaussianFromMeanAndPrecision((pMvpW * Team2Perf[game][member2] + oMvpW * (Team1PerfSum / squadSize)) * timePlayed[game], mvpV * timePlayed[game]));
                }

                Variable<double> white_delta = Team1PerfSum - Team2PerfSum;
                using (Variable.Case(outcome[game], 1))
                {
                    Variable.ConstrainTrue(white_delta > 0);
                }
                using (Variable.Case(outcome[game], 2))
                {
                    Variable.ConstrainTrue(white_delta < 0);
                }
            }

Ordinal factor analysis model

I'm hoping to implement a (probit) ordinal factor analysis model in Infer.NET. There were a few references on the old help board (from almost 10 years ago - see here) to the problem that were helpful. This is my model so far:

            // MODEL
            // Latent dimensions
            VariableArray2D<double> vX = Variable.Array<double>(rN, rP).Named("X");
            vX[rN, rP] = Variable.GaussianFromMeanAndPrecision(0.0, 1.0).ForEach(rN, rP);
            // Question weights
            VariableArray2D<double> vC = Variable.Array<double>(rP, rQ).Named("C");
            vC[rP, rQ] = Variable.GaussianFromMeanAndPrecision(0.0, 1.0).ForEach(rP, rQ);
            // Question 'errors'
            VariableArray<double> vR = Variable.Array<double>(rQ).Named("R");
            vR[rQ] = Variable.GammaFromShapeAndScale(1, 1).ForEach(rQ);
            // Continuous question states
            VariableArray2D<double> vY = Variable.Array<double>(rN, rQ).Named("Y");
            vY = Variable.MatrixMultiply(vX, vC);
            VariableArray2D<double> vYY = Variable.Array<double>(rN, rQ).Named("YY");
            vYY[rN, rQ] = Variable.GaussianFromMeanAndPrecision(vY[rN, rQ], vR[rQ]);
            // Thresholds
            VariableArray2D<double> lT = Variable.Array<double>(rQ, rR).Named("LT");
            VariableArray2D<double> uT = Variable.Array<double>(rQ, rR).Named("UT");
            // Observed question values
            var vZ = Variable.Array(Variable.Array(Variable.Array<bool>(rR), rQ), rN);
            vZ[rN][rQ][rR] = Variable.IsBetween(vYY[rN, rQ], lT[rQ, rR], uT[rQ, rR]);

            // INFERENCE
            InferenceEngine engine = new InferenceEngine();
            engine.Algorithm = new VariationalMessagePassing();
            lT.ObservedValue = lowerThresholds;
            uT.ObservedValue = upperThresholds;
            vZ.ObservedValue = observed;

            var XInferred = engine.Infer(vX);

The part that seems to be giving the inference engine trouble is the Variable.IsBetween. It errors with: TruncatedGaussian requires isBetween=true. I have a hunch this might be a limitation of the VMP implementation (to do with non-conjugacy at the IsBetween node?) - I had hoped to use EP to fit the model, but the MatrixMultiply factor doesn't seem to be supported. Any pointers would be much appreciated! And sorry if any of this is basic, I'm very new to both Infer.NET and .NET in general. Thanks for a great piece of software.

ExecutionEngineException in Unity3d IL2CPP iOS

Running code using Infer.NET fails with an ExecutionEngineException on iOS when built with IL2CPP from Unity3d. Some of the instantiations of generic types used internally don't exist during compile time which causes the corresponding iOS AOT code to be missing at runtime.

Details:
Unity 2018.3
Microsoft.ML.Probabilistic.dll
Microsoft.ML.Probabilistic.Compiler.dll

ExecutionEngineException: Attempting to call method 'Microsoft.ML.Probabilistic.Models.ModelBuilder::SearchExpression<System.Boolean>' for which no ahead of time (AOT) code was generated.
  at System.Reflection.MonoMethod.Invoke (System.Object obj, System.Reflection.BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[] parameters, System.Globalization.CultureInfo culture) [0x00000] in <00000000000000000000000000000000>:0 
  at Microsoft.ML.Probabilistic.Utilities.Util.Invoke (System.Reflection.MethodBase method, System.Object target, System.Object[] parameters) [0x00000] in <00000000000000000000000000000000>:0 
  at Microsoft.ML.Probabilistic.Models.ModelBuilder.Build (Microsoft.ML.Probabilistic.Models.InferenceEngine engine, System.Boolean inferOnlySpecifiedVars, System.Collections.Generic.IEnumerable`1[T] vars) [0x00000] in <00000000000000000000000000000000>:0 
  at Microsoft.ML.Probabilistic.Models.InferenceEngine.BuildAndCompile (System.Boolean inferOnlySpecifiedVars, System.Collections.Generic.IEnumerable`1[T] vars) [0x00000] in <00000000000000000000000000000000>:0 
  at Microsoft.ML.Probabilistic.Models.InferenceEngine.GetCompiledInferenceAlgorithm (System.Boolean inferOnlySpecifiedVars, Microsoft.ML.Probabilistic.Models.IVariable var) [0x00000] in <00000000000000000000000000000000>:0 
  at Microsoft.ML.Probabilistic.Models.InferenceEngine.Infer (Microsoft.ML.Probabilistic.Models.IVariable var) [0x00000] in <00000000000000000000000000000000>:0 
  at Main.TwoCoins () [0x00000] in <00000000000000000000000000000000>:0 
  at Main.Start () [0x00000] in <00000000000000000000000000000000>:0 
--- End of stack trace from previous location where exception was thrown ---
  at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw () [0x00000] in <00000000000000000000000000000000>:0 
  at Microsoft.ML.Probabilistic.Utilities.Util.Invoke (System.Reflection.MethodBase method, System.Object target, System.Object[] parameters) [0x00000] in <00000000000000000000000000000000>:0 
  at Microsoft.ML.Probabilistic.Models.ModelBuilder.Build (Microsoft.ML.Probabilistic.Models.InferenceEngine engine, System.Boolean inferOnlySpecifiedVars, System.Collections.Generic.IEnumerable`1[T] vars) [0x00000] in <00000000000000000000000000000000>:0 
  at Microsoft.ML.Probabilistic.Models.InferenceEngine.BuildAndCompile (System.Boolean inferOnlySpecifiedVars, System.Collections.Generic.IEnumerable`1[T] vars) [0x00000] in <00000000000000000000000000000000>:0 
  at Microsoft.ML.Probabilistic.Models.InferenceEngine.GetCompiledInferenceAlgorithm (System.Boolean inferOnlySpecifiedVars, Microsoft.ML.Probabilistic.Models.IVariable var) [0x00000] in <00000000000000000000000000000000>:0 
  at Microsoft.ML.Probabilistic.Models.InferenceEngine.Infer (Microsoft.ML.Probabilistic.Models.IVariable var) [0x00000] in <00000000000000000000000000000000>:0 
  at Main.TwoCoins () [0x00000] in <00000000000000000000000000000000>:0 
  at Main.Start () [0x00000] in <00000000000000000000000000000000>:0 
 
(Filename: currently not available on il2cpp Line: -1)```

Strings Features for BayesPointMachineClassifier

I tried to made a model able to predict a string through a list of number and strings, when I tried to give a object for the mapping I canยดt because it only recibe a Vector for the feature.

IClassifierMapping<IList<Vector>, int, IList, string, Vector>

Then I made a dictionary for all the strings in the features, and take the index of each words to pass the value in the vector, but when the model was made the results allways are the same, no matter if I change the parammeters.

If I used the ML.net library with the same data and a Stochastic model, I recibe corrects values for the parammeters.

Do you have some example for strings features to the BayesPointMachineClassifier or is not possible for this kind of models take strigns.

Unable to install package Microsoft.ML.Probabilistic in my Unity3d Project using .net version 4.6

I am Unable to install package Microsoft.ML.Probabilistic for the Infer.NET framework in my Unity project. I am trying to install it by using Project > Manage Nuget Packages option in my visual studio.
Both .net and the infer.net package are Microsoft products, and yet they have not released a compatible version. When I click on install I get the following error stating that the package is not compatible with .net version 4.6:

Could not install package 'Microsoft.ML.Probabilistic 0.3.1810.501'.
You are trying to install this package into a project that targets
'.NETFramework,Version=v4.6', but the package does not contain any
assembly references or content files that are compatible with that
framework. For more information, contact the package author.

enter image description here

Below is a screenshot of my Unity3d Edit > Project Settings > Player inspector screen:
enter image description here

GaussianProductOp.ProductAverageConditional - has quality band Experimental which is less than the recommended quality band (Preview)

I have a quite simple rule and I get already the quality band issue. Can someone explain why this is happening? I have read the docs about Quality bands but I don't understand it.

This is the code I wrote that produces the warning:

Gaussian BornYoungPerLitterDist = new Gaussian(5.5, 0);
Gaussian BirthRateDist = new Gaussian(0.004102103451, 0.00000144);

Variable<double> BornYoungPerLitter = Variable.Random<double, Gaussian>(BornYoungPerLitterDist);
Variable<double> Birthrate = Variable.Random<double, Gaussian>(BirthRateDist);

InferenceEngine engine = new InferenceEngine();
Gaussian Born = engine.Infer<Gaussian>(BornYoungPerLitter * Birthrate);
Console.WriteLine(
    "Born Mean: {0:f10}, Standard Deviation: {1:f10}",
    Born.GetMean(),
    Math.Sqrt(Born.GetVariance())
);

the warning is:

Compiling model...compilation had 1 warning(s).
  [1] GaussianProductOp.ProductAverageConditional(vdouble2_use_B, vGaussian0, vGaussian1) has quality band Experimental which is less than the recommended quality band (Preview)

Can't compile with mono 

[bdeonovic@argon-login-2 test_mono]$ csc FirstExample.cs -reference:Microsoft.ML.Probabilistic.0.3.1810.501/lib/netstandard2.0/Microsoft.ML.Probabilistic.dll
Microsoft (R) Visual C# Compiler version 2.8.2.62916 (2ad4aabc)
Copyright (C) Microsoft Corporation. All rights reserved.

FirstExample.cs(2,34): error CS0234: The type or namespace name 'Models' does not exist in the namespace 'Microsoft.ML.Probabilistic' (are you missing an assembly reference?)
using System;
using Microsoft.ML.Probabilistic.Models;

public class FirstExample
{
  static public void Main()
  {
    Console.WriteLine ("Hello Mono World");
    Variable<bool> firstCoin = Variable.Bernoulli(0.5).Named("firstCoin");
    Variable<bool> secondCoin = Variable.Bernoulli(0.5).Named("secondCoin");
    Variable<bool> bothHeads = (firstCoin & secondCoin).Named("bothHeads");
    InferenceEngine engine = new InferenceEngine();
    if (engine.Algorithm is Algorithms.VariationalMessagePassing)
    {
      Console.WriteLine("This example does not run with Variational Message Passing");
      return;
    }
    Console.WriteLine("Probability both coins are heads: " + engine.Infer(bothHeads));
    bothHeads.ObservedValue = false;
    Console.WriteLine("Probability distribution over firstCoin: " + engine.Infer(firstCoin));
  }
}

Consistency between user guide on inferring Beta distribution parameters and Skills example

The FAQ advises

Can I learn the parameters of a Beta distribution, or the shape parameter of a Gamma distribution?

No. The inference algorithms in Infer.NET can theoretically learn these parameters, but only at great expense. Therefore Infer.NET doesnโ€™t include these operations..

But in the Student Skills example, unless I am misunderstanding, it appears to be inferring the parameters of a Beta distribution.

Issues found by PVS-Studio

Hello,
Developers of PVS-Studio static analyzer present their check report of the source code of 'Infer.NET' in the article, containing the review of the most suspicious code fragments they discovered.

You can read article at the official site:
(External link redacted)

Best regards,
Sergey Vasiliev

Select not found

Net core 2.1
"Select(v => Dirichlet.PointMass(v))" report
error CS1061: 'Vector[]' does not contain a definition for 'Select' and no accessible extension method 'Select' accepting a first argument of type 'Vector[]' could be found

Some FSharpWrapper operators conflict with built-in F# operators

Operators >> and << defined in src\FSharpWrapper\FSharpWrapper.fs
conflict with built-in F# function composition operators.

Example. The following code:

open Microsoft.ML.Probabilistic.FSharp

let f2 x = x * 2
let f3 x = x * 3
let f6 = f2 >> f3

results in the compiler error:

error FS0001: Expecting a type supporting the operator '>' but given a function
type. You may be missing an argument to a function.

If we comment out open Microsoft.ML.Probabilistic.FSharp then it compiles.
But then it is harder to work with F# wrapper features, especially operators.

If the choice of operators was not an intentional design decision with
awareness of the conflicts, can these operators be changed?

the exception

Microsoft.ML.Probabilistic.Models.InferenceEngineโ€็š„็ฑปๅž‹ๅˆๅง‹ๅ€ผ่ฎพๅฎš้กนๅผ•ๅ‘ๅผ‚ๅธธ
ReflectionTypeLoadException: ๆ— ๆณ•ๅŠ ่ฝฝไธ€ไธชๆˆ–ๅคšไธช่ฏทๆฑ‚็š„็ฑปๅž‹ใ€‚ๆœ‰ๅ…ณๆ›ดๅคšไฟกๆฏ๏ผŒ่ฏทๆฃ€็ดข LoaderExceptions ๅฑžๆ€ง

what that mean?

using the subarray in self assessment 2.4 of MBML handbook

I've implemented the skills model not using the Subarray factor. Instead I created hasSkills factors using a for loop and indices from the skills table (i.e., I didn't observe the indices). I would like to know the following:
1.) what the implications of my approach is. For example, is it perhaps slower than the Subarray approach?
2.) is there a way to set priors for Skills[People][QuestionsSkills] = Variable.Bernoulli(probabilityOfSkillTrue[QuestionsSkills]).ForEach(People); without having to use ObservedValue for probabilityOfSkillTrue.

My results seem to match those presented in Fig.2.18(a). Is there an easy way to implement the Subarray in my code:

 int[][] skillsIndicesForEachQuestion = new int[48][] {
                new int [] { 0 },
                new int [] { 0 },
                new int [] { 0, 6 },
                new int [] { 0, 6 },
                new int [] { 0 },
                new int [] { 0 },
                new int [] { 0, 6 },
                new int [] { 1, 6 },
                new int [] { 1, 6 },
                new int [] { 1 },
                new int [] { 1 },
                new int [] { 1, 6 },
                new int [] { 1, 6 },
                new int [] { 1, 6 },
                new int [] { 1 },
                new int [] { 1, 6 },
                new int [] { 1, 6 },
                new int [] { 1 },
                new int [] { 2 },
                new int [] { 2 },
                new int [] { 2 },
                new int [] { 2 },
                new int [] { 2 },
                new int [] { 0 },
                new int [] { 0 },
                new int [] { 0 },
                new int [] { 3 },
                new int [] { 3 },
                new int [] { 3 },
                new int [] { 3 },
                new int [] { 3 },
                new int [] { 3 },
                new int [] { 3 },
                new int [] { 3 },
                new int [] { 4 },
                new int [] { 1, 4 },
                new int [] { 4 },
                new int [] { 4 },
                new int [] { 4 },
                new int [] { 4 },
                new int [] { 5 },
                new int [] { 5 },
                new int [] { 5 },
                new int [] { 5 },
                new int [] { 5 },
                new int [] { 5 },
                new int [] { 5 },
                new int [] { 5 }
            };

        bool[][] isCorrectAnswers = new bool[22][]
            {
                new bool [] { true, true, true, true, true, false, true, true, true,
                  true, true, true, true, true, true, true, true, true,
                  true, true, true, false, true, true, true, true, true,
                  true, true, false, false, true, true, true, true, true,
                  true, true, true, true, true, true, true, true, true,
                  true, true, true },
                new bool [] { false, true, true, true, true, true, true, true, true,
                  true, true, false, true, true, true, false, false, true,
                  true, true, true, false, false, true, true, true, true,
                  true, true, false, true, false, true, true, false, true,
                  false, true, false, true, false, true, false, false, false,
                  false, false, true },
                new bool [] { false, true, false, false, true, false, true, true, true,
                  true, false, false, false, false, false, true, true, false,
                  true, true, false, false, false, false, false, true, true,
                  true, true, false, true, false, true, true, true, false,
                  false, false, true, true, true, true, true, true, false,
                  false, false, true },
                new bool [] { true, true, true, true, true, false, true, true, true,
                  true, true, true, false, true, true, true, true, true,
                  true, true, false, false, true, true, true, true, true,
                  true, true, false, true, true, false, true, false, false,
                  true, true, true, false, true, true, true, true, false,
                  true, false, true },
                new bool [] { true, true, true, true, true, true, true, true, true,
                  true, true, true, true, true, true, true, true, true,
                  true, true, true, false, true, true, true, true, true,
                  true, true, false, false, false, true, true, true, true,
                  true, true, true, true, true, true, true, true, false,
                  true, true, false },
                new bool [] { false, false, true, false, false, false, false, false, false,
                  false, true, false, false, false, false, false, false, false,
                  false, false, false, false, false, true, false, false, false,
                  false, false, true, false, false, false, false, false, false,
                  false, true, false, false, false, false, false, false, false,
                  false, false, false },
                new bool [] { true, true, true, true, true, true, true, true, true,
                  true, true, true, true, false, true, true, true, true,
                  true, true, true, false, true, true, true, true, true,
                  true, true, false, false, true, true, true, false, true,
                  true, true, true, true, true, true, true, true, true,
                  true, true, true },
                new bool [] { true, true, false, false, true, true, true, true, true,
                  true, true, true, true, true, true, true, true, true,
                  true, true, true, true, true, true, true, true, true,
                  true, true, false, false, false, true, true, true, true,
                  false, false, true, false, true, true, true, true, false,
                  true, true, true },
                new bool [] { true, true, false, true, true, true, true, true, true,
                  true, true, true, true, true, true, true, true, true,
                  true, true, true, false, true, false, true, false, true,
                  true, true, false, true, true, true, true, true, true,
                  true, true, true, true, false, true, true, true, true,
                  true, true, true },
                new bool [] { true, true, true, true, true, true, true, true, true,
                  true, true, true, false, true, true, true, true, true,
                  true, false, false, false, true, true, true, true, true,
                  true, true, false, true, true, true, true, true, true,
                  true, true, true, true, true, true, true, true, true,
                  false, false, true },
                new bool [] { false, true, true, true, true, false, true, true, true,
                  true, false, true, true, true, true, true, true, true,
                  true, true, true, false, false, true, true, true, true,
                  true, true, false, true, true, true, true, true, false,
                  true, true, true, false, true, true, true, false, true,
                  false, true, false},
                new bool [] { false, false, false, true, true, true, true, true, true,
                  true, true, false, false, false, false, true, false, true,
                  true, true, true, false, true, true, true, true, true,
                  true, true, false, false, false, false, false, false, true,
                  true, true, true, true, true, false, false, false, true,
                  true, false, false},
                new bool [] { true, true, true, true, true, false, true, true, true,
                  true, true, true, true, false, true, true, true, true,
                  true, true, true, false, true, true, true, true, true,
                  true, true, false, false, true, true, true, true, false,
                  true, true, true, false, true, true, true, true, true,
                  true, false, true},
                new bool [] { true, false, true, true, false, false, true, true, true,
                  true, true, true, false, false, true, true, true, true,
                  true, true, false, true, true, false, true, true, true,
                  true, true, false, true, false, true, true, true, true,
                  false, true, false, false, true, true, true, true, false,
                  true, false, false },
                new bool [] { true, true, true, true, true, false, true, true, true,
                  true, false, false, true, false, true, false, true, true,
                  true, false, true, false, true, false, true, true, true,
                  true, true, false, false, true, true, true, false, true,
                  true, true, true, false, true, true, true, true, true,
                  true, false, true },
                new bool [] { true, true, true, false, true, true, true, true, true,
                  true, true, true, true, true, true, true, true, true,
                  true, true, true, false, false, true, true, true, true,
                  true, true, true, true, true, true, true, false, true,
                  true, true, true, false, true, true, true, true, true,
                  true, true, true },
                new bool [] { true, true, true, true, true, true, true, true, true,
                  true, true, true, true, true, true, true, true, true,
                  false, true, true, false, true, true, true, true, true,
                  true, true, false, false, true, true, true, true, true,
                  true, true, true, false, true, true, true, true, true,
                  true, false, true },
                new bool [] { true, true, true, true, true, false, true, true, true,
                  true, true, true, true, true, true, true, true, true,
                  true, true, true, false, true, true, true, true, true,
                  true, true, false, false, true, true, true, true, true,
                  false, true, true, true, true, true, true, false, true,
                  true, false, false },
                new bool [] { true, true, false, true, true, false, true, false, true,
                  true, true, false, true, true, true, true, true, true,
                  true, true, true, false, true, true, true, true, true,
                  true, true, false, true, true, true, true, false, true,
                  true, true, true, false, true, true, true, true, false,
                  false, false, true },
                new bool [] { true, true, true, true, true, true, true, true, true,
                  true, true, true, true, true, true, true, true, true,
                  true, true, true, true, true, true, true, true, true,
                  true, true, false, true, true, true, true, false, true,
                  true, true, true, true, true, true, true, true, true,
                  true, true, true },
                new bool [] { true, true, true, true, true, true, false, true, true,
                  true, true, true, true, true, true, true, true, true,
                  true, true, true, false, true, true, true, true, true,
                  true, true, false, true, true, true, true, true, true,
                  true, true, true, true, true, true, true, true, true,
                  true, true, true },
                new bool [] { true, true, false, true, true, true, true, true, true,
                  true, true, true, true, false, true, true, true, false,
                  true, true, true, false, false, true, true, true, true,
                  true, true, false, true, true, true, true, false, true,
                  true, true, true, true, true, true, true, true, true,
                  true, true, true } };

            int numSkills = 7;
            int numQuestions = 48;
            int numPeople = 22;

            Range QuestionsSkills = new Range(numSkills);
            Range Questions = new Range(numQuestions);
            Range People = new Range(numPeople);

            VariableArray<double> probabilityOfSkillTrue = Variable.Array<double>(QuestionsSkills).Named("probabilityOfSkillTrue");

            VariableArray<VariableArray<bool>, bool[][]> Skills = Variable.Array(Variable.Array<bool>(QuestionsSkills), People).Named("Skills");
            Skills[People][QuestionsSkills] = Variable.Bernoulli(probabilityOfSkillTrue[QuestionsSkills]).ForEach(People);

            VariableArray<VariableArray<bool>, bool[][]> HasSkills = Variable.Array(Variable.Array<bool>(Questions), People).Named("HasSkills");

            for (int p = 0; p < numPeople; p++)
            {
                for (int q = 0; q < numQuestions; q++)
                {
                    if (skillsIndicesForEachQuestion[q].Length > 1)
                    {
                        var skill1 = skillsIndicesForEachQuestion[q][0];
                        var skill2 = skillsIndicesForEachQuestion[q][1];
                        HasSkills[p][q] = ((Skills[p][skill1] & Skills[p][skill2]) == Variable.Bernoulli(1));

                    }
                    else
                    {
                        var skill1 = skillsIndicesForEachQuestion[q][0];
                        HasSkills[p][q] = Skills[p][skill1];
                    }
                }
            }

            VariableArray<VariableArray<bool>, bool[][]> isCorrect = Variable.Array(Variable.Array<bool>(Questions), People).Named("isCorrect");

            using (Variable.ForEach(People))
            {
                using (Variable.ForEach(Questions))
                {
                    // add noise factor for has skills
                    using (Variable.If(HasSkills[People][Questions]))
                    {
                        isCorrect[People][Questions].SetTo(Variable.Bernoulli(0.9));
                    }
                    using (Variable.IfNot(HasSkills[People][Questions]))
                    {
                        isCorrect[People][Questions].SetTo(Variable.Bernoulli(0.2));
                    }
                }
            }

            probabilityOfSkillTrue.ObservedValue = Enumerable.Repeat(0.5, numSkills).ToArray();

            isCorrect.ObservedValue = isCorrectAnswers;

            var engine = new InferenceEngine();
            engine.Algorithm = new ExpectationPropagation();

            Bernoulli[][] SkillsPosteriors = engine.Infer<Bernoulli[][]>(Skills);

            for (int p = 0; p < numPeople; p++)
            {
                Console.WriteLine("Person {0} skills distribution: ", p + 1);
                for (int s = 0; s < numSkills; s++)
                {
                    Console.WriteLine(SkillsPosteriors[p][s]);
                }
                Console.WriteLine("------------------");
            }

Strange unsupported model with EP, because of Factor.GaussianFromMeanAndVariance()

First, the problem I'm trying to model:

Suppose you have a physical experiment for which you are using k identical measuring devices. Each device is exactly the same (same vendor, same model, etc.) and they have the same precision with which they can measure a particular aspect of an experiment.

Focusing on just one of these devices (because you have k identical ones) we know the following:

  • During the experiment the device measures the unknown quantity n times
  • The experiment is not stationary; i.e.: each measurement is not distributed around the same stationary (prior) mean, but rather each measurement's expected value is distributed around the value of the unknown quantity in the previous timestep t-1
  • Because of the nature of the experiment, the first measurement is the most accurate and as time passes between the subsequent measurements the device gets more and more inaccurate.
  • Suppose that the measured value of the device is a Gaussian distribution around the actual value with an unknown variance/precision.
  • The task is to find the variance of the measurements (which is the intrinsic inaccuracy of the devices).

Here's the model for just one measuring device:

// model
Variable<int> nItems = Variable.New<int>().Named("nItems");
Range item = new Range(nItems).Named("item");

var meanPrior = Variable.New<Gaussian>();

var variancePrior = Variable.New<Gamma>();
var variance = Variable<double>.Random(variancePrior);
variance.AddAttribute(new PointEstimate());
var time = Variable.Array<double>(item);

var measurements = Variable.Array<double>(item).Named("measurements");


using (var itemBlock = Variable.ForEach(item))
{
    using (Variable.If(itemBlock.Index == 0))
    {
        measurements[item] = Variable<double>.Random(meanPrior);
    }

    using (Variable.If(itemBlock.Index > 0))
    {
        measurements[item] = Variable.GaussianFromMeanAndVariance(measurements[itemBlock.Index - 1], variance * time[item]);
    }
}

// engine
var inferenceEngine = new InferenceEngine();

// observations
nItems.ObservedValue = 5;

meanPrior.ObservedValue = Gaussian.FromMeanAndVariance(0, 1);
variancePrior.ObservedValue = Gamma.FromShapeAndRate(1, 10);
measurements.ObservedValue = new[] {1, 1.4, 0.6, 0.5, 0.7};
time.ObservedValue = new[] {0, 1.4, 2.1, 1.15, 1.4};

var inferred = inferenceEngine.Infer<Gamma>(variance);
Console.WriteLine(inferred);

Notes:
This is a minimal model that shows what I'm trying to do, in my actual code there are N devices and the inference is running over batches of measurements and I'm trying to learn the variance by inferring the variance after each batch and using that posterior as the prior for the variance in the next batch.

The problem I'm facing, but only in my original model (NOT in the one where I'm only modelling one device, posted above) is that I'm getting a MessageTransform error that reads:

Error 0: This model is not supported with ExpectationPropagation due to Factor.GaussianFromMeanAndVariance(double sample, double mean, double variance). Try using a different algorithm or expressing the model differently in
Factor.GaussianFromMeanAndVariance(measurements_allDevices_itemMinus1_0_, vdouble3)
Details: Looking for varianceAverageConditional(sample=Microsoft.ML.Probabilistic.Distributions.Gaussian)(to_sample=Microsoft.ML.Probabilistic.Distributions.Gaussian)(mean=Microsoft.ML.Probabilistic.Distributions.Gaussian)(to_mean=Microsoft.ML.Probabilistic.Distributions.Gaussian)(variance=Microsoft.ML.Probabilistic.Distributions.Gamma)(to_variance=Microsoft.ML.Probabilistic.Distributions.Gamma)(result=Microsoft.ML.Probabilistic.Distributions.Gamma)(sample=True)(mean=False)(variance=False), found:
GaussianFromMeanAndVarianceOp.VarianceAverageConditional(Gaussian sample, Gaussian mean, Gamma variance)
GaussianFromMeanAndVarianceOp_PointVariance.VarianceAverageConditional(Gaussian sample, Gaussian mean, Gamma variance)

The relevant lines from the original model (which I'm not posting here even though the toy posted above doesn't produce the error message, because it's fairly big, distributed in multiple files, there are other variables which are derived from these measurements, etc.) are as follows:

using (Variable.ForEach(allDevices))
{
    using (var itemBlock = Variable.ForEach(item))
    {
        using (Variable.If(itemBlock.Index == 0))
        {
            measurements[allDevices][itemBlock.Index] = Variable<double>.Random(meanPrior);
        }

        using (Variable.If(itemBlock.Index > 0))
        {
            measurements[allDevices][itemBlock.Index] = Variable.GaussianFromMeanAndVariance(measurements[allDevices][itemBlock.Index - 1], variance * time[allDevices][item]);
        }
    }
}

allDevices: range for enumerating all devices
item: range for enumerating all measurements for a given device
meanPrior: A Gaussian distribution
variance: the variance around the measurement made by the device, a Gamma distribution
time: a 2D array (Variable.Array(Variable.Array<double>(item), allDevices)) that holds the amount of time (in seconds) passed since the last measurement made by a particular device.

I'm really baffled why I'm getting an error message in the original model when I'm trying to create a Gaussian distributed variable from a Gaussian mean and a Gamma distributed variance prior (times a double). It's super confusing that when I tried to whittle away the (seemingly) unrelevant parts (like reading the data from file, just modelling one device, etc.) the error message went away, but when I compare the variables they are defined in the same way in both.

F# builders for simpler stochastic block syntax (DSL)

F# builders for simpler stochastic block syntax (DSL)

I would like to share some helpers used in my F# scripts, either as a
suggestion for Infer.NET FSharpWrapper.fs or just a tip for F# folks.

The below F# module defines block builders for stochastic
If, IfNot, Case, Switch, ForEach, Repeat.

The module

[<AutoOpen>]
module InferNetBlocks
open Microsoft.ML.Probabilistic.Models

/// The base builder literally wrapping a code block.
/// Derived builders have custom overrides, e.g. Run.
type BlockBuilder () =
    member __.Delay (f) =
        f
    member __.Run (f) =
        f ()
    member __.Zero () =
        ()
    member __.Return (x) =
        x
    member __.Combine (a, b) =
        b a
    member __.For (sequence, body) =
        for item in sequence do
            body item
    member __.While (guard, body) =
        while guard () do
            body ()
    member __.TryWith (body, handler) =
        try body ()
        with exn -> handler exn
    member __.TryFinally (body, compensation) =
        try body ()
        finally compensation ()
    member __.Using (disposable:#System.IDisposable, body) =
        try body disposable
        finally if not (isNull disposable) then disposable.Dispose ()

/// Creates a stochastic if statement when the variable is true.
/// Example: If variable { ... }
type If (variable) =
    inherit BlockBuilder ()
    member __.Run (f) =
        use __ = Variable.If variable
        f ()

/// Creates a stochastic if statement when the variable is false.
/// Example: IfNot variable { ... }
type IfNot (variable) =
    inherit BlockBuilder ()
    member __.Run (f) =
        use __ = Variable.IfNot variable
        f ()

/// Creates a stochastic case statement when the integer variable has the specified value.
/// Example: Case (variable, value) { ... }
type Case (variable, value) =
    inherit BlockBuilder ()
    member __.Run (f) =
        use __ = Variable.Case (variable, value)
        f ()

/// Creates a stochastic switch statement using the specified condition variable.
/// Example: Switch variable { ... }
type Switch (variable) =
    inherit BlockBuilder ()
    member __.Run (f) =
        use __ = Variable.Switch variable
        f ()

/// Creates a 'for each' block.
/// Example: ForEach range { ... }
type ForEach (range) =
    inherit BlockBuilder ()
    member __.Run (f) =
        use __ = Variable.ForEach range
        f ()

/// Creates a 'repeat' block.
/// Example: Repeat count { ... }
type Repeat (count) =
    inherit BlockBuilder ()
    member __.Run (f) =
        use __ = Variable.Repeat count
        f ()

Examples with stochastic blocks

From my version of causality example (ForEach, If, IfNot):

// Loop over the data points
ForEach N {
    If doB.[N] {
        B.[N] <- Variable.Bernoulli probBIntervention
    }
}

// First model: A causes B
If AcausesB {
    ForEach N {
        A.[N] <- Variable.Bernoulli 0.5
        IfNot doB.[N] {
            B.[N] <- Variable.NotEqual (A.[N], Variable.Bernoulli q)
        }
    }
}

// Second model: B causes A
IfNot AcausesB {
    ForEach N {
        IfNot doB.[N] {
            B.[N] <- Variable.Bernoulli 0.5
        }
        A.[N] <- Variable.NotEqual (B.[N], Variable.Bernoulli q)
    }
}

From my version of Gaussian mixture example (ForEach, Switch):

ForEach n {
    z.[n] <- Variable.Discrete weights
    Switch z.[n] {
        data.[n] <- Variable.VectorGaussianFromMeanAndPrecision (means.[z.[n]], precs.[z.[n]])
    }
}

From my version of "A Murder Mystery" (Case):

Case (murderer, Major) {
    weapon.SetTo (Variable.Discrete (0.1, 0.9))
    hair.SetTo (Variable.Bernoulli 0.5)
}
Case (murderer, Miss) {
    weapon.SetTo (Variable.Discrete (0.8, 0.2))
    hair.SetTo (Variable.Bernoulli 0.05)
}

Error when running inference using softmax observed values

Hi!

I am trying to model observed softmax data using Infer.Net and I get an error no matter which algorithm I use. I have a more complicated model myself, but I have tried to reduce the issue to a simplified Gaussian-softmax example model below:

    class Program
    {
        static void Main(string[] args)
        {
            var testData = new Vector[]
            {
                Vector.FromArray(0.1, 0.3, 0.5, 0.1),
                Vector.FromArray(0.05, 0.5, 0.2, 0.25),
                Vector.FromArray(0.2, 0.4, 0.3, 0.1),
                Vector.FromArray(0.15, 0.2, 0.35, 0.3),
                Vector.FromArray(0.15, 0.3, 0.4, 0.15)
            };
            Model(testData);
            Console.ReadLine();
        }

        static void Model(Vector[] data)
        {
            var priorMean = Variable.VectorGaussianFromMeanAndPrecision(Vector.Constant(4, 0.0), PositiveDefiniteMatrix.Identity(4)).Named("priorMean");
            var priorCov = Variable.WishartFromShapeAndRate(4.0, PositiveDefiniteMatrix.Identity(4)).Named("priorCov");
            var prior = Variable.VectorGaussianFromMeanAndPrecision(priorMean, priorCov).Named("prior");
            var numDocs = Variable.New<int>().Named("numDocs");
            var docR = new Range(numDocs);
            var arrvals = Variable.Array<Vector>(docR).Named("arrvals");
            arrvals[docR] = Variable.Softmax(prior).ForEach(docR);

            // Observations
            numDocs.ObservedValue = data.Length;
            arrvals.ObservedValue = data;

            // var alg = new ExpectationPropagation(); 
            // var alg = new VariationalMessagePassing();
            var alg = new GibbsSampling();
            var ieng = new InferenceEngine(alg);
            var compAlg = ieng.GetCompiledInferenceAlgorithm(priorMean, priorCov);
            Console.WriteLine(compAlg.Marginal(priorMean.NameInGeneratedCode));
        }
    }

The errors I get for each individual algorithm are as follows.
For expectation propagation and Gibbs sampling, I get that the model is unsupported because of MMath.Softmax. E.g., the exception for Gibbs sampling:

Error 0: This model is not supported with GibbsSampling due to MMath.Softmax(Vector softmax, IList<double> x). Try using a different algorithm or expressing the model differently Gibbs Sampling requires the conditionals to be conjugate in
MMath.Softmax(prior_rep_uses[0][index0])
Details: System.MissingMethodException: xAverageConditional not found in SoftmaxOp_Bouchard_Sparse,SoftmaxOp_Bouchard,SoftmaxOp_BL06_LBFGS,SoftmaxOp_Bohning,SoftmaxOp_Taylor,SoftmaxOp_KM11,SoftmaxOp_KM11_Sparse2,SoftmaxOp_KM11_Sparse,SoftmaxOp_BL06,VectorSoftmaxOp_KM11,SoftmaxOp_KM11_LBFGS,SoftmaxOp_KM11_LBFGS_Sparse,GammaSoftmaxOp using parameter types: [softmax] Vector,[x] PlaceHolder,[to_x] VectorGaussian,[result] VectorGaussian

For VMP, the model succesfully compiles, but I get a null-pointer exception:

Exception thrown: 'System.NullReferenceException' in Microsoft.ML.Probabilistic.dll
An unhandled exception of type 'System.NullReferenceException' occurred in Microsoft.ML.Probabilistic.dll
Object reference not set to an instance of an object.

Any idea of whether I am misspecifying something here, or there is a workaround to solve the issue?

Thank you very much in advance!

Package install error

I use this command:
Install-Package Microsoft.ML.Probabilistic -Version 0.3.1912.403

but get this message:
Could not install package 'Microsoft.ML.Probabilistic 0.3.1912.403'. You are trying to install this package into a project that targets '.NETFramework,Version=v4.6.1', but the package does not contain any assembly references or content files that are compatible with that framework.

I tried changing to .NETFramework 4.5.1, 4.6, 4.6.1, and 4.7.2 but nothing work
Help.

MatrixMultiply factor with fixed output

Hello!

I need help or some explanation of error.
Well, I have a graph described by following code:

public void Reset(int mapSize)
{
    var m = new Range(4);
    var n = m.Clone();
    var k = m.Clone();
    var identity = new double[,]
        {
            { 1, 0, 0, 0 },
            { 0, 1, 0, 0 },
            { 0, 0, 1, 0 },
            { 0, 0, 0, 1 },
        };

    VariableArray2D<double> calibration = Variable.Array<double>(n, m).Named("calib matrix");
    calibration.ObservedValue = identity;
    RelativePosition = Variable.Array<double>(n, k).Named("relative motion");
    RelativePosition.ObservedValue = identity;

    StatePrior = Variable.Array<Gaussian>(m, n);
    StatePrior.ObservedValue = Util.ArrayInit(m.SizeAsInt, n.SizeAsInt, (i, j) => Gaussian.FromMeanAndVariance(Rand.Normal(), 1));

    State = Variable.Array<double>(m, n);
    State[m, n] = Variable.GaussianFromMeanAndVariance(0, 1).ForEach(m, n);
    State[m, n].InitialiseTo(StatePrior[m, n]);

    State = Variable.MatrixMultiply(State, RelativePosition).Named("state");

    Range mapSzRng = new Range(mapSize);

    var mapPrior = Variable.Array<Gaussian>(k, mapSzRng);
    mapPrior.ObservedValue = Util.ArrayInit(k.SizeAsInt, mapSzRng.SizeAsInt, (i, j) => Gaussian.FromMeanAndVariance(Rand.Normal(), 1));

    Map = Variable.Array<double>(k, mapSzRng).Named("Map");
    Map[k, mapSzRng] = Variable.GaussianFromMeanAndVariance(0, 1).ForEach(k, mapSzRng);
    Map[k, mapSzRng].InitialiseTo(mapPrior[k, mapSzRng]);

    var calibState = Variable.MatrixMultiply(calibration, State).Named("mm_calib_state");

    Observation = Variable.MatrixMultiply(calibState, Map).Named("observations");
}

The built graph is
image

In this graph observed nodes are: relative motion, observations and state prior. I need to infer Map and State.
But when I try to infer Map or State, I have the following error:

Unhandled Exception: Microsoft.ML.Probabilistic.Compiler.CompilationFailedException: MessageTransform failed with 2 error(s) and 0 warning(s):
Error 0: This model is not supported with VariationalMessagePassing due to Factor.MatrixMultiply(double[,] matrixMultiply, double[,] A, double[,] B). Try using a different algorithm or expressing the model differently in
Factor.MatrixMultiply(mm_calib_state_use, Map_use)
Details: Variational Message Passing does not support a MatrixMultiply factor with fixed output. using parameter types: [matrixMultiply] double[,],[A] DistributionStructArray2D<Gaussian, double>,[to_A] DistributionStructArray2D<Gaussian, double>,[B] DistributionStructArray2D<Gaussian, double>,[to_B] DistributionStructArray2D<Gaussian, double>,[result] DistributionStructArray2D<Gaussian, double>
Error 1: This model is not supported with VariationalMessagePassing due to Factor.MatrixMultiply(double[,] matrixMultiply, double[,] A, double[,] B). Try using a different algorithm or expressing the model differently in
Factor.MatrixMultiply(mm_calib_state_use, Map_use)
Details: Variational Message Passing does not support a MatrixMultiply factor with fixed output. using parameter types: [matrixMultiply] double[,],[A] DistributionStructArray2D<Gaussian, double>,[to_A] DistributionStructArray2D<Gaussian, double>,[B] DistributionStructArray2D<Gaussian, double>,[to_B] DistributionStructArray2D<Gaussian, double>,[result] DistributionStructArray2D<Gaussian, double>

Can somebody to explain what is mean MatrixMultiply factor with fixed output? And may be there is some workarounds exist to solve this problem?

Thanks in advance!

How to implement a dynamic bayes network with Infer.net?

I want to build a Dynamic Bayes Network.

I am completly new to probalistic progamming, factor graphs and simulation. I came up with a small scenario with cats and mice.
The cats and mice have a certain birthrate and deathrate, but also cats eat the mice. I came up with something close to a factor
graph that I implemented with the Infer.Net modeling API. You can see an Image of it here https://github.com/SebastianKapunkt/urban-garden-mouse-cats/blob/master/KatzenM%C3%A4useFactorGraph.png (Its in german :D).

Now I want to know how many cats I need so that the mice population is stable. For that I to continously see the populaiton change of both.
Currently I just have a uggly version of this and I am also not quite sure if the model that I build is build the infer.net way.

You can find all my code in this repository: https://github.com/SebastianKapunkt/urban-garden-mouse-cats

In the file RunCarAndMouseModel.cs is all the code that will run the Model and the value I came up by researching the internet (my expoert knowgled of this case :D).
In the other files is the code to create the model with its rules.

Some more question I have:

  • How can I continously observe the population change?
  • Why is the model build like 10 times when I run the inference to get the new population?
  • How do I prevent in my case the warning "has quality band Experimental which is less than the recommended quality band"

Error running truncated gaussian tutorial

Hi,

I'm running the truncated gaussian tutorial:

https://dotnet.github.io/infer/userguide/Truncated%20Gaussian%20tutorial.html

and I'm getting the following error:

Microsoft.ML.Probabilistic.Compiler.CompilationFailedException HResult=0x80131500 Message=MessageTransform failed with 3 error(s) and 0 warning(s): Error 0: This model is not supported with ExpectationPropagation due to Factor.Difference(double difference, double a, double b). Try using a different algorithm or expressing the model differently in Factor.Difference(x_use, threshold) Details: [0] System.ArgumentException: Gamma is not of type WrappedGaussian for argument 1 of method DoublePlusOp.AAverageConditional(WrappedGaussian sum = Gamma, WrappedGaussian b = double) [1] System.ArgumentException: Gamma is not of type WrappedGaussian for argument 1 of method DoublePlusOp.AAverageConditional(WrappedGaussian sum = Gamma, double b = double) [2] System.ArgumentException: Gamma is not of type TruncatedGaussian for argument 1 of method DoublePlusOp.AAverageConditional(TruncatedGaussian sum = Gamma, double b = double) [3] System.ArgumentException: Gamma is not of type double for argument 1 of method DoublePlusOp.AAverageConditional(double sum = Gamma, TruncatedGaussian b = double) [4] System.ArgumentException: Gamma is not of type Gaussian for argument 1 of method DoublePlusOp.AAverageConditional(Gaussian Sum = Gamma, Gaussian b = double) [5] System.ArgumentException: Gamma is not of type double for argument 1 of method DoublePlusOp.AAverageConditional(double Sum = Gamma, Gaussian b = double) [6] System.ArgumentException: Gamma is not of type Gaussian for argument 1 of method DoublePlusOp.AAverageConditional(Gaussian Sum = Gamma, double b = double) [7] System.ArgumentException: Gamma is not of type double for argument 1 of method DoublePlusOp.AAverageConditional(double Sum = Gamma, double b = double) Error 1: This model is not supported with ExpectationPropagation due to Factor.Difference(double difference, double a, double b). Try using a different algorithm or expressing the model differently in Factor.Difference(x_use, threshold) Details: [0] System.ArgumentException: Gamma is not of type WrappedGaussian for argument 1 of method DoublePlusOp.SumAverageConditional(WrappedGaussian a = Gamma, WrappedGaussian b = double) [1] System.ArgumentException: Gamma is not of type WrappedGaussian for argument 1 of method DoublePlusOp.SumAverageConditional(WrappedGaussian a = Gamma, double b = double) [2] System.ArgumentException: Gamma is not of type double for argument 1 of method DoublePlusOp.SumAverageConditional(double a = Gamma, WrappedGaussian b = double) [3] System.ArgumentException: Gamma is not of type double for argument 1 of method DoublePlusOp.SumAverageConditional(double a = Gamma, TruncatedGaussian b = double) [4] System.ArgumentException: Gamma is not of type TruncatedGaussian for argument 1 of method DoublePlusOp.SumAverageConditional(TruncatedGaussian a = Gamma, double b = double) [5] System.ArgumentException: Gamma is not of type Gaussian for argument 1 of method DoublePlusOp.SumAverageConditional(Gaussian a = Gamma, Gaussian b = double) [6] System.ArgumentException: Gamma is not of type double for argument 1 of method DoublePlusOp.SumAverageConditional(double a = Gamma, Gaussian b = double) [7] System.ArgumentException: Gamma is not of type Gaussian for argument 1 of method DoublePlusOp.SumAverageConditional(Gaussian a = Gamma, double b = double) [8] System.ArgumentException: Gamma is not of type double for argument 1 of method DoublePlusOp.SumAverageConditional(double a = Gamma, double b = double) Error 2: This model is not supported with ExpectationPropagation due to Factor.IsPositive(bool isPositive, double x). Try using a different algorithm or expressing the model differently in Factor.IsPositive(vdouble2_use) Details: [0] System.ArgumentException: Gamma is not of type Gaussian for argument 2 of method IsPositiveOp.XAverageConditional(Bernoulli isPositive = bool, Gaussian x = Gamma) [1] System.ArgumentException: Gamma is not of type double for argument 2 of method IsPositiveOp.XAverageConditional(Bernoulli isPositive = bool, double x = Gamma) [2] System.ArgumentException: Gamma is not of type Gaussian for argument 2 of method IsPositiveOp.XAverageConditional(bool isPositive = bool, Gaussian x = Gamma) [3] System.ArgumentException: Gamma is not assignable from TruncatedGaussian for result of method IsPositiveOp.XAverageConditional [4] System.ArgumentException: Gamma is not assignable from TruncatedGaussian for result of method IsPositiveOp.XAverageConditional [5] System.ArgumentException: Gamma is not of type Gaussian for argument 2 of method IsPositiveOp_Proper.XAverageConditional(Bernoulli isPositive = bool, Gaussian x = Gamma)

Source=Microsoft.ML.Probabilistic.Compiler
StackTrace:
at Microsoft.ML.Probabilistic.Compiler.ModelCompiler.GetTransformedDeclaration(ITypeDeclaration itd, MethodBase method, AttributeRegistry2 inputAttributes) at Microsoft.ML.Probabilistic.Compiler.ModelCompiler.CompileWithoutParams(ITypeDeclaration itd, MethodBase method, AttributeRegistry2 inputAttributes)
at Microsoft.ML.Probabilistic.Models.InferenceEngine.Compile()
at Microsoft.ML.Probabilistic.Models.InferenceEngine.GetCompiledInferenceAlgorithm(Boolean inferOnlySpecifiedVars, IVariable var)
at Microsoft.ML.Probabilistic.Models.InferenceEngine.Infer(IVariable var)
at snip....

Code:

    private static void TruncatedGauss()
    {
      var x = Variable.GammaFromMeanAndVariance(0, 1).Named("x");

      var threshold = Variable.New<double>().Named("threshold");
      Variable.ConstrainTrue(x > threshold);

      var engine = new InferenceEngine();
      engine.Algorithm = new ExpectationPropagation();

      threshold.ObservedValue = 0.4;
      var inferred = engine.Infer(x).ToString();
    }

Zero positive results in error within 'specify'

<ignore this issue. I realized I sent it to the wrong 'infer' project>

(package version 0.5.1)

While I understand the intent with throwing an error if the result is not in the input data, it makes it hard to script a function as I now have to watch for it and simulate output (as the input may contain no positive results). Maybe a warning rather than an error?

samples <- data.frame(result=c(rep(FALSE, 100), rep(TRUE, 0)))
bootstrap <- infer::specify(samples, response=result, success="TRUE") # also, allowing logicals in 'success' would be nice.

Search engine

There is a possibility to make an infer model, the training features must have labels and doubles. It is possible with infer or I have to stay with ML.
What do you recommend, to make a smart search engine, based on geography, time and user preferences?

Help writing parameter estimator

Problem statement: I have an airflow system which uses sonic nozzles to meter flow through a test piece. There is a regulator before the nozzle call it p and a resulting output call it g(p) at the test piece after the nozzle. g(p) must fall between a certain range in order to take a measurement.
We use a PID loop to adjust p to get a value (lower < g(p) < upper). Currently we have no starting points to make a first guess of p for the PID loop to start, so we make a terrible first guess that is low and let the PID loop correct up.

I'm hoping to estimate a good initial guess that I can use to start the PID loop. Once the g(p) is inside the bounds and if the measurement is good, I would like to update the prior distribution using the observed p and g(p), and then estimate the optimum guess for p for the next trial based on an ideal g(p) = target. So essentially E[ p | g(p) = target]

Currently the bad initial guess takes the PID loop algorithm 30sec per test to find an optimum g(p), my target is <15s.

I've been reading through the tutorials and I'm having trouble getting started. Most of the examples don't quite match this case, but it is essentially a signal processing problem. Any help would be appreciated.

Problem with installation

Hello!

I installed .NET Core 2.1 from the link https://www.microsoft.com/net/download

Then I cloned the repo.

After that, I tried to run command dotnet add package Microsoft.ML.Probabilistic and I get the error Could not find any project in C:\Users\chintan.b.shah\Desktop\infer`.`

Can you please help with installation?

Many thanks.

Chintan

Using buffers in a custom factor

I've been implementing a custom factor according to this guide and so far it's worked pretty well.

I'd like to use Monte Carlo integration to compute one of the NCVMP messages. To do so, I think I need to use the Buffers attribute so that I can access the previous setting of the natural parameters. I added the following tag to my class:

[Buffers("UDirectionTimesConcentration", "XMean", "XVariance", "InnerProductMean", "InnerProductPrecision")]

And the requisite Init methods (with [Skip] tags) and accessor methods (with [Fresh] tags). Example:

[Skip]
public static Vector UDirectionTimesConcentrationInit([IgnoreDependency] VonMisesFisher u)
{
    return Vector.Zero(u.Dimension);
}

[Fresh]
public static Vector UDirectionTimesConcentration([Proper] VonMisesFisher u, Vector result)
{
    return u.GetDirectionTimesConcentration(result);
}

And finally, add the message function:

public static VonMisesFisher UAverageLogarithm(Vector UDirectionTimesConcentration, Vector XMean, double InnerProductMean, double InnerProductVariance, VonMisesFisher result)
{
    VonMisesFisher u = VonMisesFisher.FromDirectionTimesConcentration(UDirectionTimesConcentration);
    int numSamples = 1000;
    Vector suff = Vector.Zero(u.Dimension);
    for (int i = 0; i < numSamples; i++)
    {
        Vector s = u.Sample();
        suff += -0.5 * (s.Inner(XMean) - InnerProductMean) / InnerProductVariance * (s - u.GetDirection());
    }
    return VonMisesFisher.FromDirectionTimesConcentration(suff / numSamples);
}

Now, how do I link this implementation to the standard message function which has signature

public static VonMisesFisher UAverageLogarithm(Gaussian innerProduct, VectorGaussian x, VonMisesFisher result)

? Thanks.

EP comparison to other PPLs and roadmap (HMC etc)

Hi,

I have an extensive background in dotnet stack however for about 1 year and half I have been using python to perform all things machine learning. For last few months I have been studying bayesian statistics and probabilistic programming. Needless to say, for me C# is a far superior language than python however what matters the most is the libraries available in a particular domain. Therefore, seeing libraries like Infer.net gives me a ray of hope of using a better technology stack (i.e. dotnet).

That said, I have explored this library and have a decent idea on how it works. Have read (and executed as well) some chapters of MBML book. I am familiar with pymc3, pyro, webppl, edward and tensorflow_probability. All these PPLs or embedded PPLs have their respective take on how models should be expressed, executors/compilers, accelerator support and provide implementation of various inference algorithms.

As I did my feasibility I have noticed following -

  • Infer.net has three inference algorithms - EP, Gibbs and VI with EP being the default. While running the code corresponding to MBML book, I tried to change the algorithm to Gibbs and VI and almost all of the time (tried 3-4 projects) I got an unsupported error. I also see lot of examples in infer.net that have assertions or exception if EP is not the selected algorithm. This shows that even though other inference engines are there they are not first class citizens.

  • PyMC3 and Pyro do not support EP. They do implement variety of MCMC and VI based algorithms. I know you can not speak for why these libraries have not (yet ?) implemented EP but it somewhere indicates to me that they value other algorithms overs EP. Does this mean EP is not state of art ?

  • Edward (I am least familiar with it) lists various inference algos here - http://edwardlib.org/api/ed/inferences ... I do not see an explicit mention of EP but I have seen an example of it here http://edwardlib.org/api/inference-compositionality

  • Stan (again superficial knowledge of stan ecosystem) but if you look at the landing page (https://mc-stan.org/) I do not see mentioning of EP. They do mention others. May be they support it; I am not sure about this one.

May be my understanding is incorrect but it seems that expectation propagation algorithm is not favored by many other PPLs and frameworks. At least it is not a high priority for them where as Infer.net is primarily EP.

Seeing this contradiction i.e. Infer.net making EP the first class citizen and others not even supporting it, I feel I am missing something fundamental here. I would sincerely appreciate if you could correct my understanding and guide.

Even if my understanding is incorrect and there are scenarios where EP works better than other inference algorithms I find that in comparison to other frameworks Infer.net is limited in the number of supported algorithms. Is there a roadmap/plan to implement more inference engines (exact and approximate inference - Junction tree, ADVI, HMC etc)

Many thanks for this hard work

Regards
Kapil

DisctreteChar possible NRE calling Write

DiscreteChar was updated to drop the use of use rangeCount, with that a bug was introduced.

There is a a null check:

infer/src/Runtime/Distributions/DiscreteChar.cs

Line 1658 in 532cfd5

if (Ranges != null)

Immediately after that if, without setting the value of Ranges, there is a use of the Length property:

infer/src/Runtime/Distributions/DiscreteChar.cs

Line 1664 in 532cfd5

writeInt32(Ranges.Length); // For compatibility with old readers

This will lead to a NullReferenceException in some cases.

Allow approximating parameters of distributions with no conjugate prior (Dirichlet/Gamma/etc.) with transformed (Vector)Gaussians

Hi Infer.Net developers!

As far as I understand, there are some distributions like Dirichlet or Gamma, which the FAQ states that the parameters cannot be inferred of. I am wondering whether it could be useful to allow transformed Gaussian priors on these, a bit like Automated Differential Variational Inference (ADVI) without heavy penalty.

As far as I understand, the EP and VMP algorithm already calculate the moments of messages during message passing, and the transformed gaussian would in this sense be an almost direct approximation using these moments (albeit a crude one). In the multi-variate case (VectorGaussian) one would also potentially get a correlation, which could be useful.

For example, I want to allow for observed simplex data (to be drawn from a Dirichlet) in a model and it seems that there is no easy way to get a prior for the Dirichlet to be inferred (or even get MAP estimates of these parameters).

Thanks in advance!

Hierarchical model with pooling of precision

I'm trying to fit a hierarchical model with pooling of mean and variance:

In pseudocode that would be something like:

mu_mean = Gaussian(0,1)
mu_prec = Gamma(0.001, 0.001)
tau_mean = Gaussian(0,1)
tau_prec = Gamma(0.001, 0.001)

group_mu[N] = Gaussian(mu_mean, mu_prec).ForEach(N)
group_tau[N] = Gamma(tau_mean, tau_prec).ForEach(N)

y[N,K] = Gaussian(group_mu[N], group_tau[N]).ForEach(K)

The problem is that Gamma is not an conjugate prior of Gamma and this wont fit together or sometimes it will fit with some kicking, but then it won't converge (even if it does converge in unpooled configuration).

I've tried with something like

sd[N] = Gaussian(0,1)
y[N,K] = (group_mu[N] + sd[N] * group_tau[N]).ForEach(K)

...but then the compiler complains about multiplication.

Is there a way to describe such a hierarchical model with pooling of variances/precisions?
Or is there an example of such code. I've tried to replicate the Rats example, but can't figure out how to add variance pooling (yes, I'm a newbie).

P.S. The pseudocode is simpler than my real code... in reality, I have jagged arrays and I'm trying to fit some probabilities (converted to Log space) of an event-per-trial occurring. Usually the data is mixed: where there are just few trials, I model that with Poisson, whereas where numbers are very big, I fit probabilities directly (usually I have both for each group).

Sorry for littering the github issues - is there a better place to ask questions (I'm having techical issues with posting on the social.microsoft.com forum)?

Baeysian PCA on Poisson data

Hi,

I'm trying to build a PCA model to fit poisson data. My use case is nearly the same as the PCA tutorial, except that my observed data is Poisson distributed.

For example instead of Variable.GaussianFromMeanAndPrecision in
https://github.com/dotnet/infer/blob/master/src/Tutorials/BayesianPCA.cs#L102
my case would contain something like
data[observation, feature] = Variable.Poisson(events * T[observation, feature]).

My problem is that Variable.Poisson expects mean rates in Log space, so the above does not compile.
The error code is saying Gaussian is not assignable from Gamma for result of method PoissonOp.MeanAverageLogarithm (interesting thing regarding this error is that my code doesn't contain any gamma distribution - they are all gaussian - yet error report complains about gamma).

On the other hand, if I transform my principal components into log space and do a
data[observation, feature] = Variable.Poisson(events * Variable.Exp( T[observation, feature] )) then the code compiles, but I can't get correct results because MatrixMultiply doesn't do the right thing -- in log-space, weights and factors should be added rather than multiplied.

I'd be very thankful for any hints on how to approach this.

How to infer a Beta distribution?

I'm trying to infer a Beta distribution given some data. Fitting a Gaussian on the same data works whereas fitting a Beta distribution results in:

System.MissingMethodException:
varianceAverageConditional not found in BetaFromMeanAndVarianceOp using parameter types:
[sample] double,[mean] Gaussian,[to_mean] Gaussian,[variance] Gamma,[to_variance] Gamma,[result] Gamma

I've tried substituting Beta with a Gama to no avail.
Is there a workaround for working with Beta distributions?

My sample code is below (note the commented Gaussian vs Beta):

let data = [|0.5; 0.1; 0.1; 0.11; 0.2; 0.4 |]
let N = Range data.Length

let mean = Variable.GaussianFromMeanAndVariance(0.1, 1.0).Named("mean")
let var = Variable.GammaFromShapeAndScale(1.0, 1.0).Named("var")

let y = Variable.Array N
y.[N] <- Variable.BetaFromMeanAndVariance(mean, var).ForEach(N)
//y.[N] <- Variable.GaussianFromMeanAndVariance(mean, var).ForEach(N) // <- this works
y.ObservedValue <- data

let engine = InferenceEngine ()
engine.Infer mean |> printfn "mean: %A" 
engine.Infer var  |> printfn "var:  %A"

Visualizing Graphs

Hi team,
I'd like to thanks for the awesome job. As a matter of curiosity, because I want to use Infer.net to show models for students, I'd like to know if it is possible to generate visual graphs like that from Christopher Bishop presentations.
And I'd like to know how to use Microsoft.ML.Probabilistic.Visualizers.Windows (if it is possible).
Thanks,
Ely

How to run infer.net on Mono under linux?

hi,
need some help...
my requirement is to call infer.net model(c# dll) in c++ under Linux environment, and my current method is using mono embedding:https://www.mono-project.com/docs/advanced/embedding/
the general c# code works fine, but the infer.net code will fail.
the obvious difference is infer.net, actually running infer.net on mono also failed. I tested the simple FirstExample.cs:
csc -r:Microsoft.ML.Probabilistic.Compiler.dll -r:mscorlib.dll FirstExample.cs
and got error:

Microsoft (R) Visual C# Compiler version 3.3.1-beta4-19462-11 (66a912c9)
Copyright (C) Microsoft Corporation. All rights reserved.

FirstExample.cs(10,18): error CS0518: Predefined type 'System.Object' is not defined or imported
FirstExample.cs(12,23): error CS0518: Predefined type 'System.Void' is not defined or imported
FirstExample.cs(27,26): error CS0518: Predefined type 'System.String' is not defined or imported
FirstExample.cs(27,16): error CS0518: Predefined type 'System.Void' is not defined or imported

running code without infer.net on mono is ok, runing FirstExample.cs with dotnet(dotnet run) is also ok.
so my question is how to run infer.net on Mono?
or is there any better method to meet my needs?
thanks.

TraceMessages in self assessment 2.3 of MBML handbook

I'm coding self assessment 2.3 (part 3) in the MBML handbook and would like to see if they match those in the table:

Screenshot 2020-05-19 at 10 24 22

I've implemented TraceMessages() in this way, but it does not print anything to console:

           // csharp skill prior 
            var csharpSkill = Variable.Bernoulli(0.5).Named("csharpSkill");

            //csharp isCorrect p(isCorrect|csharp)
            //0         0         = 0.8
            //0         1         = 0.2
            //1         0         = 0.1
            //1         1         = 0.9

            Variable<bool> isCorrect1 = Variable.New<bool>();

            // add noise factor for csharp skill
            using (Variable.If(csharpSkill))
            {
                isCorrect1.SetTo(Variable.Bernoulli(0.9));
            }
            using (Variable.IfNot(csharpSkill))
            {
                isCorrect1.SetTo(Variable.Bernoulli(0.2));
            }

            // sql skill prior
            var sqlSkill = Variable.Bernoulli(0.5).Named("sqlSkill");

            Variable<bool> isCorrect2 = Variable.New<bool>();

            // add noise factor for sql skill
            using (Variable.If(sqlSkill))
            {
                isCorrect2.SetTo(Variable.Bernoulli(0.9));
            }
            using (Variable.IfNot(sqlSkill))
            {
                isCorrect2.SetTo(Variable.Bernoulli(0.2));
            }

            //csharp   sql   hasSkills p(hasSkills|csharp, sql)
            //0         0         0           = 1
            //0         0         1           = 0
            //0         1         0           = 1
            //0         1         1           = 0
            //1         0         0           = 1
            //1         0         1           = 0
            //1         1         0           = 0
            //1         1         1           = 1

            // and factor for csharp skill and sql skill
            var hasSkills3 = ((csharpSkill & sqlSkill) == Variable.Bernoulli(1)).Named("hasSkills3");
            
            Variable<bool> isCorrect3 = Variable.New<bool>();

            // add noise factor for has skills
            using (Variable.If(hasSkills3))
            {
                isCorrect3.SetTo(Variable.Bernoulli(0.9));
            }
            using (Variable.IfNot(hasSkills3))
            {
                isCorrect3.SetTo(Variable.Bernoulli(0.2));
            }

            // and factor for csharp skill and sql skill
            var hasSkills4 = ((csharpSkill & sqlSkill) == Variable.Bernoulli(1)).Named("hasSkills4");
            
            Variable<bool> isCorrect4 = Variable.New<bool>();

            // add noise factor for has skills
            using (Variable.If(hasSkills4))
            {
                isCorrect4.SetTo(Variable.Bernoulli(0.9));
            }
            using (Variable.IfNot(hasSkills4))
            {
                isCorrect4.SetTo(Variable.Bernoulli(0.2));
            }

            var engine = new InferenceEngine();

            engine.Algorithm = new ExpectationPropagation();

            engine.NumberOfIterations = 10;

            isCorrect1.ObservedValue = true;
            isCorrect2.ObservedValue = false;
            isCorrect3.ObservedValue = false;
            isCorrect4.ObservedValue = false;

            csharpSkill.AddAttribute(new TraceMessages());
            sqlSkill.AddAttribute(new TraceMessages());
            hasSkills3.AddAttribute(new TraceMessages());
            hasSkills4.AddAttribute(new TraceMessages());

            Console.WriteLine(engine.Infer(csharpSkill));
            Console.WriteLine(engine.Infer(sqlSkill));

Point estimates and gradients from accumulated messages

I'm reading through the Trueskill2 paper, mostly to about learn Infer.net as a PPL.

I'm quoting from the 9th page of the above paper:

Our final approach exploits the fact that each parameter connects to a million factors. In such a model, the parameter will tend to have a sharp posterior distribution. Therefore we chose to drop the uncertainty in the parameter posteriors, and approximate them as point masses during each iteration. This not only saves memory on messages, but also saves computation. At the end of each iteration, we update each parameter using Rprop [Riedmiller and Braun,1993], where the gradient is computed by accumulating the Expectation Propagation messages into that parameter. The incoming messages are computed once and discarded. This procedure reaches reasonable parameter estimates in about 100 iterations, i.e. 100 forward and backward sweeps through the data.

I'm curious about the implementation of the above in Infer.net. I've put together a small example, I'm curious if this is what they meant in the paper:

First, the example:

var observedTosses = new[] {true, false, true, true, true, false, false, false } ;

var p = Variable.BetaFromMeanAndVariance(1, 1.0/12);

var numTosses = observedTosses.Length;
var toss = new Range(numTosses);
var tosses = Variable.Array<bool>(toss).Named("tosses");

tosses[toss] = Variable.Bernoulli(p).ForEach(toss);

var inferenceEngine = new InferenceEngine();

tosses.ObservedValue = observedTosses;
var inferredProbability = inferenceEngine.Infer<Beta>(p);

Console.WriteLine(inferredProbability);

For this particular set of observed variables, we get Beta(5,5)[mean=0.5].

Now, when the paper talks about accumulating the EP messages [coming] into the variable, would that be implemented like so:

var observedTosses = new[] {true, false, true, true, true, false, false, false};

var p = Variable.BetaFromMeanAndVariance(0.5, 1.0/12);
p.AddAttribute(new ListenToMessages());

var numTosses = observedTosses.Length;
var toss = new Range(numTosses);
var tosses = Variable.Array<bool>(toss).Named("tosses");

tosses[toss] = Variable.Bernoulli(p).ForEach(toss);

var inferenceEngine = new InferenceEngine();
inferenceEngine.MessageUpdated += (algorithm, args) => { Console.WriteLine(args.Message); };

tosses.ObservedValue = observedTosses;
var inferredProbability = inferenceEngine.Infer<Beta>(p);

Console.WriteLine(inferredProbability);

With the above method I do see the following messages whenever p gets updated:

Beta(2,1)[mean=0.6667]
Beta(1,2)[mean=0.3333]
Beta(2,1)[mean=0.6667]
Beta(2,1)[mean=0.6667]
Beta(2,1)[mean=0.6667]
Beta(1,2)[mean=0.3333]
Beta(1,2)[mean=0.3333]
Beta(1,2)[mean=0.3333]
Beta(1,1)[mean=0.5]
Beta(5,5)[mean=0.5]
Beta(5,5)[mean=0.5]
Beta(5,5)[mean=0.5]

I'm not sure how I'd get the gradient though. Also how does this relates to the next thing that I've tried, which is setting the variable to be a PointEstimate?

I've done this by adding the PointEstimate attribute to the p variable, like so:

var observedTosses = new[] {true, false, true, true, true, false, false, false};

var p = Variable.BetaFromMeanAndVariance(0.5, 1.0 / 12);
p.AddAttribute(new PointEstimate());

var numTosses = observedTosses.Length;
var toss = new Range(numTosses);
var tosses = Variable.Array<bool>(toss).Named("tosses");

tosses[toss] = Variable.Bernoulli(p).ForEach(toss);

var inferenceEngine = new InferenceEngine();

tosses.ObservedValue = observedTosses;
var inferredProbability = inferenceEngine.Infer<Beta>(p);

Console.WriteLine(inferredProbability);

The output for the above is Beta.PointMass(0.5).

The interesting part comes now, where we combine the two attributes:

var observedTosses = new[] {true, false, true, true, true, false, false, false};

var p = Variable.BetaFromMeanAndVariance(0.5, 1.0 / 12);
p.AddAttribute(new PointEstimate());
p.AddAttribute(new ListenToMessages());

var numTosses = observedTosses.Length;
var toss = new Range(numTosses);
var tosses = Variable.Array<bool>(toss).Named("tosses");

tosses[toss] = Variable.Bernoulli(p).ForEach(toss);

var inferenceEngine = new InferenceEngine();
inferenceEngine.MessageUpdated += (algorithm, args) => { Console.WriteLine(args.Message); };

tosses.ObservedValue = observedTosses;
var inferredProbability = inferenceEngine.Infer<Beta>(p);

Console.WriteLine(inferredProbability);

If I do this, then I can see a couple of the following message in the console:
.Microsoft.ML.Probabilistic.Factors.RpropBufferData.

To me, this indicates that the whole procedure of updating a point estimate with Rprop based on the gradient, which is calculated by accumulating the incoming messages, is done automatically by Infer.NET by only adding the PointEstimate attribute? That would be pretty cool, actually.

Probabilities in Lbl1-Lbl3 are not correct after reset

In the function OnReset() add
string fmtStr = "{0:0.0000}";

        Lbl1.Content = String.Format(fmtStr, cProbs[0]);
        Lbl2.Content = String.Format(fmtStr, cProbs[1]);
        Lbl3.Content = String.Format(fmtStr, cProbs[2]);

right after
label1.Content = "P ( Car Inside )\nP ( Host Chooses )";

This will ensure the prior probabilities (1/3,1/3,1/3) are correctly displayed...

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.