probml / pyprobml Goto Github PK

Some of the examples include a call to utils/util.load_mat that depends on a DATA_DIR being available where the .mat files are supposed to be.

Here's what grep -r -i --include \*.py load_mat gives me when run in examples.

prostateComparisonOrig.py:#X = util.load_mat('prostate')
naiveBayesBowDemo.py:data = util.load_mat('XwindowsDocData')
logregSATdemo.py:data = util.load_mat('sat/sat.mat')
gaussHeightWeight.py:data = util.load_mat('heightWeight')
newsgroupsVisualize.py:data = util.load_mat('20news_w100')
shuffledDigitsDemo.py:data = util.load_mat('mnistAll')
utils/util.py:# def load_mat(matName):
surfaceFitDemo.py:data = util.load_mat('moteData/moteData.mat')
kmeansHeightWeight.py:data = util.load_mat('heightWeight')

How would @murphyk recommend this is handled? Should the data directory be replicated in this repo so that the datasets become available for the scripts?

Test body
This is figure number 4.18 in the book. its matlab code is here

Test body
This is figure number 17.20 in the book. its matlab code is here

In Chapter 3, on page 55 under equation 3.57, we have

is the softplus function, that maps from R to R+, to ensure the predicted standard deviation is non-negative.

Are we talking R+ or non-negative R_{\geq 0}? R+ would be R_{> 0)

Test body
This is figure number 8.26 in the book. its matlab code is here

Test body
This is figure number 4.23 in the book. its matlab code is here

Test body
This is figure number 7.6 in the book. its matlab code is here

Test body
This is figure number 5.13 in the book. its matlab code is here

If Virginica is poisonous, shouldn't the cost of predicting Virginica to be Setosa or Versicolor be 10 not the way in the book? I understand the point of asymmetric cost but the numbers don't seem to go with the story.

Test body
This is figure number 2.21 in the book. its matlab code is here

Here you are discussion the half normal distribution defined from [0, to \infty) but using the notation \mathfrak{N}_+ which general denotes (0, \infty) when + is used.

Is this the correct nomenclature?

In examples like linregPolyVsDegree.py there are calls to a preprocessor_create function which seems to be part of an old version of utils.util but has since been removed.

I guess we should bring back some of those functions to get the examples working again?

Test body
This is figure number 5.15 in the book. its matlab code is here

Test body
This is figure number 21.19 in the book. its matlab code is here

Test body
This is figure number 4.25 in the book. its matlab code is here

Test body
This is figure number 6.2 in the book. its matlab code is here

Test body
This is figure number 3.10 in the book. its matlab code is here

In section 2.3.2, you say:

sj ∈ [0,∞] are the lengths of the two sides. Hence the hypothesis space is H = R2 × R2+, where R+ is the set of positive reals.

0 is not a positive number and since the domain is inclusive of 0, we should say R nonnegative R_{\geq 0} or however you would like to note it.

The draft date is 18 Jan 2021; I just started reading it yesterday so I think this is the most up to date version.

I apologise if this has already been raised, but examples/mnist1NNdemo.py expects a /Data/MNIST directory with the training data, and I'm not sure where to get this from.

I searched through the backlog and found PR #36, where @Duane321 wanted to add those files but he didn't have the right access.

Please let me know if they're already on the repo and I missed them.

Hi, Thanks for providing this excellent book. I just start reading it today.
I believe the sentense underlined in the picture should be "The false positive rate is defined as one minues the specifivity".

Test body
This is figure number 4.16 in the book. its matlab code is here

Test body
This is figure number 2.20 in the book. its matlab code is here

Test body
This is figure number 8.23 in the book. its matlab code is here

Test body
This is figure number 6.3 in the book. its matlab code is here

Test body
This is figure number 4.19 in the book. its matlab code is here

Test body
This is figure number 4.7 in the book. its matlab code is here

In chapter 3, on page 50, under equation 3.27, it says we require

0\leq f_c(x; \theta)\leq 1

I am a little rusty, I have to admit, but should that be the norm of the function? Feel free to reject this if this is wrong.

Did you mean to say ".... degrees... 20 fit to 21 datapoints...." instead of '.... 21 fit to 21 datapoints...'?

In the new format, you are listing each of the chapters by its number. It would be really helpful to specify the topic as well. This way a person could find what they are looking for without having to have the book in front of them.

Test body
This is figure number 8.13 in the book. its matlab code is here

Test body
This is figure number 4.24 in the book. its matlab code is here

This sentence, in section 2.4.4, sounds a bit off. Might just be me but take a look.

The advantage of the Bayesian approach is that by capturing uncertainty in the parameters θ, we will more accurately model uncertainty in our forecasts p(y|x, D) than we would using a plug-in approximation p(y|x, θˆ).

The fragment in question is:

"...than we would [be?] using a plug-in approximation..."

Hi, this is not an issue but a question about the estimated release date for the new edition of the book. I'm really excited about this. Thank you very much for your work.

Test body
This is figure number 4.13 in the book. its matlab code is here

Test body
This is figure number 8.27 in the book. its matlab code is here

Test body
This is figure number 8.15 in the book. its matlab code is here

Hi, I have been reading your textbook along with the new codes, and I think in sensor_fusion_2d.py there is a misplaced parenthesis on line 26 for the posterior mean calculation. According to (4.125), post['sigma'] should be multiplied with the sum of A.T.dot(sy_inv).dot(y.reshape(4,1) - py['mu']) and smu_inv.dot(pmu['mu']).reshape(2,1). The right parenthesis for post['sigma'].dot() should be moved to the end.

Test body
This is figure number 3.7 in the book. its matlab code is here

You need to include a ../ for the figures in order for the code to work properly in the file binomialBetaPosteriorDemo.py within the examples directory.
Hope this helps.

So as I'm working on porting code over, I was wondering if compatibility with Python 2.7 is still required.

Most if not all of the packages used in the code (numpy, scipy, etc) already support Python 3, and adding support for 2.7 will just create some additional work where I have to have all the packages installed for both versions and always test for both before making changes.

I would recommend to just move to Python 3 for the whole code-base, but I wanted @murphyk 's input on this.

Hello Kevin,

I was wondering if there is a plan to port all of the code examples from the ML book to Python?

If that's the case I'd love to help out :)

Test body
This is figure number 4.17 in the book. its matlab code is here

The code refers to example/linearRegression.py which was renamed examples/linregPmtkEmulator.py here.

Minor PR to follow.

The link for individual chapters in the README gives 404, cod you please fix the link?

Test body
This is figure number 8.15 in the book. its matlab code is here

I've started porting over some code and though I would start with ensuring that existing code runs fine first before starting to port Matlab code.

The first example I took is bfgs_autograd which currently results in this error:

Traceback (most recent call last):
  File "bfgs_autograd.py", line 35, in <module>
    main()
  File "bfgs_autograd.py", line 30, in main
    params, obj = bfgs_auto(Xtrain, ytrain)
  File "bfgs_autograd.py", line 21, in bfgs_auto
    result = minimize(obj_fun, params,  method='BFGS', jac=grad_fun)
  File "/home/tvas/anaconda3/envs/mlpp/lib/python3.5/site-packages/scipy/optimize/_minimize.py", line 444, in minimize
    return _minimize_bfgs(fun, x0, args, jac, callback, **options)
  File "/home/tvas/anaconda3/envs/mlpp/lib/python3.5/site-packages/scipy/optimize/optimize.py", line 921, in _minimize_bfgs
    old_old_fval = old_fval + np.linalg.norm(gfk) / 2
  File "/home/tvas/anaconda3/envs/mlpp/lib/python3.5/site-packages/numpy/linalg/linalg.py", line 2129, in norm
    sqnorm = dot(x, x)
TypeError: unsupported operand type(s) for *: 'function' and 'function'

If I instead switch over to using the commented out grad_fun = autograd.grad(obj_fun) instead of grad_fun = lambda params: gradient(params, Xtrain, ytrain)it works fine.

I'll open a small PR for this, I've tested it in Python 3.5 with the Anaconda distribution, scipy 0.18, autograd 1.1.6, numpy 1.11.1

Error running dtreeDemoIris.m on GNU Octave, version 4.2.2
With NaN installed and having fisheriris.m in the same directory.
Colab runtime with no accelerator and normal memory config
Octave error :

error: load: unable to determine file format of 'fisheriris.m'
error: called from
    dtreeDemoIris at line 9 column 1

If you want to contribute a Python version of a figure, follow these steps:

• Check the list of open issues for ones with a name like "Convert foo to Python". Such issues have the tag 'Figures'.
  Each issue has a link to the Matlab file that was used to generate the figure, and a figure number.
  Figure numbers refer to this version of the book: https://github.com/probml/pml-book/releases/tag/2021-03-08.
  Consult the pdf file to see what the figure should look like.

• Follow these guidelines for how to contribute code.

• Test your code works in Google colab, and make a pull request. Please include the generated image(s) in your PR, and add a tag which mentions the issue you are solving.

• Here is an example of good PR.

• If you are a student and want to be considered for the Google Summer of Code, please apply here after March 29th. You should have at least 2 PRs succesfully added to this repo before applying. Please follow the style guidelines mentioned above.

Test body
This is figure number 5.15 in the book. its matlab code is here