warn macro needs logger arg; problem with pmap; saving MAlgoBGP not working about smm.jl HOT 15 CLOSED

from smm.jl.

from smm.jl.

I double-checked several times as I was very puzzled. The objective function is evaluated correctly in every chain, then the chain is correctly updated within next_eval!: everything works correctly but the updated chains are not carried over outside of pmap and stored in algo as I expected. However you would have noticed this yourself if you tested the algorithm, so I do not know what is going on. Also, I agree that my correction is gross but it worked out in my case: there is certainly a better way to write it though. The objective function is (algo options are below)

x -> objective_function(x, M)

with

function objective_function(ev::MomentOpt.Eval, M::Simulation.Model)

    # Collect information from Eval object
    MomentOpt.start(ev)
    MomentOpt.setValue(ev, 1.0)
    Parameters.@unpack fxy, u0, v0, λ, δx, δy, δz, ψm, ψf, ψ, r, γ, S0, n0, θ = M
    Kθ = length(θ); θ = zeros(Kθ)

    # Print newly assigned values for the parameter vector
    for tuple in ev.params
        println("Parameter $(tuple[1]) is $(tuple[2])")
    end

    # Replace parameter vector
    for i in 1:Kθ
        θ[i] = ev.params[Symbol("th$i")]
    end

    # Collect empirical moments
    moments_hat = collect(values(ev.dataMoments))
    W = collect(values(ev.dataMomentsW))

    # Compute new production functions
    m = deepcopy(M)
    m = Simulation.add_parameters(m, fxy, u0, v0, λ, δx, δy, δz, ψm, ψf, ψ, r,
        θ, γ; n0 = n0, S0 = S0)

    # Solve the model, compute value function and simulated moments
    m = Simulation.solve_model(m)

    if m.status == 1
        m = Simulation.get_moments_sim(m, match = true)
        Parameters.@unpack moments_sim = m
        W = W / sum(W)
        target = sum( ( (moments_hat - collect(values(moments_sim))) ./ moments_hat ).^2 .* W)
        MomentOpt.setValue(ev, target)
        MomentOpt.setMoment(ev, Dict(moments_sim))
        m.n0 = [m.nx m.ny]; m.S0 = m.Sxyz; m.m0 = m.mxyz # hereon use these as initial values
        ev.status = 1
    end

    MomentOpt.setSolution(ev, m)
    MomentOpt.finish(ev)

    return ev

end

Algo options are

Dict("N" => nchains,
    "maxiter" => 4,
    "maxtemp" => 5,
    "coverage" => 0.025,
    "sigma_update_steps" => 10,
    "sigma_adjust_by" => 0.01,
    "smpl_iters" => 100000,
    "parallel" => true,
    "maxdists" => [0.05 for i in 1:nchains],
    "acc_tuner" => 15.0,
    "animate" => false)

from smm.jl.

And to make sure it is clear, AlgoBGP stops because it fails the following test

https://github.com/floswald/MomentOpt.jl/blob/d50ef47b77773f5addfc1e31d52756c64a1a998d/src/mopt/AlgoBGP.jl#L502

although as I told you next_eval! works smoothly within pmap.

from smm.jl.

from smm.jl.

so this is very strange. I cannot reproduce this. can you please do Pkg.checkout("MomentOpt"), start a new session and so using MomentOpt; MomentOpt.serialNormal() ? this runs for me correctly and is the basis of a just newly added unit test. all tests pass on my machine, travis and appveyor are currently running dying on me because neither can properly load PyPlot.

from smm.jl.

the output from a 20-iterations example MomentOpt.serialNormal(20) looks something like this:

julia> MomentOpt.history(x.chains[1])
20×9 DataFrames.DataFrame
│ Row │ iter │ value     │ accepted │ curr_val  │ best_val  │ prob      │ exchanged │ p1         │ p2         │
├─────┼──────┼───────────┼──────────┼───────────┼───────────┼───────────┼───────────┼────────────┼────────────┤
│ 1   │ 1    │ 1.45175   │ true     │ 1.45175   │ 1.45175   │ 1.0       │ 0         │ 0.2        │ -0.2       │
│ 2   │ 2    │ 0.760537  │ true     │ 0.760537  │ 0.760537  │ 1.0       │ 3         │ 0.308089   │ 0.0291797  │
│ 3   │ 3    │ 0.629384  │ true     │ 0.629384  │ 0.629384  │ 1.0       │ 0         │ 0.181651   │ -0.254984  │
│ 4   │ 4    │ 0.673976  │ false    │ 0.629384  │ 0.629384  │ 0.585606  │ 0         │ -0.123411  │ -0.767274  │
│ 5   │ 5    │ 0.802517  │ false    │ 0.629384  │ 0.629384  │ 0.12523   │ 0         │ 0.697288   │ 0.235541   │
│ 6   │ 6    │ 0.782923  │ false    │ 0.629384  │ 0.629384  │ 0.158426  │ 0         │ 0.141543   │ -0.101378  │
│ 7   │ 7    │ 0.723521  │ false    │ 0.629384  │ 0.629384  │ 0.323149  │ 0         │ 0.166334   │ -0.134403  │
│ 8   │ 8    │ 0.599857  │ true     │ 0.599857  │ 0.599857  │ 1.0       │ 2         │ 0.516552   │ -0.0241237 │
│ 9   │ 9    │ 0.507343  │ true     │ 0.507343  │ 0.507343  │ 1.0       │ 0         │ 0.431551   │ -0.176718  │
│ 10  │ 10   │ 0.704049  │ false    │ 0.507343  │ 0.507343  │ 0.0943759 │ 0         │ 0.523783   │ 0.0817499  │
│ 11  │ 11   │ 0.346174  │ true     │ 0.346174  │ 0.346174  │ 1.0       │ 2         │ 1.35379    │ -0.258162  │
│ 12  │ 12   │ 0.23485   │ true     │ 0.23485   │ 0.23485   │ 1.0       │ 2         │ 0.36084    │ -1.24317   │
│ 13  │ 13   │ 0.178655  │ true     │ 0.178655  │ 0.178655  │ 1.0       │ 0         │ 0.677804   │ -1.50555   │
│ 14  │ 14   │ 0.0803123 │ true     │ 0.0803123 │ 0.0803123 │ 1.0       │ 0         │ 0.767662   │ -1.31983   │
│ 15  │ 15   │ 0.180196  │ false    │ 0.0803123 │ 0.0803123 │ 0.301616  │ 0         │ 0.469792   │ -0.717618  │
│ 16  │ 16   │ 0.0236958 │ true     │ 0.0236958 │ 0.0236958 │ 1.0       │ 0         │ 0.909039   │ -0.814352  │
│ 17  │ 17   │ 0.0842257 │ true     │ 0.0842257 │ 0.0236958 │ 0.483667  │ 0         │ 0.642016   │ -0.770849  │
│ 18  │ 18   │ 0.188059  │ true     │ 0.188059  │ 0.0236958 │ 0.287655  │ 0         │ 0.419304   │ -0.7793    │
│ 19  │ 19   │ 0.460388  │ false    │ 0.188059  │ 0.0236958 │ 0.0380842 │ 0         │ 0.0501502  │ -0.893285  │
│ 20  │ 20   │ 0.545659  │ false    │ 0.188059  │ 0.0236958 │ 0.0136885 │ 0         │ -0.0208657 │ -0.785931  │

from smm.jl.

The checkout didn't say anything interesting. I removed the package and cloned the latest version (with fixed warn-logger issue) but I get the same error.
I'm in DEBUG mode here. You can see that on each worker I successfully draw values (at least the initial values) for my 5 parameters and solve my model within pmap. I remember previously checking and including a line to print the result of my objective function to make sure it was correct. Also, always by printing their respective outcomes, I can tell you that doAcceptReject! and set_eval! also work correctly. I do not understand why it is not working for me. It doesn't matter as I fixed it myself, but I expected it to show up for you too. runMOpt! gives

17:24:11:INFO:Main:Starting estimation loop.
17:24:11:INFO:Main:Starting estimation loop.
2017-12-08 17:24:11:INFO:Main:Starting estimation loop.
17:24:11:DEBUG:Main:iteration 1
17:24:11:DEBUG:Main:iteration 1
2017-12-08 17:24:11:DEBUG:Main:iteration 1
	From worker 5:	Parameter th1 is 0.8
	From worker 5:	Parameter th2 is 0.6
	From worker 5:	Parameter th3 is 0.6
	From worker 5:	Parameter th4 is 0.9
	From worker 5:	Parameter th5 is 1.0
	From worker 3:	Parameter th1 is 0.8
	From worker 3:	Parameter th2 is 0.6
	From worker 3:	Parameter th3 is 0.6
	From worker 3:	Parameter th4 is 0.9
	From worker 3:	Parameter th5 is 1.0
	From worker 4:	Parameter th1 is 0.8
	From worker 4:	Parameter th2 is 0.6
	From worker 4:	Parameter th3 is 0.6
	From worker 4:	Parameter th4 is 0.9
	From worker 4:	Parameter th5 is 1.0
	From worker 2:	Parameter th1 is 0.8
	From worker 2:	Parameter th2 is 0.6
	From worker 2:	Parameter th3 is 0.6
	From worker 2:	Parameter th4 is 0.9
	From worker 2:	Parameter th5 is 1.0
	From worker 5:	Compute iterate 1 with error 0.0007456979517434618
	From worker 5:	Converged in 1 steps
	From worker 4:	Compute iterate 1 with error 0.0007456979517434618
	From worker 4:	Converged in 1 steps
	From worker 3:	Compute iterate 1 with error 0.0007456979517434618
	From worker 3:	Converged in 1 steps
	From worker 2:	Compute iterate 1 with error 0.0007456979517434618
	From worker 2:	Converged in 1 steps
17:24:43:WARN:Main:caught exception AssertionError("algo.i == algo.chains[ic].iter")
17:24:43:WARN:Main:caught exception AssertionError("algo.i == algo.chains[ic].iter")
2017-12-08 17:24:43:WARN:Main:caught exception AssertionError("algo.i == algo.chains[ic].iter")
AssertionError: algo.i == algo.chains[ic].iter

from smm.jl.

from smm.jl.

from smm.jl.

ok I got it. you are right with your pmap idea. it seems i never tested this new version on more than 1 processes! d'oh. can you post your snippet with the version of pmap that you use now? one needs to insert the returned evs also into the correct chain, upon making next_eval! to become next_eval

from smm.jl.

you could also submit a pull request against the current master to speed this up - i'll only be back on this monday.

from smm.jl.

ok done. can you please checkout the lastest version again - maybe try Pkg.checkout("MomentOpt",pull=true) this time. if it does nothing just go into ~/.julia/v0.6/MomentOpt and do a git pull manually. then please confirm that

1. Pkg.test("MomentOpt") runs
2. your applicatoin runs

then you can close that. thanks!

from smm.jl.

Thanks a lot for these corrections and sorry for the late reply. I did the checkout and it worked.

1. My application works correctly with the updated MomentOpt. I believe you fixed the main problem I pointed out with pmap. The way you fix it is definitely better than what I suggested: I also realized after applying my fix - assigning the output of pmap to algo.chains directly - that I had compromised the right order of the chains.
2. Pkg.test gave me three errors, but they seem related to the animated gifs, and not to BGP chains. Here is the summary of the test. I can post details if you want.

Test Summary:                             | Pass  Error  Total
Running MomentOpt tests                   |   70      3     73
  AlgoBGP                                 |    8      2     10
    Constructor                           |    8             8
    serialNormal() runs                   |           1      1
    parallelNormal() runs                 |           1      1
  Testing BGPChains constructor           |   26            26
  Testing Eval object                     |   17            17
  Mopt type                               |    8             8
  testing objfunctions                    |    6             6
  verifying read and write for Slices     |    4      1      5
  handling of failing objective on remote |    1             1

from smm.jl.

thanks @edoardociscato, i saw that the tests also fail on travis because I used the Dataframes.isna function - which seems to be deprecated in the newest version of Dataframes which travis uses. fixed now, and tests should finally pass. I will close this issue now 🎉 as it seems to be resolved (please reopen if not!). thanks a lot for reporting and your help, that was quite a major bug! now you can move on to fix my next open source project 😝

from smm.jl.