Giter VIP home page Giter VIP logo

quickpomdps's Issues

Project.toml

@lassepe and @rejuvyesh was reading your comments in the recent PR about the Project.toml. Would it make sense to just create a new JuliaProject.toml in the current directory (if one does not already exist)?

Another idea would be to see what diffeqpy does.

Installing dependencies every time the package is loaded?

If I understand correctly, right now any time someone calls import quickpomdps, we run julia.install() and install some julia package dependencies. This seems annoying and unnecessary. We should probably check whether they are already installed, perhaps using some kind of try/catch?

Can't solve POMDPs described using QuickPOMDP()

Hello,
This package seems to only support pomdps described using DiscreteExplicitPOMDP() as in the example in examples/tiger.py.
When I try to solve a pomdp described using QuickPOMDP, the following error occurs:

Traceback (most recent call last):
  File "/home/bill/anaconda3/envs/uncertain/lib/python3.6/site-packages/julia/pseudo_python_cli.py", line 308, in main
    python(**vars(ns))
  File "/home/bill/anaconda3/envs/uncertain/lib/python3.6/site-packages/julia/pseudo_python_cli.py", line 59, in python
    scope = runpy.run_path(script, run_name="__main__")
  File "/home/bill/anaconda3/envs/uncertain/lib/python3.6/runpy.py", line 263, in run_path
    pkg_name=pkg_name, script_name=fname)
  File "/home/bill/anaconda3/envs/uncertain/lib/python3.6/runpy.py", line 96, in _run_module_code
    mod_name, mod_spec, pkg_name, script_name)
  File "/home/bill/anaconda3/envs/uncertain/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "tiger.py", line 50, in <module>
    policy = solve(solver, m)
RuntimeError: <PyCall.jlwrap (in a Julia function called from Python)
JULIA: UndefRefError: access to undefined reference
Stacktrace:
 [1] getproperty(::Type{T} where T, ::Symbol) at ./Base.jl:28
 [2] #s12#1 at /home/bill/.julia/packages/Tricks/Omjx9/src/Tricks.jl:20 [inlined]
 [3] #s12#1(::Any, ::Any, ::Any, ::Any) at ./none:0
 [4] (::Core.GeneratedFunctionStub)(::Any, ::Vararg{Any,N} where N) at ./boot.jl:527
 [5] reward(::QuickPOMDPs.QuickPOMDP{UUID("936c9395-802f-48a6-b0da-f1480db0aef3"),String,String,String,NamedTuple{(:stateindex, :isterminal, :obsindex, :states, :observations, :discount, :actions, :observation, :actionindex, :initialstate, :transition, :reward),Tuple{Dict{String,Int64},Bool,Dict{String,Int64},Array{String,1},Array{String,1},Float64,Array{String,1},PyCall.var"#fn#26"{PyCall.var"#fn#25#27"{PyObject}},Dict{String,Int64},POMDPModelTools.Uniform{Set{String}},PyCall.var"#fn#26"{PyCall.var"#fn#25#27"{PyObject}},PyCall.var"#fn#26"{PyCall.var"#fn#25#27"{PyObject}}}}}, ::String, ::String, ::String) at /home/bill/.julia/packages/QuickPOMDPs/ELi8v/src/quick.jl:227
 [6] reward at /home/bill/.julia/packages/POMDPModelTools/cpJHo/src/underlying_mdp.jl:26 [inlined]
 [7] macro expansion at /home/bill/.julia/packages/DiscreteValueIteration/FjeJj/src/vanilla.jl:120 [inlined]
 [8] macro expansion at ./timing.jl:233 [inlined]
 [9] solve(::DiscreteValueIteration.ValueIterationSolver, ::POMDPModelTools.UnderlyingMDP{QuickPOMDPs.QuickPOMDP{UUID("936c9395-802f-48a6-b0da-f1480db0aef3"),String,String,String,NamedTuple{(:stateindex, :isterminal, :obsindex, :states, :observations, :discount, :actions, :observation, :actionindex, :initialstate, :transition, :reward),Tuple{Dict{String,Int64},Bool,Dict{String,Int64},Array{String,1},Array{String,1},Float64,Array{String,1},PyCall.var"#fn#26"{PyCall.var"#fn#25#27"{PyObject}},Dict{String,Int64},POMDPModelTools.Uniform{Set{String}},PyCall.var"#fn#26"{PyCall.var"#fn#25#27"{PyObject}},PyCall.var"#fn#26"{PyCall.var"#fn#25#27"{PyObject}}}}},String,String}; kwargs::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}) at /home/bill/.julia/packages/DiscreteValueIteration/FjeJj/src/vanilla.jl:102
 [10] solve at /home/bill/.julia/packages/DiscreteValueIteration/FjeJj/src/vanilla.jl:67 [inlined]
 [11] solve(::QMDP.QMDPSolver{DiscreteValueIteration.ValueIterationSolver}, ::QuickPOMDPs.QuickPOMDP{UUID("936c9395-802f-48a6-b0da-f1480db0aef3"),String,String,String,NamedTuple{(:stateindex, :isterminal, :obsindex, :states, :observations, :discount, :actions, :observation, :actionindex, :initialstate, :transition, :reward),Tuple{Dict{String,Int64},Bool,Dict{String,Int64},Array{String,1},Array{String,1},Float64,Array{String,1},PyCall.var"#fn#26"{PyCall.var"#fn#25#27"{PyObject}},Dict{String,Int64},POMDPModelTools.Uniform{Set{String}},PyCall.var"#fn#26"{PyCall.var"#fn#25#27"{PyObject}},PyCall.var"#fn#26"{PyCall.var"#fn#25#27"{PyObject}}}}}) at /home/bill/.julia/packages/QMDP/tu6hY/src/vanilla.jl:34
 [12] invokelatest(::Any, ::Any, ::Vararg{Any,N} where N; kwargs::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}) at ./essentials.jl:710
 [13] invokelatest(::Any, ::Any, ::Vararg{Any,N} where N) at ./essentials.jl:709
 [14] _pyjlwrap_call(::Function, ::Ptr{PyCall.PyObject_struct}, ::Ptr{PyCall.PyObject_struct}) at /home/bill/.julia/packages/PyCall/tqyST/src/callback.jl:28
 [15] pyjlwrap_call(::Ptr{PyCall.PyObject_struct}, ::Ptr{PyCall.PyObject_struct}, ::Ptr{PyCall.PyObject_struct}) at /home/bill/.julia/packages/PyCall/tqyST/src/callback.jl:44
 [16] macro expansion at /home/bill/.julia/packages/PyCall/tqyST/src/exception.jl:95 [inlined]
 [17] #109 at /home/bill/.julia/packages/PyCall/tqyST/src/pyfncall.jl:43 [inlined]
 [18] disable_sigint at ./c.jl:446 [inlined]
 [19] __pycall! at /home/bill/.julia/packages/PyCall/tqyST/src/pyfncall.jl:42 [inlined]
 [20] _pycall!(::PyObject, ::PyObject, ::Tuple{Array{String,1}}, ::Int64, ::Ptr{Nothing}) at /home/bill/.julia/packages/PyCall/tqyST/src/pyfncall.jl:29
 [21] _pycall! at /home/bill/.julia/packages/PyCall/tqyST/src/pyfncall.jl:11 [inlined]
 [22] #_#116 at /home/bill/.julia/packages/PyCall/tqyST/src/pyfncall.jl:86 [inlined]
 [23] (::PyObject)(::Array{String,1}) at /home/bill/.julia/packages/PyCall/tqyST/src/pyfncall.jl:86
 [24] top-level scope at none:4
 [25] eval(::Module, ::Any) at ./boot.jl:331
 [26] exec_options(::Base.JLOptions) at ./client.jl:272
 [27] _start() at ./client.jl:506>

Here's my code:

from quickpomdps import *
from julia.POMDPs import solve, pdf
from julia.QMDP import QMDPSolver 
from julia.POMDPSimulators import stepthrough
from julia.POMDPPolicies import alphavectors 
from julia.POMDPModelTools import SparseCat, Uniform, Deterministic
 

def transition_function(s, a):
    if a == 'listen':
        return Deterministic(s)
    else:
        return Uniform(['left','right'])

def observation(s, a, sp):
    if a == 'listen':
        if sp == 'left':
            return SparseCat(['left', 'right'], [0.85,0.15])
        else:
            return SparseCat(['right', 'left'], [0.85, 0.15])
        end
    else:
        return Uniform(['left', 'right'])

def reward(s, a):
    if a == 'listen':
        return -1.0
    elif s == a: # the tiger was found
        return -100.0
    else: # the tiger was escaped
        return 10.0

 

m = QuickPOMDP(states=['left','right'],
    actions=['left','right', 'listen'],
    observations=['left', 'right'],
    initialstate=Uniform(['left','right']),
    discount = 0.95,
    transition=transition_function,
    observation=observation,
    reward=reward 
    )

solver = QMDPSolver()  
policy = solve(solver, m)

print('alpha vectors:')
for v in alphavectors(policy):
    print(v)

print()

rsum = 0.0
for step in stepthrough(m, policy, max_steps=10):
    print('s:', step.s)
    print('b:', [pdf(step.b, x) for x in S])
    print('a:', step.a)
    print('o:', step.o, '\n')
    rsum += step.r

print('Undiscounted reward was', rsum)

Thanks for your time and help!

support pyjulia 0.6

Right now I think our pyproject.toml says we are incompatible with pyjulia 0.6. @rejuvyesh do you know what we need to do to fix this?

Including state dependent actions in model

@zsunberg I've been using quickpomdps successfully with a fixed set of actions. Now I need to use state dependent actions as described in documentation but I get errors when I try to do so. I've tested it by modifying the Light-Dark problem as below:

def actions(s):
    if s > 0:
        return [-12, -2, 0, 1, 10] 
    else:
        return [-10, -1, 0, 2, 12] 

m = QuickPOMDP(
    states = range(-r, r+2),
    actions = actions,
    discount = 0.95,
    isterminal = lambda s: s < -r or s > r,
    obstype = Float64,
    transition = transition,
    observation = observation,
    reward = reward,
    initialstate = Uniform(range(-r//2, r//2+1))
)

and I get the following error when running QuickPOMDP:

TypeError: actions() missing 1 required positional argument: 's'

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.