michelleho-24 / aa229 Goto Github PK

This project addresses the problem of planning for continuous belief spaces for POMDPs or Partially Observable Markov Decision Processes. Instead of planning in the state space, planning in the belief space allows for planning over all possible distributions of the state space. However, planning in the belief space introduces nonlinear, under-actuated stochastic system dynamics even when using a discrete space, even before introducing other constraints to the problem. Platt et al. (2010) address the system dynamics using an iterative linear quadratic regulator (iLQR) controller to make linear approximations of the dynamics to achieve continuous belief space planning. Chen et al. (2021) build upon this approach by introducing chance constraints and linearizing them with barrier functions to continue using the iLQR approach. However, neither works consider wider classes of constraints such as temporal, non-convex, or hybrid constraints. This work lays the foundation to introduce such wider classes by creating the necessary differential POMDP and belief iLQR packages in Julia. Then, I show that the approach described by Platt et al. can be recreated and performs well against solvers that discretize the belief space. From here, the next step is to recreate the results from Chen et al. and finally introduce the wider class of constraints.