shushman / cs238project.jl Goto Github PK

As I said in #5 , the energyUsageLikelihood method returns a number and not a distribution. The energyMeanSTD() function will be used to return the mean and STD of the sensor energy model. This will be used for directly updating the belief over battery used, using Gaussian addition rules rather than particles.

They apparently allow for compiler optimization BUT must be carefully constructed as they cannot be changed after initialization. This will effect how we are initializing those arrays currently. Static Arrays can be used for:

1. The location - StaticArray{Int64,2} [Probably not useful since 2D]
2. The world map - StaticArray{Bool,2}(size,size)
3. The observed word map - StaticArray{Tuple{Bool,Float64},2}

Since we are doing a finite horizon grid world, I have set the discount factor to 1.0. However, it may still be worth investigating if a non-one discount factor is required for online POMDP solvers.

Make sure that the sensors handle trying to go out-of-bounds in a rollout (https://github.com/Shushman/CS238-Project/blob/master/juliaCode/Sensors.jl#L32)

As you see here (https://github.com/Shushman/CS238-Project/blob/master/juliaCode/UAVpomdp.jl#L179), I assume the energyUsageLikelihood function returns a number, not a distribution (according to what your Sensors.jl considers). Given some value, in our case the battery used, it should calculate the prob. density at that point.

For example, if the value is 4.5 and the energy distribution is 4.5,2.0 then the result should be 1/(sqrt(2*pi)*2)

Simple change but correct your assumptions, if any, that Distributions.Normal works with variance. It actually works with STD. Change the code in Sensors.jl to reflect this

Currently all our rewards are negative as they represent penalties. I think we should either add a positive reward for reaching the goal or penalize a movement action that does not change position. Otherwise, there may not be an incentive for the agent to go towards the goal.

To ensure there is sufficient incentive to explore, we may have to set the cost of entering an NFZ cell to be really high so it does not just go along the path from start to goal, without using any sensors.