Giter VIP home page Giter VIP logo

sogmp's Introduction

SOGMP++/SOGMP: Stochastic Occupancy Grid Map Prediction in Dynamic Scenes

Implementation code for our paper "Stochastic Occupancy Grid Map Prediction in Dynamic Scenes"(arXiv) in Conference on Robot Learning (CoRL) 2023. Two stochastic occupancy grid map (OGM) predictor algorithms (i.e. SOGMP and SOGMP++) implemented by pytorch. Video demos can be found at multimedia demonstrations. Here are three GIFs showing the occupancy grid map prediction comparison results (0.5s, or 5 time steps into the future) of our proposed SOGMP++, SOGMP algorithms, and ConvLSTM, PhyDNet, DeepTracking, and SOGMP_NEMC baselines on three different datasets with different robot models.

  • OGM-Turtlebot2: turtlebot2_prediction_demo
  • OGM-Jackal: jackal_prediction_demo
  • OGM-Spot: spot_prediction_demo

Requirements

  • python 3.7
  • torch 1.7.1
  • tensorboard

OGM-Datasets

The related datasets can be found at OGM-datasets
There are three different datasets collected by three different robot models (i.e. Turtlebot2, Jackal, Spot).

  • 1.OGM-Turtlebot2: collected by a simulated Turtlebot2 with a maximum speed of 0.8 m/s navigates around a lobby Gazebo environment with 34 moving pedestrians using random start points and goal points
  • 2.OGM-Jackal: extracted from two sub-datasets of the socially compliant navigation dataset (SCAND), which was collected by the Jackal robot with a maximum speed of 2.0 m/s at the outdoor environment of the UT Austin
  • 3.OGM-Spot: extracted from two sub-datasets of the socially compliant navigation dataset (SCAND), which was collected by the Spot robot with a maximum speed of 1.6 m/s at the Union Building of the UT Austin

Usage: SOGMP (The inference speed is faster than SOGMP++)

  • Download OGM-datasets and decompress them to the home directory:
cd ~
tar -zvxf OGM-datasets.tar.gz
  • Training:
git clone https://github.com/TempleRAIL/SOGMP.git
cd SOGMP 
git checkout sogmp
sh run_train.sh ~/data/OGM-datasets/OGM-Turtlebot2/train ~/data/OGM-datasets/OGM-Turtlebot2/val
  • Inference Demo on OGM-Turtlebot2 dataset:
git clone https://github.com/TempleRAIL/SOGMP.git
cd SOGMP 
git checkout sogmp
sh run_eval_demo.sh  ~/data/OGM-datasets/OGM-Turtlebot2/test

Usage: SOGMP++ (The prediction accuracy is higher than SOGMP)

  • Download OGM-datasets and decompress them to the home directory:
cd ~
tar -zvxf OGM-datasets.tar.gz
  • Training:
git clone https://github.com/TempleRAIL/SOGMP.git
cd SOGMP 
git checkout sogmp++
sh run_train.sh ~/data/OGM-datasets/OGM-Turtlebot2/train ~/data/OGM-datasets/OGM-Turtlebot2/val
  • Inference Demo on OGM-Turtlebot2 dataset:
git clone https://github.com/TempleRAIL/SOGMP.git
cd SOGMP 
git checkout sogmp++
sh run_eval_demo.sh  ~/data/OGM-datasets/OGM-Turtlebot2/test

Citation

@inproceedings{xie2023stochastic,
  title={Stochastic Occupancy Grid Map Prediction in Dynamic Scenes},
  author={Xie, Zhanteng and Dames, Philip},
  booktitle={Conference on Robot Learning},
  pages={1686--1705},
  year={2023},
  organization={PMLR}
}

@article{xie2023stochastic,
  title={Stochastic Occupancy Grid Map Prediction in Dynamic Scenes},
  author={Xie, Zhanteng and Dames, Philip},
  journal={arXiv preprint arXiv:2210.08577},
  year={2023}
}

sogmp's People

Contributors

zzuxzt avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

Forkers

nuyoah0123 dtbinh

sogmp's Issues

Take too much time

I am running this program on a device with an Nvidia GeForce RTX 3070 GPU and 32GB of memory. And it takes approximately 1.5-1.6s to decode 10 frames. I found that most of the time was spent by the prediction of pytorch(the part for t in range(T):).

It takes around 0.3s for a single for loop of the part for t in range(T):. So if I want to predict 10 frames it's gonna be (1+2+3+...+9+10)*0.03≈1.65s.

Is there any way to make the decode demo run faster? As we want to deploy the program on mobile robots so the real-time performance is highly demanded.

`

        start = time.time()
        # multi-step prediction: 10 time steps:
        for j in range(SEQ_LEN):
            print('----------------------Decode the %dth frame-----------------------------'%(j+1))
            # Create input grid maps:
            input_gridMap = copy.deepcopy(input_gridMap_bak)
            # current position and velocities: 
            obs_pos_N = positions[:, SEQ_LEN-1]
            vel_N = velocities[:, SEQ_LEN-1]
            # Predict the future origin pose of the robot: t+n 
            T = j+1 #int(t_pred)
            noise_std = [0, 0, 0]#[0.00111, 0.00112, 0.02319]
            pos_origin = input_gridMap.origin_pose_prediction(vel_N, obs_pos_N, T, noise_std)
            # robot positions:
            pos = positions[:,:SEQ_LEN]
            # Transform the robot past poses to the predicted reference frame.
            x_odom, y_odom, theta_odom = input_gridMap.robot_coordinate_transform(pos, pos_origin)
            # Lidar measurements:
            distances = scans[:,:SEQ_LEN]
            # the angles of lidar scan: -135 ~ 135 degree
            angles = torch.linspace(-(135*np.pi/180), 135*np.pi/180, distances.shape[-1]).to(device)
            # Lidar measurements in X-Y plane: transform to the predicted robot reference frame
            distances_x, distances_y = input_gridMap.lidar_scan_xy(distances, angles, x_odom, y_odom, theta_odom)
            # discretize to binary maps:
            input_binary_maps = input_gridMap.discretize(distances_x, distances_y)
            # binary occupancy maps:
            input_binary_maps = input_binary_maps.unsqueeze(2)
            # feed the batch to the network:
            num_samples = 32 #1
            inputs_samples = input_binary_maps.repeat(num_samples,1,1,1,1)
            timestamp8 = time.time()
            for t in range(T):
                prediction, kl_loss = model(inputs_samples)
                prediction = prediction.reshape(-1,1,1,IMG_SIZE,IMG_SIZE)
                inputs_samples = torch.cat([inputs_samples[:,1:], prediction], dim=1) 
            timestamp9 = time.time()
            print('pytorch prediction takes %.4f' %(timestamp9 - timestamp8))
            predictions = prediction.squeeze(1)
            # mean and std:
            pred_mean = torch.mean(predictions, dim=0, keepdim=True)
            prediction_maps[j, 0] = pred_mean.squeeze()
        end = time.time()
        print('prediction 10 time steps takes:%.4f s'%(end - start))`

image
I would appreciate for any help.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.