That Sounds Right: Auditory Self-Supervision for Dynamic Robot Manipulation

Project Page

For robot rollouts and data collection: Refer sound-supervised-bot

1. Clone the repo

git clone [email protected]:abitha-thankaraj/audio-robot-learning.git
cd audio-robot-learning/

2. Install the required python packages:

conda env create -f setup/environment.yaml

3. Install the aurl library

pip install -e . 

4. Download the dataset here. Update path to the data_dir in your dataset configs.

To pretrain the self supervised encoder for AuRL:

cd aurl
python pretrain.py dataset=<task_name>/mel ssl=<aurl_variant>

To finetune aurl:

cd aurl
python train.py encoder_dir=<ssl_encoder_save_dir> dataset=<task_name>/mel