This is an experimental project that will be subject to lots of changes.

pip install acids-msprior

MSPrior assumes you have

1. A pretrained RAVE model exported without streaming as a torchscript .ts file
2. The dataset on which RAVE has been trained (a folder of audio files).

MSPrior operates on the latent representation yielded by RAVE. Therefore, we start by encoding the entirety of the audio dataset into a latent dataset.

msprior preprocess --audio /path/to/audio/folder --out_path /path/to/output/folder --rave /path/to/pretrained/rave.ts

MSPrior has several possible configurations. The default is a ALiBi-Transformer with a skip prediction backend, which can run in realtime on powerful computers (e.g. Apple M1-2 chips, GPU enabled Linux stations). A less demanding configuration is a large GRU. Both configurations can launched be using

msprior train --config configuration  --db_path /path/to/preprocessed/dataset --name training_name --pretrained_embedding /path/to/pretrained/rave.ts

Here are the different configurations available

The configurations decoder_only and recurrent are readily usable, the seq2seq variants depends on another project called rave2vec that will be open sourced in the near future.

Export your model to a .ts file that you can load inside the nn~ external for Max/MSP and PureData.

msprior export --run /path/to/your/run

WARNING

If you are training on top of a continuous rave (i.e. anything but the discrete configuration), you shoud pass the --continuous flag during export

msprior export --run /path/to/your/run --continuous

Once exported, you can load the model inside MaxMSP following the image below.

Note that additional inputs (e.g. semantic) are only available when using seq2seq models. The last output yields the perplexity of the model.

This work is funded by the DAFNE+ N° 101061548 project, and is led at IRCAM in the STMS lab.