[Paper] [Dataset] [BibTeX]

MTDiffuser is a medical image translation model that can handle various translation tasks based on prompt with just on trainning. We train MTDiffuser on 512×512 images from the SynthRAD datasets and Gold Atlas datasets and Gold Atlas datasets. (a-b) MTDiff performs CT-to-MRI, MRI-to-CT, and CBCT-to-CT (from left to right) modality conversion in the head and pelvic region. (c) MTDiffuser also has anatomical consistency in the conversion of continuous slices.

A suitable conda environment named ldm can be created and activated with:

conda env create -f environment.yml
conda activate mtdiff

The Diffuser model is trained on the collection of SynthRAD datasets and Gold Atlas datasets. We have processed the data, such as resampling, cropping, etc. The processed data can be downloaded here. We also have provided some processed data here for quick test.

After download and unzip the data to the user directory, you can specify the data directory in the models/autoencoder/autoencoder_kl_64x64x4.yaml and models/ldm/mtldm_v1_8_128.yaml. Training and testing samples can be specified through files which can be find in jsons/

We currently provide the following checkpoints:

• autoencode_v1_kl_8.ckpt: 120k steps at resolution 512x512 on SynthRAD and Gold Atlas.
• mtldm-v1_8_128.ckpt: 120k steps at resolution 512x512 on SynthRAD and Gold Atlas.