cddu / multimodal-learning Goto Github PK

This repo contains accompanying code for my master's thesis. This includes implementations of a multimodal variational autoencoder (VAE), and incorporates variants of the PixelCNN architecture. The goal is to learn representations from multiple image modalities, and to provide a generative model for realizing plausible, new configurations in data space. See jointvae.py for a multimodal VAE implementation on image data, and see multimodalvae.py for a multimodal VAE implementation on image and language data. Various deep neural network architectures for the VAE are implemented in layers.py.