tf-neural-process's Introduction

tf-neural-process

Tensorflow implementation of Neural Process family

Full code and experiment is based on deepmind repository [GIT, LICENSE]
This repository aims to write more generic code and split model from experiment.
Paper:
1. Conditional Neural Process [arxiv]
2. Neural Process [arxiv]
3. Attentive Neural Process [arxiv]

Sample

Jupyter notebook sample is here.

1. Conditional Neural Process

Model definition [GIT]

encoder_output_sizes = [128, 128, 128, 128]
decoder_output_sizes = [128, 128, 1]

model = neural_process.ConditionalNP(encoder_output_sizes, decoder_output_sizes)

Sample image

2. Neural Process

Model definition [GIT]

z_output_sizes = [128, 128, 128, 128]
enc_output_sizes = [128, 128, 128, 128]
dec_output_sizes = [128, 128, 1]

model = neural_process.NeuralProcess(z_output_sizes, enc_output_sizes, dec_output_sizes)

Sample image

3. Attentive Neural Process

Model definition [GIT]

z_output_sizes = [128, 128, 128, 128]
enc_output_sizes = [128, 128, 128, 128]
cross_output_sizes = [128, 128, 128, 128]
dec_output_sizes = [128, 128, 1]

self_attention = neural_process.Attention(attention_type='multihead', proj=[128, 128])
cross_attention = neural_process.Attention(attention_type='multihead', proj=[128, 128])

model = neural_process.AttentiveNP(z_output_sizes,
                                   enc_output_sizes,
                                   cross_output_sizes,
                                   dec_output_sizes,
                                   self_attention,
                                   cross_attention)