email me at massimo.p.caccia at gmail.com is you would like to collaborate
- Classics (papers before Deep Learning era. Motivates and sets the stage for Continual Learning)
- Influencials (well cited papers. More general contributions like proposing new frameworks, evaluations, etc)
- Surveys
- Prior-Focused Methods
- Dynamic Architectures Methods
- Rehearsal Methods
- Hybrid Methods
- Meta Continual Learning (learning to continually learn)
- Continual Meta Learning (continually learning to learn)
- Lifelong Reinforcement Learning
- Continual Generative Modeling
- Relevants (non-continual learning papers that can help us e.g. Generalization, Biology, Meta-Learning, etc)
- Unclassified (yet!)
{Family, Single-Head, Task Agnostic, Online, Supervised Learning, Generative Modeling, Reinforcement Learning}
list of interesting codebases for continual learning
Continual Learning (CL), Catastrophic Forgetting (CF), Generative Replay (GR), Continual Meta Learning (CML), Meta Continual Learning (MCL)
Lifelong robot learning (1995)
argues knowledge transfer is essential if robots are to learn control with moderate learning times
Towards Robust Evaluations of Continual Learning (2018) [summary]
Proposes desideratas and reexamines the evaluation protocol
Efficient Lifelong Learning with A-GEM (2018)
More efficient GEM; Introduces online continual learning
Elastic Weight Consolidation (EWC) (2017)
Introduces prior-focused methods
Gradient Episodic Memory (GEM) (2017)
a model that alliviates CF via constrained optimization
Continual Learning with Deep Generative Replay (GR) (2017)
Introduces generative replay
An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networks (2013)
Investigates CF in neural networks
Continual learning: A comparative study on how to defy forgetting in classification tasks (2019)
Extensive empirical study of CL methods (in the multi-head setting)
Continual Learning for Robotics (2019)
Extensive review of CL methods and their applications in robotics and framework proposition for continual learning
Continual Lifelong Learning with Neural Networks: A Review (2018)
A extensive review of CL
Improving and Understanding Variational Continual Learning (2019)
Improved results and interpretation of VCL.
Uncertainty-guided Continual Learning with Bayesian Neural Networks (2019)
Uses Bayes by Backprop for variational Continual Learning.
Uncertainty-based Continual Learning with Adaptive Regularization (2019)
Introduces VCL with uncertainty measured for neurons instead of weights.
Task Agnostic Continual Learning Using Online Variational Bayes (2018)
Introduces an optimizer for CL that relies on closed form updates of mu and sigma of BNN; introduce label trick for "class learning" (single-head)
Overcoming Catastrophic Interference using Conceptor-Aided Backpropagation (CAB) (2018)
"Conceptor-Aided Backprop" (CAB): gradients are shielded by conceptors against degradation of previously learned tasks
Overcoming catastrophic forgetting with hard attention to the task (HAT) (2018)
Introducing a "hard attention" idea with binary masks
Riemannian Walk for Incremental Learning: Understanding Forgetting and Intransigence (2018)
Formalizes the shortcomings of multi-head evaluation, as well as the importance of replay in single-head setup. Presenting an improved version of EWC.
Elastic Weight Consolidation (EWC) (2017)
Introduces prior-focused methods
Memory Aware Synapses: Learning what (not) to forget (2017)
Importance of parameter measured based on their contribution to change in the learned prediction function.
Variational Continual Learning (VCL) (2017)
Introduces the idea of using previous task's posterior as the new task's prior in a BNN.
Synaptic Intelligence (SI) (2017)
Importance of parameter measured based on their contribution to change in the loss.
Learning without Forgetting (2016)
Functional regularization through distillation (keeping the output of the updated network on the new data close to the output of the old network on the new data)
Continual Learning Using Bayesian Neural Networks (2019)
Learns a separate set of posterior distributions for each weight for each task (for a BNN), which are merged using EM updates from time to time (thus posteriors are GMMs)
Incremental Learning Through Deep Adaptation (2018)
Progressive neural networks (2016)
Each task have a specific model connected to the previous ones
Orthogonal Gradient Descent for Continual Learning (2019)
projecting the gradients from new tasks onto a subspace in which the neural network output on previous task does not change and the projected gradient is still in a useful direction for learning the new task
Gradient based sample selection for online continual learning (2019)
sample selection as a constraint reduction problem based on the constrained optimization view of continual learning
Online Continual Learning with Maximaly Interfered Retrieval (MIR) (2019) [summary]
Controlled sampling of memories for replay to automatically rehearse on tasks currently undergoing the most forgetting
Efficient Lifelong Learning with A-GEM (2018)
More efficient GEM; Introduces online continual learning
Generative replay with feedback connections as a general strategy for continual learning (2018)
smarter GR
Gradient Episodic Memory (GEM) (2017)
a model that alliviates CF via constrained optimization (doesn't increase loss on previous stored data)
Continual Learning with Deep Generative Replay (GR) (2017)
Introduces generative replay
Meta-Learning Representations for Continual Learning (MRCL) (2019)
Learns how to continually learn i.e. learns how to do online updates without forgetting.
Learning to Learn without Forgetting by Maximizing Transfer and Minimizing Interference (MER) (2018)
Learns how to update the model such that weight sharing maximises transfer and minimizes interference, via REPTILE
Task Agnostic Continual Learning via Meta Learning (2019)
Introduces What and How framework; enables Task Agnostic CL with meta learned task inference
Online Meta-Learning (2019) [summary]
defines Online Meta-learning; propsoses Follow the Meta Leader (FTML) (~ Online MAML)
Reconciling meta-learning and continual learning with online mixtures of tasks (2018)
Meta-learns a tasks structure; continual adaptation via non-parametric prior
Deep Online Learning via Meta-Learning: Continual Adaptation for Model-Based RL (2018)
Formulates an online learning procedure that uses SGD to update model parameters, and an EM with a Chinese restaurant process prior to develop and maintain a mixture of models to handle non-stationary task distribution
Continual Unsupervised Representation Learning (2019)
Introduces unsupervised continual learning (no task label and no task boundaries)
Generative Models from the perspective of Continual Learning (2018)
Extensive evaluation of CL methods for generative modeling
Reccurent Independant Mechanisms (2019)
A Meta-Transfer Objective for Learning to Disentangle Causal Mechanisms (2019)
propose to meta-learn causal structures based on how fast a learner adapts to new distributions arising from sparse distributional changes, e.g. due to interventions, actions of agents and other sources of non-stationarities
Modular Meta-learning (2018)
Trains different modular (neural nets) structures on a set of related tasks and generalize to new tasks by composing the learned modules in new ways
An Empirical Study of Example Forgetting during Deep Neural Network Learning (2018)
(i) certain examples are forgotten with high frequency, and some not at all; (ii) (un)forgettable examples generalize across neural architectures; and (iii) based on forgetting dynamics, a significant fraction of examples can be omitted from the training data set while still maintaining state-of-the-art generalization performance
Continual learning with hypernetworks (2019)
family = {prior, rehearsal, dynamic, MCL, CML, hybrid}
| Title | Family | Single-Head | Task Agnostic | Online | Supervised | Generative | RL |
|---|---|---|---|---|---|---|---|
| Elastic Weight Consolidation (EWC) | Prior | ✔️ | ✔️ | ||||
| Continual Learning with Deep Generative Replay (GR) | Rehearsal | ✔️ | ✔️ | ||||
| Gradient based sample selection for online continual learning | Rehearsal | ✔️ | ✔️ | ✔️ | ✔️ | ||
| Generative replay with feedback connections as a general strategy for continual learning | Rehearsal | ✔️ | ✔️ | ✔️ | |||
| Task Agnostic Continual Learning via Meta Learning | Meta | ✔️ | ✔️ | ✔️ | |||
| Reconciling meta-learning and continual learning with online mixtures of tasks | Meta | ✔️ | ✔️ | ✔️ | |||
| Task Agnostic Continual Learning Using Online Variational Bayes | Meta | ✔️ | ✔️ | ✔️ | ✔️ | ||
| Meta-Learning Representations for Continual Learning (MRCL) | Meta | ✔️ | ✔️ | ✔️ | ✔️ |
Continual Learning Data Former
A pytorch compatible data loader to create and use sequence of tasks for Continual Learning
Gradient Episodic Memory for Continual Learning
Reproduce paper. Nice repo because baselines and GEM are seamlessly interchangable
Generative Models from the perspective of Continual Learning
Complete repo for experiments in Generative Continual Learning
Online Continual Learning with Maximally Interfered Retrieval
Reproduce paper
Learning to Learn without Forgetting by Maximizing Transfer and Minimizing Interference (MER)
Reproduce paper
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent