Machine Learning Curriculum

Machine Learning is a branch of Artificial Intelligence dedicated at making machines learn from observational data without being explicitly programmed.

Let sacrifice some time to learn Machine Learning so that you can be lazy later.

Machine Learning in General

A computer program is said to learn from experience E with respect to some class of tasks T and performance measure P if its performance at tasks in T, as measured by P, improves with experience E.

• Artificial Intelligence, Revealed a quick introduction by Yann LeCun, mostly about Machine Learning ideas
• How do I learn machine learning? - Quora
• Intro to Machine Learning - Udacity hands on scikit-learn (python) programming learning on core ML concepts
• Machine Learning: Supervised, Unsupervised & Reinforcement - Udacity the 2 instructors are hilarious
• Machine Learning Mastery
• Andrew Ng's Course on Coursera recommended for people who want to know the details of ML algorithms under the hood, understand enough maths to be dangerous and do coding assignments in Octave programming language (Note: Andrew said that you do not need to know Calculus beforehand but he talked about it quite often so you will regret if you don't know Calculus 😆)
• ML Recipes - YouTube Playlist a really nicely designed concrete actionable content for ML introduction
• Machine Learning is Fun Part 1 simple approach to machine learning for non-maths people
• Machine Learning with Python - YouTube Playlist
• Machine Learning Yearning by Andrew Ng
• Machine Learning Crash Course: Part 1
• https://www.kadenze.com/courses/machine-learning-for-musicians-and-artists-iv
• Rules of Machine Learning: Best Practices for ML Engineering

Reinforcement Learning

Building a machine that senses the environment and then chooses the best policy (action) to do at any given state to maximize its expected long-term scalar reward is the goal of reinforcement learning.

• Advanced Topics: RL 2015 (COMPM050/COMPGI13) by David Silver (the guy behind AlphaGo)
• An Introduction Book by Richard S. Sutton and Andrew G. Barto
• Deep Reinforcement Learning: Pong from Pixels
• Lecture 10: Reinforcement Learning - YouTube
• A Survey Paper
• Deep Reinforcement Learning: A Tutorial - OpenAI
• CS 294: Deep Reinforcement Learning, Spring 2017

Deep Learning

A set of machine learning techniques specialized at training deep artificial neural networks (DNN). The simplest kind of DNN is a Multilayer Perceptron (MLP).

• A friendly introduction to Deep Learning and Neural Networks
• A Neural Network Playground Tinker with a simple neural network designed to help you visualize the learning process
• Deep Learning Demystified - Youtube explain inspiration of deep learning from real neurons to artificial neural networks
• Learn TensorFlow and deep learning, without a Ph.D. This 3-hour course (video + slides) offers developers a quick introduction to deep-learning fundamentals, with some TensorFlow thrown into the bargain.
• A Guide to Deep Learning by YN^2 a curated maths guide to Deep Learning
• Practical Deep Learning For Coders Course at Fast.ai taught by Jeremy Howard (Kaggle's #1 competitor 2 years running, and founder of Enlitic)
• Deep learning - Udacity recommended for visual learner who knows some ML, this course provides high level ideas of deep learning, dense intuitive details put in a short amount of time, you will use TensorFlow inside the course
• Deep Learning Resources (Papers, Online Courses, Books) - deeplearning4j.org
• Introduction to Deep Neural Networks - deeplearning4j.org
• NVIDIA Deep Learning Institute because GPU are efficient at training Neural Networks, NVIDIA notices this market !
• Deep Learning Book recommended for math nerds who want to understand the theoretical side
• Unsupervised Feature Learning and Deep Learning
• DeepMind Publications
• DeepLearning.TV - YouTube broad overview of deep learning, no implementation, just pure ideas
• CS224d: Deep Learning for Natural Language Processing
• Deep Learning Summer School, Montreal 2015
• UFLDL Deep Learning Tutorial
• Neural networks class - YouTube Playlist
• http://deeplearning.net/
• https://developer.nvidia.com/deep-learning
• http://neuralnetworksanddeeplearning.com/index.html a hands-on online book for deep learning maths intuition, I can say that after you finish this, you will be able to explain deep learning in a fine detail.
• https://github.com/lisa-lab/DeepLearningTutorials
• https://www.kadenze.com/courses/creative-applications-of-deep-learning-with-tensorflow-i You will implement a lot of things inside TensorFlow such as Autoencoders, Convolutional neural net, Feedforward neural nets, Generative models (Generative Adversarial Networks, Recurrent networks), visualizing the network, etc. You will have lots of assignments to finish. The course director (Parag) is also approachable and active.
• Deep Learning Lectures by Yann LeCun Why not learn Deep Learning from the guy who invented Convolutional nets?
• 6.S094: Deep Learning for Self-Driving Cars a course at MIT
• The Neural Network Zoo a bunch of neural network models that you should know about (I know about half of them so don't worry that you don't know many because most of them are not popular or useful in the present)

Convolutional Neural Networks

DNNs that work with grid data like sound waveforms, images and videos better than ordinary DNNs. They are based on the assumptions that nearby input units are more related than the distant units. They also utilize translation invariance. For example, given an image, it might be useful to detect the same kind of edges everywhere on the image. They are sometimes called convnets or CNNs.

• CS231n: Convolutional Neural Networks for Visual Recognition a course taught at Stanford university
• How Convolutional Neural Networks work - Youtube technical explanation including pooling operations, ReLU, fully connected layer, optimization using gradient descent
• Neural Network that Changes Everything - Computerphile
• A Beginner's Guide To Understanding Convolutional Neural Networks

Recurrent Neural Networks

DNNs that have states. They also understand sequences that vary in length. They are sometimes called RNNs.

• http://karpathy.github.io/2015/05/21/rnn-effectiveness/
• http://colah.github.io/posts/2015-08-Understanding-LSTMs/
• http://www.wildml.com/2015/09/recurrent-neural-networks-tutorial-part-1-introduction-to-rnns/

Glossary

• An overview of gradient descent optimization algorithms
• Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs) an activation function that is trying to be better than its predecessor (ReLU) in a lot of way

Open Source Trained Models

• Self-Driving Car by Udacity
• deepdream inceptionism - a deep model that takes an image and hallucinates animals/buildings from it
• Magenta: Music and Art Generation with Machine Intelligence
• SyntaxNet (Parsey McParseface)
• Neural Storyteller convert image caption into a romantic one
• https://github.com/facebookresearch/deepmask sharp object segmentation on image at pixel-level
• https://github.com/facebookresearch/multipathnet convnet for classifying DeepMask+SharpMask model above
• https://github.com/facebookresearch/fastText Library for fast text representation and classification; Tutorial
• https://github.com/tensorflow/models

Interesting Techniques & Applications

• How do GANs intuitively work? this is my article explaining GANs, I try to be as intuitive as possible, GANs are so awesome that I can't just ignore and not talk about it. You can also watch the official tutorial by Ian Goodfellow.
• https://deepart.io/ transfer image style to other image
• http://www.somatic.io/
• WaveNet: A Generative Model for Raw Audio by DeepMind
• Jukedeck Musical AI
• StackGAN StackGAN: Text to Photo-realistic Image Synthesis with Stacked Generative Adversarial Networks
• Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling 3D-GAN, generating 3D models using GAN

Nice Blogs & Vlogs to Follow

• http://colah.github.io/ this guy knows how to explain
• https://karpathy.github.io/ this guy has taught some courses on Deep Nets
• http://www.wildml.com/ this is like a repository on ML
• https://adeshpande3.github.io/adeshpande3.github.io/
• http://culurciello.github.io/
• Sirajology's YouTube Playlists lots of dense short hilarious introduction to ML
• Two Minute Papers on Deep Learning Playlist
• http://www.leviathan.ai/
• Welch Labs

Libraries and Frameworks

Glancing at their GitHub statistics can give you an estimate for how active/popular each of them is.

• Python Deep Learning Frameworks Reviewed 2017 read this to decide which framework is appropriate for you
• scikit-learn (Python) general machine learning library, high level abstraction, geared towards beginners
• TensorFlow (Python); Learning TensorFlow; Installing on Windows; Fresh Install on Ubuntu 16.04; Serving; Awesome; computation graph framework built by Google, has nice visualization board, probably the most popular framework nowadays for doing Deep Learning
• Theano (Python) another popular deep learning framework
• Lasagne (Python) Lightweight library to build and train neural networks in Theano
• Keras: Deep Learning library for Theano and TensorFlow (Python)
• Caffe (Python) originally created to tackle computer vision problems
• Microsoft Cognitive Toolkit (CNTK) Microsoft's framework (previously known as Computational Network Toolkit)
• Torch (LuaJIT) the most popular scientific computing framework for LuaJIT
• PyTorch (Python) PyTorch is a deep learning framework that puts Python first.
• MXNet: A Scalable Deep Learning Framework supports multiple language interfaces
• MinPy (Python) NumPy interface with mixed backend execution (MXNet, autograd)
• Chainer (Python) A flexible framework of neural networks for deep learning
• DeepLearning4j (Java) not so popular, preferable for you if you like Java
• Bonsai (Inkling) a simplification layer for machine learning
• bitfusion - Software to Manage Deep Learning & GPUs contains Amazon Machine Images for many Deep Learning libraries including TensorFlow

Cutting-Edge Research

Steal the most recent techniques introduced by smart computer scientists (could be you).

• https://research.facebook.com/ai/
• http://research.google.com/pubs/MachineIntelligence.html
• https://www.openai.com/
• https://www.openai.com/requests-for-research/
• State of the art performance on each ML task
• http://www.gitxiv.com/

Practitioner Community

• https://www.kaggle.com
• https://gym.openai.com
• https://universe.openai.com/
• /r/MachineLearning
• https://www.facebook.com/groups/DeepNetGroup/

Thoughtful Insights for Future Research

• Pedro Domingos: "The Master Algorithm" - Talks at Google
• The AI Revolution: The Road to Superintelligence
• https://ai100.stanford.edu/2016-report
• Why does Deep Learning work so well? - The Extraordinary Link Between Deep Neural Networks and the Nature of the Universe
• These are three of the biggest problems facing today's AI
• Four Questions For: Geoff Hinton Geoff Hinton is referred to as "godfather of neural networks"
• What product breakthroughs will recent advances in deep learning enable? - Quora

Uncategorized

• Artificial Intelligence: A Modern Approach (Online Book)
• The Principles of Modern Game AI
• Scipy Lecture Notes
• https://www.youtube.com/user/aicourses
• The Fundamentals of Neuroscience learn how our brain works so that you can discover new deep learning breakthrough

Other Big Lists

• https://github.com/ZuzooVn/machine-learning-for-software-engineers
• https://github.com/josephmisiti/awesome-machine-learning
• https://github.com/ujjwalkarn/Machine-Learning-Tutorials
• https://github.com/terryum/awesome-deep-learning-papers
• https://github.com/ChristosChristofidis/awesome-deep-learning
• https://github.com/Developer-Y/cs-video-courses#machine-learning
• Open Source Deep Learning Curriculum an open source deep learning curriculum which contains a lot of course recommendations for you to consider enrolling. (Many recommendations are the same as in this list)
• Deep Learning Resources by Jeremy D. Jackson

I am confused, too much links, where do I start?

Well, if you are really a beginner and want some concrete suggestions from me please read this issue: off99555#4

Disclaimer

This is a really big list because I also point to other people's list to ensure that most of the resources are accessible from this page without you looking anywhere else.

NOTE: There is no particular rank for each link. The order in which they appear does not convey any meaning and should not be treated differently.

How to contribute to this list

1. Fork this repository, then apply your change.
2. Make a pull request and tag me if you want.
3. That's it. If your edition is useful, I'll merge it.

Or you can just submit a new issue containing the resource you want me to include if you don't have time to send a pull request.

The resource you want to include should be free to study.