For Novice

If you have no idea about Machine Learning and Scientific Computing, I suggest you learn the following materials while you are reading Machine Learning or Deep Learning books. You don’t have to master these materials, but a basic understanding is essential. It’s hard to open a meaningful conversation if the person has no idea about matrix or single variable calculus.

Title Author or Source
Introduction to Algorithms Erik Demaine and Srinivas Devadas
Single Variable Calculus David Jerison
Multivariable Calculus Denis Auroux
Differential Equations Arthur Mattuck, Haynes Miller, Jeremy Orloff, John Lewis
Linear Algebra Gilbert Strang

Theory of Computation, Learning Theory, Neuroscience, etc

Title Author or Source
Introduction to the Theory of Computation Michael Sipser
Artificial Intelligence: A Modern Approach Stuart Russell and Peter Norvig
Pattern Recognition and Machine Learning Christopher Bishop
Machine Learning: A probabilistic perspective Kevin Patrick Murphy
CS229 Machine Learning Course Materials Andrew Ng at Stanford University
Reinforcement Learning: An Introduction Richard S. Sutton and Andrew G. Barto
Probabilistic Graphical Models: Principles and Techniques Daphne Koller and Nir Friedman
Convex Optimization Stephen Boyd and Lieven Vandenberghe
An Introduction to Statistical Learning with application in R Gareth James, Daniela Witten, Trevor Hastie and Robert Tibshirani
Neuronal Dynamics: From single neurons to networks and models of cognition Wulfram Gerstner, Werner M. Kistler, Richard Naud and Liam Paninski
Theoretical Neuroscience: Computational and Mathematical Modeling of Neural Systems Peter Dayan and Laurence F. Abbott
Michael I. Jordan Reading List of Machine Learning Hacker News

Fundamentals of Deep Learning

Title Author or Source
Deep Learning in Neural Networks: An Overview Jürgen Schmidhuber
Deep Learning Book Yoshua Bengio, Ian Goodfellow and Aaron Courville
Learning Deep Architectures for AI Yoshua Bengio
Representation Learning: A Review and New Perspectives Yoshua Bengio, Aaron Courville, Pascal Vincent
Reading lists for new MILA students MILA Lab, University of Montreal
Tutorial on Variational Autoencoders Carl Doersch

Tutorials, Practical Guides, and Useful Software

Title Author or Source
Machine Learning Andrew Ng
Neural Networks for Machine Learning Geoffrey Hinton
Deep Learning Tutorial MILA Lab, University of Montreal
Unsupervised Feature Learning and Deep Learning Tutorial AI Lab, Stanford University
CS231n: Convolutional Neural Networks for Visual Recognition Stanford University
CS224d: Deep Learning for Natural Language Processing Stanford University
Theano MILA Lab, University of Montreal
Caffe Berkeley Vision and Learning Center (BVLC) and community contributor Yangqing Jia
Torch 7 active contributors
neon Nervana
cuDNN NVIDIA
ConvNetJS Andrej Karpathy
DeepLearning4j  
Chainer: Neural network framework Preferred Networks, Inc
Keras fchollet and active contributors
TensorFlow TensorFlow Team
PyTorch PyTorch Team
CoLaboratory Google