Best Introductory Machine Learning Books for The Serious

If you want to get serious in learning machine learning, these books are the classics that cannot go wrong.

Pattern Recognition and Machine Learning by Christopher Bishop, 2006

  • Free PDF from Microsfot
  • Known as the PRML, this 700 page book is for the advanced undergraduates or beginning graduate students.
  • You definitely need some fundamental in math
    • Probability theory (ie. Expectation, Conditional Probability)
    • Linear Algebra (ie. vector and matrixes)
    • A little bit of calculus
  • The book was published in 2006, so it does not have all the new and shiny deep neural network based model, but it provides a very solid understanding in the Bayesian view on machine learning that will benefit you even today.
  • It covers topics in:
    • Linear Regression/Classification
    • Neural Network
    • Kernel Method
    • Graphical Model
    • Mixture Model and Expectation Maximization
    • Approximate Inference
    • Sampling Methods
    • Hidden Markov Model
  • Don’t read this book on Kindle

An Introduction to Statistical Learning with Applications in R by Gareth James, Daniela Witten, Trevor Hastie, Robert Tibshirani, 2014

  • Free PDF
  • This book is specialized for 2 things:
    • It covers all the traditional machine learning methods
    • It uses R
  • The tradition ML models are included:
    • Linear Regression
    • Logistic Regression
    • LDA (linear discriminant analysis), QDS
    • K-Nearest Neighbor
    • Regularization:
      • Ridge Regression
      • Lasso Regression
    • Principal Component Analysis (PCA)
    • Polynomial Regression
    • Tree-based Methods
      • Decision Tree
      • Bagging Trees
      • Random Forest
      • Boosted Trees
    • Support Vector Machine (SVM)
    • Unsupervised learning: K-mean Clustering
  • This book is a great book for everything except Deep Learning because R is not so specialized in Deep Learning frameworks.

Machine Learning: A Probabilistic Perspective by Kevin Murphy, 2012

  • This book is 1000 pages and quite a comprehensive ML that cover so many topics.
  • It is really quite a text book on its own. It will gives you the knowledge from basic probability theory to statistical thinking.
  • It does not just give you an overview all the ML models but it always starts from the fundamental concept and connecting them to the model.
  • It includes basically any class of model that you will encounter.
  • However, this book is quite overwhelming with all the details to just study in one course. It’s more suitable to be taught in multiple course series.
  • It covers both traditional and deep learning methods.
  • I could classify this book as a advanced book for the experts to reference to.

Deep Learning by Ian Goodfellow, Yoshua Bengio and Aaron Courville, 2016

  • Website:
  • This is “the deep learning” book by Ian Goodfellow.
  • This is a bible for many ML practitioners.
  • However, this isn’t a great book for self study according to many reviewers on this book. Sometimes, great researchers might not always be great teachers.

The Elements of Statistical Learning: Data Mining, Inference, and Prediction by Trevor Hastie, Robert Tibshirani and Jerome Friedman

  • Free PDF
  • Another very comprehensive book with more rigorous math.
  • It goes into depth on the derivation on many algorithms.
  • Definitely a great reference for those who want to study applied statistics.
  • It’s cover topics in both traditional and modern machine learning:
    • Linear regression with Lasso (the author of the book is one of the creator of Lasso)
    • Tree Models – boosting and additive trees
    • Neural Network
    • Support Vector Machines
    • K-Nearest Neighbor
    • Unsupervised learning
      • Spectral clustering
      • kernel PCA
      • Matrix factorization
      • Google page rank
    • Random forests
    • Ensemble Learning
    • Graphical models


Many free links are provided for the book but often reading the book in printed copies is much easier to digest, so I am promoting these books as marking for these books. Please buy them if they are useful for your career.

Related Posts

Leave a Reply

Your email address will not be published. Required fields are marked *