Popular Posts

Monday, May 14, 2012

Machine Learning Resources for Beginners and Beginners++

I have been learning ML for sometime now and I have spent some time on finding what are some of the good resources for ML. 

For Beginners I would say if you look beyond beginners section  you might be overwhelmed by the amount of content. Just relax and eat it bit by bit. This is not a race.


My most visited page was random copied resource list on Machine Learning. As now I have some idea about what all the links are about I will try to reorganize the page with time(Possibly subcategorize). 
Pre-requisites
Machine Learning Lectures:
Beginners
  1.  Machine Learning Course by Andrew Ng at Coursera
    Thinking about what should be the first course in ML. The only course that comes to my mind is Machine Learning Course by Andrew Ng at Coursera. Practically no prerequisites(maybe Calculus). I highly recommend doing the programming exercises. Keeps your interests going. Not too mathematical. There is also a very short tutorial on Octave/Matlab(Just enough for doing Homeworks).
  2.  Linear Algebra Course by Gilbert Strang
    (I believe the best on internet)
    We also need to understand about spaces. This is a good place to get the intuition about vector spaces. Video Lectures accompanies a text book. I would like to suggest lecture notes by
  3. Learning from Data by Abu Mostafa at Caltech
    Course is more detailed than ML by Andrew Ng at Coursera. Very clear explanation of the content. Video Lectures accompanies a textbook.
  4.  Linear and Integer Programming by Sriram Sankaranarayanan at Coursera/UoColorado
    To start getting a flavour of optimization this is an excellent place to start. This also shows the advantage of abstraction in solving problem. 
Beginners++
  1.  Machine Learning by Andrew Ng at Stanford SEE
    This is a longer version of classroom course taught by Andrew Ng at Stanford. This course covers a lot of topics not covered in above courses. This courses accompanies an excellent set of lecture notes(Highly recommended).
    see lecture notes because sometimes lectures are not clearly explained but still is good for introductory ML.
  2.  Probabilistic Graphical Models by Daphne Koller at Coursera
    http://openclassroom.stanford.edu/MainFolder/CoursePage.php?course=ProbabilisticGraphicalModels
    This course is like marriage of Probability and Graph Theory which is a significant chunk in Machine Learning. It involves efficient inference methods and how graphs help us. The programming assignments are not very easy like previous courses. This is widely used in NLP and Computer Vision.
  3. Machine learning  Lectures by Mathematical Monk
    If wondering who is he, then visit http://www.dam.brown.edu/people/jmiller/teaching.html
    This is a course covering a lot of methods similar to Machine Learning Course by Andrew Ng. But the methods covered are wider(Also more probabilistic) than Andrew Ng's ML course.
  4. Introduction to ML This and This by Alex Smola 10-701 at CMU
    This is a foundation course for PhD students.

  5. Scalable Machine Learning by Alex Smola
    http://alex.smola.org/teaching/berkeley2012/
    This course deals with Scalablity issues and jargons in Machine Learning. It is also good for Statistics, Graphical Models, Recommender Systems, Kernels. This course should be complemented with a practical Project considering Scalability issues.
  6. Introduction to Recommender Systems
    Not too difficult course but the whole course is on recommender Systems.
  7. Machine Learning By Pedro Domingos at Coursera/UoW
    This course also covers lot of topics well explained. This could be done independently(Worth considering atleast for the topics not covered in previous course)
  8. Mining Massive Data Sets by Jure Leskovec at Coursera/Stanford
    Deals with scalability issues of ML/Datamining.
  9. Neural Networks for Machine Learning by Geoffrey Hinton
    Great introductory course to Neural Network. This course has included all the recent advanced in NNs. At the end of the course it might get difficult to understand. You might need to complement the end of the course with some ML text on inference(including approximate inference).
  10. Deep Learning by Yann Lecunn
    Deep Learning course dealing with practicality. Doing work on GPU etc.
  11. Neural Networks by Hugo Larochelle
    This is a fast paced course in Neural Networks. Great if you have some background of bits and pieces of NN/Inference/Graphical Models.
  12. Big Data, Large Scale Machine Learning by Langford, Lecunn
    Again a practical course for dealing with scalability,
    online learning etc.
  13. Convex Optimization I  by Stephen Boyd at Stanford SEE

     CVX101 Convex Optimization by Stephen Boyd

    A great course on Convex Optimization. I would not say this is an easy course. But totally worth the effort put in. This also complements a text book freely available.
  14. Convex Optimization II  by Stephen Boyd
    Optimization course covering some other useful topics not covered in previous course.
  15. CS281: Advanced Machine Learning by Ryan Adams at Harvard
    This is a real advanced course on Machine Learning. Covering practically everything a general ML course can have(except really specialized topics).
  16. Advanced Optimization and Randomized Methods by AlexSmola/SuvritSra at CMU
    https://www.youtube.com/playlist?list=PLjTcdlvIS6cjdA8WVXNIk56X_SjICxt0d
    http://people.kyb.tuebingen.mpg.de/suvrit/teach/ee227a/lectures.html
    Advanced optimization covering many new topics missing in basic optimization course. Many of these topics are useful in large scale optimization.
  17. Introduction to robotics
  18. Introduction to linear dynamical systems by Stephen Boyd

  19. Probabilistic graphical models - advanced methods (Spring 2012) by murphy at stanford

  20.  For inference and information theory(Mackay) [Lecture9-Lecture 14 Recommended]:
    This is a course on information theory and inference. After first half the course practically changes to an ML course mostly inference and Neural Networks.
  21. Machine Learning by nando freitas at UBC
  22. Game Theory Part I and Part II at coursera
    Help to build decision theory intuitions and concepts. Also one of the branches of AI.
ML Books, Notes, Links:
Shelf Books(pic from Stephen Gould):


ML Book list: (link to Michel Jordan recommendation)



Standard ML Text
Pattern Recognition and Machine Learning Christopher M. Bishop
The Elements of Statistical Learning: Data Mining, Inference, and Prediction Trevor Hastie
Machine Learning: A Probabilistic Perspective Kevin P. Murphy
Probabilistic Graphical Models: Principles and Techniques Daphne Koller




Probability Theory
Probability and Random Processes Geoffrey R. Grimmett
Probability Theory: The Logic of Science E.T. Jaynes
Probability: Theory and Examples Richard Durrett
A User's Guide to Measure Theoretic Probability David  Pollard


Statistics
All of Statistics: A Concise Course in Statistical Inference Larry Wasserman
All of Nonparametric Statistics Larry Wasserman
Statistical Inference Roger L. Berger


Bayesian Theory and Practice
Bayesian Core: A Practical Approach to Computational Bayesian Statistics Jean-Michel Marin
The Bayesian Choice Christian P. Robert
Bayesian Data Analysis Andrew Gelman


Large Sample Theory and Asymptotic Statistics
A Course in Large Sample Theory Thomas S. Ferguson
Elements of Large-Sample Theory E.L. Lehmann
Asymptotic Statistics A.W. van der Vaart




Monte Carlo Statistical Methods Christian P. Robert
Introduction to Nonparametric Estimation Alexandre B. Tsybakov
Large-Scale Inference Bradley Efron


Optimization
Linear Algebra and Its Applications Gilbert Strang
Matrix Computations Gene H. Golub
Introduction to Linear Optimization Dimitris Bertsimas
Numerical Optimization Jorge Nocedal
Introductory Lectures on Convex Optimization: A Basic Course Y. Nesterov
Convex Optimization Stephen Boyd
Nonlinear Programming Dimitri P. Bertsekas


Information Theory
Elements of Information Theory Thomas M. Cover
Information Theory, Inference and Learning Algorithms David J.C. MacKay


Analysis
Introductory Functional Analysis with Applications Erwin Kreyszig

I will add more soon!!!
Dont Look DOWN!!!  Will edit and organize stuff after this soon
________________________________________________________________________________



http://www.convexoptimization.com/dattorro/convex_optimization.html
http://www.convexoptimization.com/wikimization/index.php/

EE464: Semidefinite Optimization and Algebraic Techniques
Convex Analysis lecture notes by Nemirovski
http://www2.isye.gatech.edu/~nemirovs/
http://www2.isye.gatech.edu/~nemirovs/OptIII_TR.pdf 
http://www2.isye.gatech.edu/~nemirovs/OPTIII_LectureNotes.pdf
http://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-253-convex-analysis-and-optimization-spring-2012/lecture-notes/MIT6_253S12_lec_comp.pdf  (DIMITRI P. BERTSEKAS)

Probabilistic Models for Cognition by Noah Good Man and Joshua Tenenbaum

Compressed sensing

http://www.sms.cam.ac.uk/collection/1117766/#!

http://www.brainshark.com/brainshark/brainshark.net/portal/title.aspx?pid=zCdz10BfTRz0z0#!

More ML books: 
http://www.reddit.com/r/MachineLearning/comments/1jeawf/machine_learning_books/

A lot of lectures related to AI by Coursera
https://www.coursera.org/category/cs-ai


A cool intro to machine learning with python examples
Programming Collective Intelligence: Building Smart Web 2.0 Applications by Toby Segaran

If you want to read a book on ML then read 
  • The Elements of Statistical Learning(good book freely downloadable)
http://www.stanford.edu/~hastie/local.ftp/Springer/ESLII_print5.pdf

  • Pattern Recognition and Machine Learning by Christopher Bishop
  • Machine Learning by Tom Mitchell 
  • Machine Learning A Probabilistic Perspective by Kevin Murphy(I choose this)
some more resources on ML class resources link by Andrew Ng:
https://share.coursera.org/wiki/index.php/ML:Useful_Resources
For optimization read 
  • Convex Optimization by Stephen Boyd(good book freely downloadable)
  • numerical optimization nocedal wright
http://www.stanford.edu/~boyd/cvxbook/bv_cvxbook.pdf
  • Non-Linear Programming by Dimitri P Bertsekas

For Scalability of Machine Learning read

  • Scaling Up Machine Learning: Parallel and Distributed Approaches 

by Ron Bekkerman, Mikhail Bilenko, John Langford

For Graphical Models

  • Probabilistic Graphical Models: Principles and Techniques by Daphne Koller 

More books:

http://www.reddit.com/r/MachineLearning/comments/1jeawf/machine_learning_books/

http://pindancing.blogspot.in/2010/01/learning-about-machine-learniing.html

A very strong ML community:  

http://metaoptimize.com/

http://metaoptimize.com/qa

 Andrew Moores slides http://www.autonlab.org/tutorials/

http://videolectures.net/mlss04_bishop_gmvm/     (graphical models and variational methods bishop)

http://videolectures.net/mlss06tw_wainwright_gmvmm/   (GMvariational methodsmessage passing)

http://www.cs.jhu.edu/~jason/tutorials/variational.html (High level explaination)

MATLAB tutorial: basic and advanced:

http://code.google.com/p/yagtom/

http://lukstafi.blogspot.de/2013/10/artificial-intelligence-university.html

__________________________________________________________________________________

Dont look down.. :P ML and stuff

ML and random stuff... :D

Compressed Sensing

https://sites.google.com/site/igorcarron2/cs
http://dsp.rice.edu/cs
http://nuit-blanche.blogspot.fr/ 

UFLDL
http://web.eecs.umich.edu/~honglak/teaching/eecs598/schedule.html
http://www.cs.toronto.edu/~hinton/deeprefs.html
http://deeplearning.stanford.edu/wiki/index.php/Main_Page

Tom Minka's page 

http://alumni.media.mit.edu/~tpminka/

http://www.stats.ox.ac.uk/~teh/teaching/npbayes.html#modernbnp

http://mlg.eng.cam.ac.uk/zoubin/course05/index.html

http://mlg.eng.cam.ac.uk/teaching/4f13/1213/  (Machine Learning 2013 cambridge / Murphy/PRML textbook)

Gatsby Machine Learning Qualifying Exam Topic List 

http://www.cs.princeton.edu/~blei/courses.html 

http://mlg.eng.cam.ac.uk/zoubin/p8-07/index.html (Image search and modelling)

Advanced Topics in Machine Learning ( subspace learning, manifold learning, subspace clustering, manifold clustering)

http://www.vision.jhu.edu/teaching/learning10/

Short Python writeup.

http://alumni.media.mit.edu/~tpminka/PLE/python/python.html

Hadoop in python  

http://www.michael-noll.com/tutorials/writing-an-hadoop-mapreduce-program-in-python/

http://blog.cloudera.com/blog/2013/01/a-guide-to-python-frameworks-for-hadoop/ 

graphics:

http://inst.eecs.berkeley.edu/~cs184/fa12/onlinelectures1.html

 http://www.youtube.com/user/raviramamoorthi/videos?view=1&flow=grid

https://graphics.stanford.edu/wikis/cs348b-11/Lectures#Goals 

Large scale ML and data mining: 

http://www.stanford.edu/class/cs224w/

http://www.stanford.edu/class/cs246/handouts.html

http://www.stanford.edu/group/mmds/

http://www.cs.cornell.edu/Courses/cs6784/2010sp/ 

 NETLOGO tutorial 

http://i-programmer.info/programming/other-languages/5613-getting-started-with-netlogo.html 

5 comments:

  1. Best Organized resource !!! Thanks

    ReplyDelete
  2. can't ask anything more ............

    ReplyDelete
  3. Well organized resource!! it really save to so much time to find relevant stuff from web .... Nice one!! Thanks!!

    ReplyDelete
  4. I second all the above! Really good. Thanks!

    ReplyDelete
  5. Excellent List. Thank you

    ReplyDelete