Popular Posts

Sunday, April 28, 2013

From Mike Jordan on what people should learn for ML.


From a post from news.ycombinator.com
I personally think that everyone in machine learning should be (completely) familiar with essentially all of the material in the following intermediate-level statistics book:
1.) Casella, G. and Berger, R.L. (2001). "Statistical Inference" Duxbury Press.
For a slightly more advanced book that's quite clear on mathematical techniques, the following book is quite good:
2.) Ferguson, T. (1996). "A Course in Large Sample Theory" Chapman & Hall/CRC.
You'll need to learn something about asymptotics at some point, and a good starting place is:
3.) Lehmann, E. (2004). "Elements of Large-Sample Theory" Springer.
Those are all frequentist books. You should also read something Bayesian:
4.) Gelman, A. et al. (2003). "Bayesian Data Analysis" Chapman & Hall/CRC.
and you should start to read about Bayesian computation:
5.) Robert, C. and Casella, G. (2005). "Monte Carlo Statistical Methods" Springer.
On the probability front, a good intermediate text is:
6.) Grimmett, G. and Stirzaker, D. (2001). "Probability and Random Processes" Oxford.
At a more advanced level, a very good text is the following:
7.) Pollard, D. (2001). "A User's Guide to Measure Theoretic Probability" Cambridge.
The standard advanced textbook is Durrett, R. (2005). "Probability: Theory and Examples" Duxbury.
Machine learning research also reposes on optimization theory. A good starting book on linear optimization that will prepare you for convex optimization:
8.) Bertsimas, D. and Tsitsiklis, J. (1997). "Introduction to Linear Optimization" Athena.
And then you can graduate to:
9.) Boyd, S. and Vandenberghe, L. (2004). "Convex Optimization" Cambridge.
Getting a full understanding of algorithmic linear algebra is also important. At some point you should feel familiar with most of the material in
10.) Golub, G., and Van Loan, C. (1996). "Matrix Computations" Johns Hopkins.
It's good to know some information theory. The classic is:
11.) Cover, T. and Thomas, J. "Elements of Information Theory" Wiley.
Finally, if you want to start to learn some more abstract math, you might want to start to learn some functional analysis (if you haven't already). Functional analysis is essentially linear algebra in infinite dimensions, and it's necessary for kernel methods, for nonparametric Bayesian methods, and for various other topics. Here's a book that I find very readable:
12.) Kreyszig, E. (1989). "Introductory Functional Analysis with Applications" Wiley.


Source: https://news.ycombinator.com/item?id=1055389

http://homepages.inf.ed.ac.uk/sgwater/reading_list.html
http://cocosci.berkeley.edu/tom/bayes.html#general 

Video lectures on functional analysis:
http://www.youtube.com/watch?v=ebesx6pF8mg&list=PLBC73B96341ECF455
 http://www.youtube.com/watch?v=7IIw_U8rv4Q&list=PL2B92DCEAB0A249CD

Monday, April 22, 2013

Probabilistic programming

This is nice a nice post about probabilistic programming.
http://radar.oreilly.com/2013/04/probabilistic-programming.html
http://tm.durusau.net/?cat=413

http://zinkov.com/posts/2012-06-27-why-prob-programming-matters/

workshop and tutorials
http://projects.csail.mit.edu/church/wiki/Probabilistic_Models_of_Cognition
http://projects.csail.mit.edu/church/wiki/Church
http://probabilistic-programming.org/wiki/NIPS*2012_Workshop
http://nbviewer.ipython.org/urls/raw.github.com/CamDavidsonPilon/Probabilistic-Programming-and-Bayesian-Methods-for-Hackers/master/Chapter1_Introduction/Chapter1_Introduction.ipynb

Probabilistic Programming and Bayesian Methods for Hackers Using Python and PyMC
https://github.com/CamDavidsonPilon/Probabilistic-Programming-and-Bayesian-Methods-for-Hackers
Collection of ipython notebooks
https://github.com/ipython/ipython/wiki/A-gallery-of-interesting-IPython-Notebooks