I have been learning ML for sometime now and I have spent some time on finding what are some of the good resources for ML.
For Beginners I would say if you look beyond beginners section you might be overwhelmed by the amount of content. Just relax and eat it bit by bit. This is not a race.
My most visited page was random copied resource list on Machine Learning. As now I have some idea about what all the links are about I will try to reorganize the page with time(Possibly subcategorize).
Prerequisites
 Programming
Before starting I would recommend to learn some programming which also supports Numerical operations like matrix multiplications or basic numerical operations like matrix decompositions etc(Do not worry if you dont know these terms, learn to program and focus on that). I would suggest to learn Python with libraries called Numpy and Scipy. Other alternatives are Matlab/Octave/R. For C++ Armadillo is a great option which seems promising to me.
 AlgorithmsKnowing basic algorithms is essential. It also help you implement ideas to programs. Any algorithms course would be enough. There are few at coursera. Any one of the 2nd or 3rd set is good enough.
Set 1: Algorithmic Thinking
Set 2: Algorithms, Part I & Part II by robert sedgewick(Princeton) at Coursera
Set 3: Algorithms: Design and Analysis, Part 1 & Part 2 by tim roughgarden(Stanford) at Coursera
 Calculus
Calculus is very very essential prerequisite to Machine Learning. You can always come back for lookups. Multivariable Calculus by MIT OCW
Courses at Coursera
https://www.coursera.org/course/m2o2c2
https://www.coursera.org/learn/calculus1
https://www.coursera.org/course/sequence
 ProbabilityAnyone of the three should be good enough. You can always come back for lookups.
Probability Primer by Mathematical Monk at Youtube
Probabilistic Systems Analysis and Applied Probability by MIT OCW
Machine Learning Lectures:
Beginners
 Machine Learning Course by Andrew Ng at Coursera
Thinking about what should be the first course in ML. The only course that comes to my mind is Machine Learning Course by Andrew Ng at Coursera. Practically no prerequisites(maybe Calculus). I highly recommend doing the programming exercises. Keeps your interests going. Not too mathematical. There is also a very short tutorial on Octave/Matlab(Just enough for doing Homeworks).
 Linear Algebra Course by Gilbert Strang
(I believe the best on internet)
We also need to understand about spaces. This is a good place to get the intuition about vector spaces. Video Lectures accompanies a text book. I would also like to suggest lecture notes by vandenberge (pdf) which are great for implementing various matrix factorizations as recursive solution.
 Learning from Data by Abu Mostafa at Caltech
Course is more detailed than ML by Andrew Ng at Coursera. Very clear explanation of the content. Video Lectures accompanies a textbook.
 Linear and Integer Programming by Sriram Sankaranarayanan at Coursera/UoColorado
To start getting a flavour of optimization this is an excellent place to start. This also shows the advantage of abstraction in solving problem.
Beginners++
 Machine Learning by Andrew Ng at Stanford SEE
This is a longer version of classroom course taught by Andrew Ng at Stanford. This course covers a lot of topics not covered in above courses. This courses accompanies an excellent set of lecture notes(Highly recommended). see lecture notes because sometimes lectures are not clearly explained but still is good for introductory ML.
 Probabilistic Graphical Models by Daphne Koller at Coursera
http://openclassroom.stanford.edu/MainFolder/CoursePage.php?course=ProbabilisticGraphicalModels
This course is like marriage of Probability and Graph Theory which is a significant chunk in Machine Learning. It involves efficient inference methods and how graphs help us. The programming assignments are not very easy like previous courses. This is widely used in NLP and Computer Vision.
 Machine learning Lectures by Mathematical Monk
If wondering who is he, then visit http://www.dam.brown.edu/people/jmiller/teaching.html
This is a course covering a lot of methods similar to Machine Learning Course by Andrew Ng. But the methods covered are wider(Also more probabilistic) than Andrew Ng's ML course.
 Introduction to ML This and This by Alex Smola 10701 at CMU
This is a foundation course for PhD students.
 Scalable Machine Learning by Alex Smola
http://alex.smola.org/teaching/berkeley2012/
This course deals with Scalablity issues and jargons in Machine Learning. It is also good for Statistics, Graphical Models, Recommender Systems, Kernels. This course should be complemented with a practical Project considering Scalability issues.
 Introduction to Recommender Systems
Not too difficult course but the whole course is on recommender Systems.
 Machine Learning By Pedro Domingos at Coursera/UoW
This course also covers lot of topics well explained. This could be done independently(Worth considering atleast for the topics not covered in previous course)
 Mining Massive Data Sets by Jure Leskovec at Coursera/Stanford
Deals with scalability issues of ML/Datamining.
 Neural Networks for Machine Learning by Geoffrey Hinton
Great introductory course to Neural Network. This course has included all the recent advanced in NNs. At the end of the course it might get difficult to understand. You might need to complement the end of the course with some ML text on inference(including approximate inference).
 Deep Learning by Yann Lecunn
Deep Learning course dealing with practicality. Doing work on GPU etc.
 Neural Networks by Hugo Larochelle
This is a fast paced course in Neural Networks. Great if you have some background of bits and pieces of NN/Inference/Graphical Models.
 Deep Learning by Nando D. F. Oxford
https://www.cs.ox.ac.uk/people/nando.defreitas/machinelearning/
 Wow course on reinforcement learning by david silverhttps://www.youtube.com/playlist?list=PL5X3mDkKaJrL42i_jhE4Np6E2Ol62Ofa
 Wow course on approximate dynamic programming AKA Reinforcement Learning by Dimitri P. Bertsekas https://www.youtube.com/playlist?list=PLiCLbsFQNFAxOmVeqPhI5er1LGf2L9I4
http://web.mit.edu/dimitrib/www/DP_Slides_2015.pdf
http://web.mit.edu/dimitrib/www/Abstract_DP_Online_Complete.pdf (Beauty)
http://arxiv.org/abs/1405.6757 Proximal RL
 Advanced Inference / Submodular functions course by Jeff Bilmes
https://www.youtube.com/channel/UCvPnLF7oUh4pm575fZcUxg/videos
Shelf Books(pic from Stephen Gould):
ML Book list: (
link to Michel Jordan recommendation)


Standard ML Text 

Pattern Recognition and Machine Learning  Christopher M. Bishop 
The Elements of Statistical Learning: Data Mining, Inference, and Prediction  Trevor Hastie 
Machine Learning: A Probabilistic Perspective  Kevin P. Murphy 
Probabilistic Graphical Models: Principles and Techniques  Daphne Koller 
A Probabilistic Theory of Pattern Recognition Stochastic Modelling and Applied Probability  Gabor Lugosi 


Probability Theory 

Probability and Random Processes  Geoffrey R. Grimmett 
Probability Theory: The Logic of Science  E.T. Jaynes 
Probability: Theory and Examples  Richard Durrett 
A User's Guide to Measure Theoretic Probability  David Pollard 


Statistics 

All of Statistics: A Concise Course in Statistical Inference  Larry Wasserman 
All of Nonparametric Statistics  Larry Wasserman 
Statistical Inference  Roger L. Berger 


Bayesian Theory and Practice 

Bayesian Core: A Practical Approach to Computational Bayesian Statistics  JeanMichel Marin 
The Bayesian Choice  Christian P. Robert 
Bayesian Data Analysis  Andrew Gelman 


Large Sample Theory and Asymptotic Statistics 

A Course in Large Sample Theory  Thomas S. Ferguson 
Elements of LargeSample Theory  E.L. Lehmann 
Asymptotic Statistics  A.W. van der Vaart 




Monte Carlo Statistical Methods  Christian P. Robert 
Introduction to Nonparametric Estimation  Alexandre B. Tsybakov 
LargeScale Inference  Bradley Efron 


Optimization 

Linear Algebra and Its Applications  Gilbert Strang 
Matrix Computations  Gene H. Golub 
Introduction to Linear Optimization  Dimitris Bertsimas 
Numerical Optimization  Jorge Nocedal 
Introductory Lectures on Convex Optimization: A Basic Course  Y. Nesterov 
Convex Optimization  Stephen Boyd 
Nonlinear Programming  Dimitri P. Bertsekas 


Information Theory 

Elements of Information Theory  Thomas M. Cover 
Information Theory, Inference and Learning Algorithms  David J.C. MacKay 


Analysis 

Introductory Functional Analysis with Applications  Erwin Kreyszig 



Dont Look DOWN!!! Will edit and organize stuff after this soon
________________________________________________________________________________
Road map to EP (Minka)
http://www.convexoptimization.com/dattorro/convex_optimization.html
http://www.convexoptimization.com/wikimization/index.php/
EE464: Semidefinite Optimization and Algebraic Techniques
Convex Analysis lecture notes by Nemirovski
http://www2.isye.gatech.edu/~nemirovs/
http://www2.isye.gatech.edu/~nemirovs/OptIII_TR.pdf
http://www2.isye.gatech.edu/~nemirovs/OPTIII_LectureNotes.pdf
http://ocw.mit.edu/courses/electricalengineeringandcomputerscience/6253convexanalysisandoptimizationspring2012/lecturenotes/MIT6_253S12_lec_comp.pdf (DIMITRI P. BERTSEKAS)
Probabilistic Models for Cognition by Noah Good Man and Joshua Tenenbaum
Compressed sensing
http://www.brainshark.com/brainshark/brainshark.net/portal/title.aspx?pid=zCdz10BfTRz0z0#!
More ML books:
http://www.reddit.com/r/MachineLearning/comments/1jeawf/machine_learning_books/
A lot of lectures related to AI by Coursera
https://www.coursera.org/category/csai
A cool intro to machine learning
with python examples
Programming Collective Intelligence: Building Smart Web 2.0 Applications by Toby Segaran
If you want to read a book on ML then read
 The Elements of Statistical Learning(good book freely downloadable)
http://www.stanford.edu/~hastie/local.ftp/Springer/ESLII_print5.pdf
 Pattern Recognition and Machine Learning by Christopher Bishop
 Machine Learning by Tom Mitchell
 Machine Learning A Probabilistic Perspective by Kevin Murphy(I choose this)
some more resources on ML class resources link by Andrew Ng:
https://share.coursera.org/wiki/index.php/ML:Useful_Resources
For optimization read
 Convex Optimization by Stephen Boyd(good book freely downloadable)
 numerical optimization nocedal wright
http://www.stanford.edu/~boyd/cvxbook/bv_cvxbook.pdf
 NonLinear Programming by Dimitri P Bertsekas
For Scalability of Machine Learning read
Scaling Up Machine Learning: Parallel and Distributed Approaches
by Ron Bekkerman, Mikhail Bilenko, John Langford
For Graphical Models
Probabilistic Graphical Models: Principles and Techniques by Daphne Koller
More books:
A very strong ML community:
Andrew Moores slides http://www.autonlab.org/tutorials/
http://videolectures.net/mlss04_bishop_gmvm/ (graphical models and variational methods bishop)
http://videolectures.net/mlss06tw_wainwright_gmvmm/ (GMvariational methodsmessage passing)
http://www.cs.jhu.edu/~jason/tutorials/variational.html (High level explaination)
MATLAB tutorial: basic and advanced:
http://lukstafi.blogspot.de/2013/10/artificialintelligenceuniversity.html
__________________________________________________________________________________
Dont look down.. :P ML and stuff
ML and random stuff... :D
Random bayesian books
Doing Bayesian Data Analysis (Kruschke)
Bayesian Data Analysis (Gelman et al.)
Applied Bayesian Hierarchical Methods (Congdon)
Statistical Rethinking: A Bayesian Course (McElreath)
Bayesian Analysis Made Simple (Woodward)
The BUGS Book (Lunn et al.)
Bayesian Methods (Gill)
Bayesian Ideas and Data Analysis (Christensen et al.)
Bayesian Statistics and Marketing (Rossi et al.)
Introduction to Bayesian Econometrics (Greenberg)
Bayesian Forecasting and Dynamic Models (West and Harrison)
Bayesian Psychometric Modeling (Levy and Mislevy)
Bayesian Models (Hilbe et al.)
LargeScale Inference: Empirical Bayes Methods (Efron)
Handbook of Markov Chain Monte Carlo (Brooks et al.)
Fundamentals of Nonparametric Bayesian Inference (Ghosal and van der Vaart)
Compressed Sensing
https://sites.google.com/site/igorcarron2/cs
http://dsp.rice.edu/cs
http://nuitblanche.blogspot.fr/
UFLDL
http://web.eecs.umich.edu/~honglak/teaching/eecs598/schedule.html
http://www.cs.toronto.edu/~hinton/deeprefs.html
http://deeplearning.stanford.edu/wiki/index.php/Main_Page
Tom Minka's page
http://alumni.media.mit.edu/~tpminka/
http://www.stats.ox.ac.uk/~teh/teaching/npbayes.html#modernbnp
http://mlg.eng.cam.ac.uk/zoubin/course05/index.html
http://mlg.eng.cam.ac.uk/teaching/4f13/1213/ (Machine Learning 2013 cambridge / Murphy/PRML textbook)
http://www.cs.princeton.edu/~blei/courses.html
http://mlg.eng.cam.ac.uk/zoubin/p807/index.html (Image search and modelling)
Advanced Topics in Machine Learning ( subspace learning, manifold learning, subspace clustering, manifold clustering)
http://www.vision.jhu.edu/teaching/learning10/
Short Python writeup.
http://alumni.media.mit.edu/~tpminka/PLE/python/python.html
Hadoop in python
http://www.michaelnoll.com/tutorials/writinganhadoopmapreduceprograminpython/
http://blog.cloudera.com/blog/2013/01/aguidetopythonframeworksforhadoop/
graphics:
http://inst.eecs.berkeley.edu/~cs184/fa12/onlinelectures1.html
http://www.youtube.com/user/raviramamoorthi/videos?view=1&flow=grid
https://graphics.stanford.edu/wikis/cs348b11/Lectures#Goals
Large scale ML and data mining:
http://www.stanford.edu/class/cs224w/
http://www.stanford.edu/class/cs246/handouts.html
http://www.stanford.edu/group/mmds/
http://www.cs.cornell.edu/Courses/cs6784/2010sp/
NETLOGO tutorial
http://iprogrammer.info/programming/otherlanguages/5613gettingstartedwithnetlogo.html