What are good prequisite textbooks for someone interested in machine learning?

As mentioned in the last post, machine learning requires a good amount of background knowledge just to make sense of the topic. The core subject areas are:

• calculus
• linear algebra
• combinatorics / discrete math
  
• probability
• statistics
  
• information theory
• real analysis
• optimization theory
  

The following is a list of basic, introductory textbooks for each of these subjects :

• Calculus and Analytic Geometry by Thomas Finney. This was the plug and chug book we used in my college courses.

• Linear Algebra and its Applications by Gilbert Strang. Gilbert Strang also has another book, Introduction to Linear Algebra, which I haven't read. I'm guessing either would be fine.

• Concrete Mathematics by Ronald L. Graham, Donald E. Knuth, Oren Patashnik.

• A First Course in Probability by Sheldon Ross. Additionally helpful is William Feller's An Introduction to Probability Theory and Its Applications, Vol. 1.
  

• Mathematical Statistics with Applications by Wackerly, Mendenhall and Scheaffer. This is a pretty standard stats book.
  

• Elements of Information Theory by Cover and Thomas. I can't recommend this book highly enough.
  

• For real analysis, Marsden and Hoffman's Elementary Classical Analysis is pretty standard. It's not a great book though. I find it annoying that the proofs are left for the ends of the chapters.

• A First Course in Optimization Theory by Rangarajan K. Sundaram.
  
  
• The Mathematics of Nonlinear Programming by Anthony L. Peressini, Francis E. Sullivan and J.J. Jr. Uhl
  

These books are very basic, at the undergraduate level for the most part. However, if, like me, you weren't a math or engineering major, you will probably find them useful for quickly filling in the gaping holes in your education. Later I'll post some introductory books at the graduate level.