Thanks to all for their kind birthday wishes.
As a hobbit birthday present, I'd like to recommend my favorite CS and math books:
- Leon, S. J. (2005). Linear Algebra with Applications, 7th edition. Englewood Cliffs, NJ: Prentice Hall.
I used the third edition in the spring of 1990, when I took linear algebra from Roger Horn in the Department of Mathematical Sciences (now Applied Math and Statistics) at Johns Hopkins.
- Bogart, K. P. (1988). Discrete Mathematics. Boston, MA: D. C. Heath & Co.
A nice and simple introduction. I used this in the fall of 1990, when I took discrete math with Michael Schneider in the Department of Mathematical Sciences (now Applied Math and Statistics) at Johns Hopkins. Here's a link to the latest offering of 550.171.
- Lewis, H. R. & Papadimitriou, C. H. (1997). Elements of the Theory of Computation, 2nd edition. Englewood Cliffs, NJ: Prentice Hall.
I used the first edition (1981) in the fall of 1990, when I took Computational Models (then 600.171) with Mike Goodrich in the Computer Science Department at Johns Hopkins. It was easy to understand, and as a result, I gained a solid foundation and did very well in both 600.171 and Lenny Pitt's CS 375 (Automata Theory, Formal Languages, and Theory of Computation) in the Department of Computer Science (DCS) at the University of Illinois at Urbana-Champaign. Here's a link to the latest offering of 600.271.
- Abelson, H., Sussman, G. J., & Sussman, J. (1996). Structure and Interpretation of Computer Programs, 2nd edition. Cambridge, MA: MIT Press.
I used the first edition of SICP, aka the Purple Book, in the spring of 1991, when I took Programming Techniques (600.140) from Amy Zwarico in the Computer Science Department at Johns Hopkins. Later I worked for Amy as a grader for PT. Note that the entire SICP book is available online.
- Cormen, T. H., Leiserson, C. E., Rivest, R. L., & Stern, C. (2001). Introduction to Algorithms, 2nd edition. Cambridge, MA: MIT Press.
I first used the first edition of CLR (1990), aka the White Brick, in the fall of 1991, when I took Algorithms I (600.463) from Mike Goodrich in the Computer Science Department at Johns Hopkins. Here's a link to Mike's 1996 offering of the course. Later, in 1993 and 1994, I used it extensively in preparing for the Ph.D. comprehensive exams in theory of computation in the Department of Computer Science (DCS) at the University of Illinois at Urbana-Champaign.
- Ross, S. (2005). A First Course in Probability, 7th edition. Englewood Cliffs, NJ: Prentice Hall.
I used the third edition in the fall of 1991, when I took Intro to Probability with John Wierman in the Department of Mathematical Sciences (now Applied Math and Statistics) at Johns Hopkins.
- Russell, S. J. & Norvig, P. (2003). Artificial Intelligence: A Modern Approach, 2nd edition. Englewood Cliffs, NJ: Prentice Hall.
Ole Jakob Mengshoel introduced me to the first edition of this in the fall of 1995, when we were getting ready for the oral Ph.D. qualifying exam in artificial intelligence in the Department of Computer Science (DCS) at the University of Illinois at Urbana-Champaign. I taught Intro to AI with the first edition for three years (2000-2002) and am now on the sixth iteration of the course and the third iteration (2003-2005) using the second edition.
Honorable mention goes to:
- Nagle, H. T., Carroll, B. D., & Irwin, D. (1975). An Introduction to Computer Logic. Englewood Cliffs, NJ: Prentice Hall.
This is the textbook used by Gerard G. L. Meyer, the ultra-sarcastic French EE professor who taught digital logic at Johns Hopkins when I was a first-year student there in spring, 1990. I learned the material, and I didn't have to "paint the book yellow" (the preferred undregrad method of study according to Dr. Meyer) or "xerox it and inhale toner dust" (the preferred graduate method of study)!
- Baase, S. (1988). Computer Algorithms: Introduction to Design and Analysis, 2nd edition. Reading, MA: Addison-Wesley.
I used the second edition when I worked on an independent study project on virus detection with Steve Salzberg in the Computer Science Department at Johns Hopkins. It's where I first learned about dynamic programming, by reimplementing Algol-like pseudocode (pseudo-Pascal) for k-approximate string matching. By the way, get rid of all your "in C" and "in Java" books, especially on data structures and algorithms, and go back to the Algol-based ones. Making algorithms work in a particular programming language is part of learning them; be lazy about this and you'll learn less.
Later, I took Algorithms II with S. Rao Kosaraju and used this book as well. I've read a lot of bad reviews of the third edition (Baase and van Gelder), but if it's bad, they must have made some bad revisions, because the second edition was good. I think that the second edition gets a bum rap because it's out of print.
- Bentley, J. (1999). Programming Pearls, 2nd edition. Reading, MA: Addison-Wesley Professional.
This is an anthology of Jon Bentley's "Programming Pearls" column for Communications of the Association for Computing Machinery (CACM), the flagship magazine for the premier CS society in the United States. I first came across the first edition of this book in the fall of 1990, just browsing the
QA.76section down in the Milton S. Eisenhower library (MSEL) at Hopkins. I don't think I'd be exaggerating if I said it changed my career. After reading it, I became interested in data compression ("Squeezing Space"), formal verification (the binary search chapter), implementation of algorithms (from almost every chapter), and the craft of Computational Science and Engineering (from chapters such as "The Back of the Envelope").
- Bentley, J. (1988). More Programming Pearls: Confessions of a Coder. Reading, MA: Addison-Wesley Professional.
I found the second book of this two-volume series around the same time as the first, and in spring 1991, I went back and got it. I don't think I ended up reading it until about two years later. Hopkins was absolutely wonderful to borrowers about letting us keep books; all we had to do was request a "mass renewal" every 4 weeks and we got everything renewed. I kept about 70 books on my shelves for a year at a time, some for half the time I spent at JHU. (Well, at least I read them!)
- Tanenbaum, A. S. & Woodhull, A. S. (1997). Operating Systems: Design and Implementation, 2nd edition. Englewood Cliffs, NJ: Prentice Hall.
I have only used the first edition of this book, written by Andrew Tanenbaum alone, but IMHO, it's a great way to learn operating systems. Here's a link to the author's textbook page.
I took Greg Sullivan's OS course in the Computer Science Department at Johns Hopkins in spring, 1991. Later, I took CS 323 from H. George Friedman (then the undergrad program chair) in the Department of Computer Science (DCS) at the University of Illinois at Urbana-Champaign. I drove my TAs nuts, but between the two courses, I think I learned about what people learn in first courses in OS today, and became proficient in C and C++ for my pains. In both courses, I referred to this book, which was the official text for Greg's course.
Of all the textbooks with a "disk in the back", this one is the best I've used. The first edition came with an order card for MINIX, and my friend Tolga Soyata and I used both the PC (80286/386) and Macintosh/Mac II (68000/68020) versions.
- Giarratano, J. C. & Riley, G. D. (2005). Expert Systems: Principles and Programming, 4th edition. Boston, MA: Thompson Course Technology.
This book, aka the CLIPS Book, is a pretty good introduction to expert systems. I used the second edition in spring, 1991, when I took Steve Salzberg's mixed (undergraduate and graduate) course on expert systems in the Computer Science Department at Johns Hopkins. For some reason, taking Programming Techniques in Scheme, Expert Systems in CLIPS, and Operating Systems in C++, while also taking Differential Equations, made everything click for me. I think seeing things from every angle helped; nearly being driven crazy by the workload didn't hurt, either.
- Schmidt, D. A. (1986). Denotational Semantics: A Methodology for Language Development. Allyn and Bacon.
Amy Zwarico was a great instructor, and after taking her Scheme-based Programming Techniques class in the Computer Science Department at Johns Hopkins in spring, 1991, I decided to take her cross-listed Programming Language Theory (PLT, 600.325/425) in the fall. Amy's course resembled a 50%/50% hybrid of CIS 505 and CIS 705 here at Kansas State, so it was very challenging as a general course in programming languages. We used Standard ML of New Jersey and the required textbook was Elements of Functional Programming by Chris Reade. Paulson's ML for the Working Programmer was a recommended text, as were The Definition of Standard ML and Commentary on Standard ML. I was probably the only undergraduate who bought Definition and Commentary and read all three.
In any case, I found that I was missing some understanding of semantics, so I went looking for textbooks, and came across Dave Schmidt's first book, which he wrote as a postdoc, somewhere down on C-level in the Milton S. Eisenhower library (MSEL) at Hopkins. It was a little hard to follow, but I figured at the time that I just didn't understand the mathematical foundations well enough yet (I was right). After reading a few chapters, though, I was able to finish the problem set that Amy gave on operational, transition, and denotational semantics. I remember thinking "wow, this guy is really good!" and "wow, this stuff is really hard!"
About eight years later, I was interviewing for a faculty position, and I met the guy who wrote one of the undergrad reference texts that I used for a homework I didn't understand. It was a little intimidating, let me tell you!
- Brooks, F. P. (1995). The Mythical Man-Month, 20th Anniversary Edition. Reading, MA: Addison-Wesley Professional.
This book is a classic of software engineering, and it's one that I recommend every computer scientist re-read every 3-5 years. It describes the process, challenge, and pitfalls of creating large software systems and products as a team: how to anticipate and cope with delays, how to design against errors, and how to stay organized. I read the original in the spring of 1992 and the fall of 1994, and read the second edition in the fall of 2000, so I'm due for a refresher.
- Pearl, J. (1988). Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. San Mateo, CA: Morgan Kaufmann.
The classic of theory and research on Bayesian networks, this book by Judea Pearl remains the foremost one on probabilistic reasoning to this day. As Daphne Koller noted in 2001, it is getting a bit dated, and a new textbook is needed that covers new representations, inference and learning algorithms, and advances in graphical models of probability. I still recommend this book as the place to start, though. I was fortunate enough to pick it up at Reiters' bookstore in Washington, D.C. way back in spring, 1992. It made me take Simon Kasif's Advanced AI class (600.435), then go to work for him and Art Delcher on a new protein folding project that summer. This in turn led to some good results: in a couple of months I was able to code up the first implementation of a pure hidden Markov model (HMM)/dynamic Bayesian network (DBN)-based predictor for ab initio protein secondary fold, which I based on an HMM design that my uncle (Kai-Fu Lee) used in his dissertation and told me about. Art improved this and got the best results of the time. This in turn led to a AAAI paper and an ISMB one (my first conference publications), and probably played a role in my getting into UIUC.
Oh, and Pearl autographed my copy of the book on 10 Nov 1992, when he came to speak at JHU. I was late to Computer Graphics because I got in line to talk to him. :-)
- Lee, K.-F. (1988). Automatic Speech Recognition: The Development of the SPHINX Recognition System. Boston, MA: Kluwer Academic Publishers.
My uncle gave me a copy of this book in June, 1988, before his commencement at CMU. (Yes, it's autographed, too. ;-))
As I mentioned above in my comment on Pearl's book, I used the material on HMMs to complete an implementation for Simon Kasif that ended up being my most-cited publication to date (it's actually got a lot more citations than are listed by either CiteSeer or Google Scholar, due to my name being spelt in three different ways).
I now use the speech recognition material and the parts about Baum-Welch in my Intro AI and Machine Learning and Advanced AI courses.
- Foley, J. D., van Dam, A., Feiner, S. K., & Hughes, J. F. (1995) Computer Graphics: Principles and Practice, 2nd edition in C. Reading, MA: Addison-Wesley.
This is the only book for which I recommend the "in C" edition, and only because it's nearly identical to the out-of-print pseudo-Algol second edition from 1990. I used that when I took my first computer graphics course in fall, 1992.
Incidentally, following a classmate who inscribed this on his or her cover, I call this the "Time-Life Book on Computer Graphics" (the reason for which is evident if you take off the cover jacket and look at the full title: THE SYSTEMS PROGRAMMING SERIES). My baby cousin Jennifer almost brained herself pulling it off the shelf, which is no mean feat for an 11-month old, if you have ever used this book.
- West, D. B. (2000). Introduction to Graph Theory, 2nd edition.
I took Ed Scheinerman's course on Graph Theory in the Department of Mathematical Sciences (now Applied Math and Statistics) at Johns Hopkins in spring, 1993. Ed was a Ph.D. student of Doug West at Brown, before West went to UIUC, and he ended up using a pilot version of the first edition of this book in our course. We only had to pay copy costs.
Oddly enough, I never met West in the six years that I lived in Champaign-Urbana, IL and the five years that I attended UIUC. About a year after I came here in 1999, I found a copy of the first edition in a remnants bin of our department library. I didn't see it as a very good sign of the regard in which graph theory was held at the time in KSU-CIS.
- Hopcroft, J. E., Motwani, R., & Ullman, J. D. (2001). Introduction to Automata Theory, Languages, and Computation, 2nd edition. Reading, MA: Addison-Wesley.
I started using the first edition of this book in fall, 1993, when I took Lenny Pitt's CS 375 (Automata Theory, Formal Languages, and Theory of Computation) in the Department of Computer Science (DCS) at the University of Illinois at Urbana-Champaign. Later, I studied for the theory comps with the book, but the emphasis was more on CS 373 (Algorithm Analysis and Design).
Here is the home page for the textbook.
- Shavlik, J. & Dietterich, T. G. (1990). Readings in Machine Learning. San Mateo, CA: Morgan Kaufmann.
I used this anthology of papers extensively in the 1990s, especially starting in fall, 1993. The first time I saw it was in the spring of 1992, when I saw it on Steve Salzberg's shelf. That year, I told him I wanted to take his 600-level machine learning course. He told me it was a graduate-only seminar and recommended that I wait a couple of years, so I did. I took David C. Wilkins's course (CS 342, Knowledge Acquisition and Computer Inference) in fall, 1993 and Larry Rendell's machine learning course (CS 346) in fall, 2005. Larry was very encouraging. I ended up reading most of the papers from this book over those 2-3 years and used it to create my own machine learning course in 1999 when I came to Kansas State.
- Rosen, K. (1999). Discrete Mathematics and Its Applications, 4th edition. New York, NY: McGraw-Hill.
I used the second edition of this book in spring, 1994, in studying for the theory comps in the Department of Computer Science (DCS) at the University of Illinois at Urbana-Champaign. Later I glanced through it while working as a TA for CS 125 (Intro to Computer Science). I think it was used in CS 273 (the second of a three-course undergrad sequence in theory, probably equivalent to an advanced version of CIS 301 here, or between CIS 301 and Math 510).
Here is the home page for the textbook.
- Kamin, S. M. (1990). Programming Languages. Reading, MA: Addison-Wesley.
I used this book from fall, 1993 onwards, especially when working as a research assistant in one of the programming languages groups in the Department of Computer Science (DCS) at the University of Illinois at Urbana-Champaign and as Sam Kamin's TA for CS 325, Programming Languages. The course was roughly like CIS 505 here at KSU, with a little bit of CIS 705 thrown in (maybe 75%/25%).
- Rich, E. & Knight, K. (1991). Artificial Intelligence, 2nd edition. New York, NY: McGraw-Hill.
In fall, 1994, when I took the AI comps, almost all of the questions came from here. It's quite good, and I still use it even though Russell and Norvig is a better overall textbook. If I had to recommend just two AI textbooks, this would be the second. I mentioned this to Kevin Knight in August, 2003 when I met him at IJCAI in Acapulco.
I once missed the chance to buy this book for $11 in spring, 1994, in a Library of Computing and Information Sciences (LCIS) book sale. I swore never to miss the chance again, but didn't get a copy until I requested my examination copy as a new faculty member at KSU.
- Graham, R. L., Knuth, D. E., & Patashnik, O. (1994). Concrete Mathematics: A Foundation for Computer Science, 2nd edition. Reading, MA: Addison-Wesley Professional.
This book wasn't a textbook or recommended reference of any course I ever took, nor was it even on the reading list for our theory comps, that I can recall; however, it's a great book for undergrads and new grads in CS. I bought it for zengeneral last Christmas and I learned a lot from the first edition when I came across it in the summer of 1994 and afterwards. Here is a link to the textbook page.
- Hennessy, J. L, & Patterson, D. A. (2002). Computer Architecture: A Quantitative Approach, 3rd edition. San Mateo, CA: Morgan Kaufmann.
In the fall of 1994, I audited the first half of Dan Reed's excellent computer architecture course in the Department of Computer Science (DCS) at the University of Illinois at Urbana-Champaign, until my TA duties for Intro AI (CS 348) and Programming Languages (CS 325) swamped me. The second edition of this book was the textbook, and it was very good. I still have it.
- Haykin, S. (1998). Neural Networks: A Comprehensive Foundation, 2nd edition. Englewood Cliffs, NJ: Prentice Hall.
I came across the first edition of this book in fall, 1995 while taking CS 448 (Artificial Neural Networks) from my Ph.D. advisor, Sylvian Ray, in the Department of Computer Science (DCS) at the University of Illinois at Urbana-Champaign. At the time, his textbook was by Hossain. In 1996 and 1997 I sat in on his course twice more, and he had switched to Zurada's textbook. In 1998 he switched to Haykin on my recommendation.
- Goldberg, D. E. (1989). Genetic Algorithms in Search, Optimization, and Machine Learning. Reading, MA: Addison-Wesley Professional.
I didn't actually read this book until spring, 1997, when I audited Dave Goldberg's GA course and he became one of my Ph.D. committee members. The first time I saw it was in fall, 1993, when I was a first-year grad student in the Department of Computer Science (DCS) at the University of Illinois at Urbana-Champaign. I was in Dave Wilkins's CS 342 (Knowledge Acquisition and Computer Inference) course and got this book to use as a reference, but didn't quite understand it. Goldberg, as you may know, is a terrific lecturer and a very good writer, and I learned a tremendous amount from both his course and his book.
- Cover, T. M. & Thomas, J. A. (1991). Elements of Information Theory. New York, NY: Wiley-Interscience.
This book is a classic, and a must for just about any computer scientist. If you want to study algorithm analysis and design, theory of computation, artificial intelligence, machine learning, cognitive science, applied probability, computer architecture, computational science and engineering, numerical analysis, or bioinformatics, it's an absolute must. You owe it to yourself to pick up some rudiments of information theory (IT). I first saw the book at Hopkins but didn't read it until spring, 1997, when I audited P. R. Kumar's course in the Department of Electrical and Computer Engineering (ECE) at the University of Illinois at Urbana-Champaign. I had studied a little IT on my own before taking Larry Rendell's course, and he seemed impressed, but after I read this book, it all clicked.
- Mitchell, T. M. (1997). Machine Learning. New York, NY: McGraw-Hill.
This is hands-down the best book on machine learning. It's one of the best in artificial intelligence and in computer science in general, and would be on my short list above if it didn't deal in a relatively specialized topic. I first used it after it came out in fall, 1997, during my last year of my Ph.D. program in the Department of Computer Science (DCS) at the University of Illinois at Urbana-Champaign.
- Witten, I. H. & Frank, E. (2005). Data Mining: Practical Machine Learning Tools and Techniques, 2nd edition. San Mateo, CA: Morgan Kaufmann.
This is "the WEKA book", and has come to be one of the premier data mining textbooks in just five years since the first edition was published. As Peter Norvig wrote, I was a big fan of the first edition, and I'm excited about the second. It is an excellent comprehensive treatment of applied machine learning and covers both practical and theoretical aspects well. As a complement to Mitchell's book, it provides a firm foundation.
I ordered this book as an examination copy for CIS 690 (Data Mining Systems) in summer, 2000, and first used it as a textbook in CIS 830 (Advanced Topics in Artificial Intelligence) in spring, 2001. I received the second edition early this summer, about two weeks into my data mining course (CIS 690), or I would have used it as my official textbook.
- Neapolitan, R. E. (1990). Probabilistic Reasoning in Expert Systems: Theory and Algorithms. New York, NY: John Wiley and Sons.
This book went out of print quickly, apparently due to low sales, which is a shame, because it is excellent. It covers basics of Pearl's propagation algorithm for singly-connected Bayesian networks, and the Lauritzen-Spiegelhalter (LS) junction tree algorithm, better than the seminal works by Pearl and by Lauritzen and Spiegelhalter themselves. In fact, I haven't found a better treatment for beginners to this day.
I first came across it in 1992 when I was an undergraduate at Johns Hopkins, and finally got to reading it about nine years later, in spring, 2001. My students, Laura Kruse, Ben Perry, Julie Thornton, and Haipeng Guo (hpguo), used it to create the first implementation of the junction tree algorithm in BN-Tools, now called Bayesian Network tools in Java (BNJ) and about to be released in its fifth incarnation (BNJ v4).
- Goldberg, D. E. (2002). The Design of Innovation: Lessons Learned from and for Competent Genetic Algorithms. Boston, MA: Kluwer Academic Publishers.
I hesitated to recommend this book as a general CS text, because it covers such an apparently narrow topic. If you actually read it, though, you'll find that it's a great vade mecum for engineers in general, from one who has a tremendously fine reputation. David is really a master at making the case for elegant design principles that give rise to good science: these include automation of experiments, flexibility in design, decomposition of complex problems, isolation of tunable parameters, and development of rate theories and similar analytical techniques for convergence analysis in optimization and search. I've always admired his ability to foster excellent research, too: if you look at his books, nearly every chapter has provided a substrate for at least one breakthrough and a prominent dissertation in the field. Dave has helped launch the careers of many active GEC researchers, including my own.
Oh, and he's great at telling stories and jokes, too. If you've ever heard his deadpan delivery of Gary Bradshaw's speculation that the Wright brothers were the first to fly because they were bachelors, you know what I mean.
I pre-ordered TDOI in spring, 2002, and Ben Perry picked it up for me at GECCO-2002, where he went to present one of our papers (which, incidentally, was of two we had nominated for Best of GECCO that year).
Books I don't recommend:
- Horowitz and Sahni: Fall, 1989
- Sedgewick: Fall, 1991
- Reade: Fall, 1991
- Aho, Hopcroft and Ullman: Spring, 1992
- That stupid McGraw-Hill diff eq book: Spring, 1992
- Stoer and Bulirsch: Fall, 1992
- Deitel and Deitel, OS: Fall, 1993
- Sethi: Spring, 1994
- Winston, 3e: Spring, 1994
Books I am undecided on:
- Vector Calculus, Marsden and Tromba: Fall, 1989
- The Industrial Book (ML for the Working Programmer): Fall, 1991
- Stevens: Spring, 1992
- The Postman Book: Spring, 1992
- Bogart, Introductory Combinatorics: Fall, 1992
- Bondy and Murty: Spring, 1993
- Berge: Spring, 1993
- Garey and Johnson: Spring, 1994
- Friedman, Wand, and Haynes, 2e [1st: The Lambda book]
- Gamma et al.: Spring, 1994
- The Dragon Book: Summer, 1994
- The Dinosaur Book: Fall, 1994
- The Sailboat Book: Spring, 1996
- Mehrotra, Mohan, and Ranka: fall, 1997
- Angel, 4e [2nd]: spring, 2000
- Duda, Hart, and Stork, 2e [1st]: fall, 2000
- Dean, Allen, and Alomoinos: fall, 2000
- Han and Kamber: summer, 2001
- Haupt and Haupt: spring, 2002
- Pressman: spring, 2003
- Hearn and Baker, 3e [2nd]: fall, 2004
- Tang, Steinbach, and Kumar: spring, 2005