Speech and Language Processing (3rd Ed. Draft)

I learned speech recognition from the 2nd edition of Jurafsky's book (2008). The field has changed so much it sometimes feels unrecognizable. Instead of hidden markov models, gaussian mixture models, tri-phone state trees, finite state transducers, and so on, nearly the whole stack has been eaten from the inside out by neural networks.

But, there's benefit to the fact that deep learning is now the "lingua franca" across machine learning fields. In 2008, I would have struggled to usefully share ideas with, say, a researcher working on computer vision.

Now neural networks act as a shared language across ML, and ideas can much more easily flow across speech recognition, computer vision, AI in medicine, robotics, and so on. People can flow too, e.g., Dario Amodei got his start working on Baidu's DeepSpeech model and now runs Anthropic.

Makes it a very interesting time to work in applied AI.

This is the OG among the Computational Linguistics books. Very glad it exists and is being revised.

Newcomers to the field should glad to read through this... there is gold in there. <3

I got my start in NLP back in '08 and later in '12 with an older version of this book. Recommended!

Good old Jurafsky and Martin. Got to meet Dan Jurafsky when he visited UT back in '07 or so -- cool guy.

This one and Manning and Schutze's "Dice Book" (Foundations of Statistical Natural Language Processing) were what got me into computational linguistics, and eventually web development.

Latecomers to the field may be tempted to write this off as antiquated (though updated to cover transformers, attention, etc.) but a better framing would be that it is _grounded_. Understanding the range of related approaches is key to understanding the current dominant paradigm.

Were NLP people able to cleanly transition? I'm assuming the field is completely dead. They may actually be patient zero of the impending llm-driven unemployment outbreak.

I was eagerly waiting for a chapter on semantic similarity as I was using Universal Sentence Encoder for paraphrase detection, then LLMs showed up before that chapter :).

One can feel for the authors, it's such a struggle to write a textbook in a time when NeurIPS gets 20000 submissions and ACL has 6500 registered attendees (as of August '05), and every day, dozens of relevant ArXiv pre-prints appear.

Controversial opinion (certainly the publisher would disagree with me): I would not take out older material, but arrange it by properties like explanatory power/transparency/interpreability, generative capacity, robustness, computational efficiency, and memory footprint. For each machine learning method, an example NLP model/application could be shown to demonstrate it.

Naive Bayes is way too useful to downgrade it to an appendix position.

It may also make sense to divide the book into timeless material (Part I: what's a morphem? what's a word sense?) and (Part II:) methods and datasets that change every decade.

This is the broadest introductory book for beginners and a must-read; like the ACL family of conferences it is (nowadays) more of an NLP book (i.e., on engineering applications) than a computational linguistics (i.e., modeling/explaining how language-based communication works) book.