"it worse to have two failures than one" ->(?) "it's worse to have two failures than one"

In the version 'draft 3 june' at line 1892 there is word duplication by accident:

Classification of these relations relations can be performed by searching for characteristic patterns between pairs of words

The word relations appears twice.

By the way, so far, great read, thx a lot for opensourcing this!

"An indicator random variable is a functions" -> "An indicator random variable is a function"

Dear Professor Eisenstein,

I found a typo in the textbook. The left-hand side of the equation [15.33] has more closing parentheses than opening ones.

With best regards,

Katô Taisei

Dear Prof. Eisenstein,

I guess there should be a minus sign at the beginning of the right side of the equation:

l_{LogReg} (\theta; x^{(i)}, y^{(i)}) = - \theta f(x^{(i)}, y^{(i)}) + rest of the equation

This can be also illustrated by Equation 2.60.

Thank you for sharing your book! I enjoy reading it.

Weiwei

Hi there,

I think the selected sentence should be A high-recall classifier is preferred when false positives are cheaper than false negatives.

Thanks.

I've found some small typos in the chapter 6 of the notes, the version from October 21, 2018:

• Page 133, line 8: $p^\star(Duluth)$ → $p^\star_1(Duluth)$ (in order to be consistent with the probability of Francisco)
• Page 140, line -3: "a perplexities of around" → "perplexities of around"
• Page 140, equation [6.42] – the comma is in the exponent
• Page 141, line -5: "n-gramsor RNNs" → "n-grams or RNNs"
• Page 141, line -6: "character" overflows into the margin

And one suggestion: I think the article of Melis et al., On State of the Art of Evaluation in Neural Language Models, might be relevant for subsection 6.3.2, as it makes a strong case for hyperparameter tuning.

That said, thank you for making the notes available – they seem like a very fine resource!

Hi,

I know this is very minor but there is a typo in the notes eisenstein-nlp-notes-snapshot.pdf.

It is at the top of page 86, when discussing effective counts just above equation (5.14):

dominator -> denominator

(dominate, exterminate?)

In equation [4.2](between line 1824 and 1825), argmin -> argmax

In Chapter 7.5.3
Should the following formula includes the exp for the scores s?

Like the previous definition

Dear professor Eisenstein,

Thank you very much for making the book publicly available.
I would like to point a possible typo on the equation on section 7.5.3.1:

Should not the s function be written s_m, indicating the transition between labels y_{m-1} and y_m on the i-th position of the underlying w sequence?

Thank you very much,
Luiz C F Ribeiro

in chapter 7.3

should it be transition weights?

Dear Professor Eisenstein,

I found some errors in the printed version of Introduction to Natural Language Processing.

• On Notation (page xiii), the base of the exponential and logarithm should be e because the base-2 exponent and the base-2 logarithm do not satisfy (\exp x)' = \exp x and (\log x)' = 1 / x, respectively, which are used in, for example, equation [2.26].
• On Exercise 5-5, 'This is the "direct transfer" baseline' should be appended to the first bullet item.
• On Exercise 6-9, the definition of the Riemann zeta function is \sum_{r = 1}^\infty r^{-s}.
• In Algorithm 11 (on page 142),
  • On lines 1 and 4, k iterates from 0 to K, but it means that there are K + 1 tags. Also, there is a missing comma on line 1.
  • On line 9, b_m should be b_{m+1}.
• On equation [7.86], n (both in the numerator and the denominator) should iterate until M + 1.
• On equation [7.88], n should start from m + 1. And the following equation must be \sum_{k' \in \mathcal{Y}} \exp s_{m + 1}(k', k)\sum_{\boldsymbol{y}_{m + 1: M}: Y_{m + 1} = k'} \prod_{n = m + 2}^{M + 1} \exp s_n(y_n, y_{n - 1}).

With best regards,

Katô Taisei

Professor Eisenstein,
"It is also includes" (last 4 words on line 697, p.24): I think you meant "It also includes".
P.S. Thanks for making the draft of your textbook available. I've just started reading it and it's been very enjoyable.

The index of the first summation should be $i$ from 1 to $N$, not from 1 to $M$.

Dear @jacobeisenstein,

Thanks to always improve content quality of your NLP note. However, i found difficulty to refer the actual changes since your commit message too brief without any details. This also due to the nature of PDF file which harder to track the diff.

If you mind, i recommend to add more details about the changes in commit message, at least the location of the diff (e.g: which page and section). Thus, instead redownload your note, i can just correct it manually by my hand.

Hope this not overwhelming, thanks

There isn't an assignment listed on Tsquare. Will there be, or do we post somewhere else?

"they disjoint" -> "they are disjoint"

In chapter 2, page 30-31.

If theta is V * K, shouldn't the result of f(x, y) be K * V rather than a column, so that result of psi is of dimension K?

Hi,

I have to say that I am not totally sure, but I think that there is a mistake in the review of basic probability, on the example using Bayes rule (Lana Linguist's pattern matcher).

Page 19, equation (1.23) states that

Pr(T ∩ ¬G) = Pr(T | G) × Pr(¬G)

instead of

Pr(T ∩ ¬G) = Pr(T | ¬G) × Pr(¬G)

Because of this, the numbers are wrong, as Pr(T | ¬G) = .995 and not .005

EDIT: Pr(T | ¬G) is indeed .005, so the numbers are not wrong.

The rest of the derivation up to equation (1.28) is wrong because of this error.

In order to correct this, we should change the four equations below:

Pr(T ∩ ¬G) = Pr(T | ¬G) × Pr(¬G) = 0.995 × (1−10^−5) ≈ 0.995 (1.23)
= 0.95 × 10^−5 + 0.995 ≈ 0.005 (1.25)
= 0.95 × 10^−5 / (0.95 × 10^−5 + 0.995 × (1−10^−5)) (1.27)
≈ 0.00001 (1.28)

The pattern matcher is even less useful than previously thought!

Hi, I think there's a typo on page 33 Lines 817 to 818 where you use the word "describes" twice. Maybe you meant to say "Algorithm 1 describes the generative model, the Naive Bayes classifier, with parameters..."

• Brandon Peck

It seems like the formula 6.23 does not agree with what wikipedia gives.

I find the probability sum less than 1 according to the formula in the note.
Correct me if I'm wrong.
Thanks.

Dear professor,

Once again thank you very much for making this material publicly available. Is there any default Bibtex that we should use to cite the notes?

Thank you.

Chapter 9, page 194 states that for the membership operation with we should (can) use Dijkstra's algorithm for a DFA. Surely with a DFA the membership test is much simpler, as a string has a unique path. So we need only follow the M edges labelled with the input symbol sequence, then check if this state is a final state F. Thus no need for Dijsktra, and the complexity is linear in the string size (and the size of the DFA is irrelevant). The need for Dijkstra would come if we use a NFA that hasn't been determinised, or a FST. My comment also pertains to p197, below [9.5] which repeats the bit about shortest-path for WFSAs.

Maybe I'm missing something, please let me know.

Hi Jacob,

The footnote 13 (under line 1057) of the class note (version June 1, 2018):

A function f is convex iff αf(x i )+(1−α)f(x j ) ≥ f(αx i +(1−α)x j ),

Should 'iff' be 'if'? Thanks a lot!

Best,

S

Hi, thank you for your great book.
I am confused about the subscript in formula 3.29, which I think the zk should be zj (sorry for the uncorrect typing shape).
Thanks.

Dear @jacobeisenstein,

First of all, really appreciate for write book of NLP with recent trends. I know this is too early, but any raw plan to make this "book" become actual book? Sorry if i'm missing your statement about this.

Besides, i'm willing to read your book but seems the content not final yet. Could you give some notes which chapters or sections that might still in early stage/prone to error? is it what you mark it with "*" sign in the TOC? Thank you, can't wait for the final version.