neubig / nn4nlp-code Goto Github PK

I've got a few questions about the code, so I will just list them out. And hope some of you can help.

1. In line 166, what is "XENT"? this variable only appears once in the code.
2. In line 106, I don't understand why the probability distribution is calculated in that way?(By the ay my
   compiler said the as_array() part is not right.)
3. In line 109, "len(x)" is not right. It kept showing this up "TypeError: object of type '_dynet.Expression' has no len()". And I don't really know why the samples are picked this way?

Most files share similar data reading code, like

In most of the examples, the variable nwords is used as the effective vocabulary size, for instance, when we allocate parameters for embedding matrix.

However, there are likely many new words in dev/test set that might be added in w2i... their values are mapped to UNK, but they are still counted in len(w2i) which is likely not intended. Often this overcounting does not change the results, but it can be problematic in some cases.

In pretty much all of the dynet examples, there are unnecessary parameter(parameter_name) calls to convert parameters to expressions. This is no longer necessary, as the newest version of python dynet automatically does the conversion.

Model() should also be renamed to Parameter collection.

Thank you for your work. Can you share the original data file with us? Thanks a lot!

I find the comments in the MST decoder confusing. The MST decoder is not running a full Chu-Liu-Edmonds MST. It is based on the heuristics that Stanford CoNLL 2017 parser used. Moreover, the scores in Stanford's code are assumed to be normalized by softmax (therefore multiplicative scoring) whereas here, the code passes unnormalized scores , which are basically log-probabilities (ignoring the denominator of softmax, which of course is okay because it affects uniformly whichever node you pick as the parent). I think all division operations in MST decoder should be replaced by subtraction, and 0s should be replaced by -inf.

I've got a few questions about the code "16-reinforce" , so I will just list them out. And hope you can help.

line 106 throws an error '_dynet.Expression' object has no attribute 'as_array'
2)line 109 throws an error '_dynet.Expression' object has no attribute 'len(x)'
Can you please help me the solution for this.

Shouldn't a softmax be added to the model output?

I would like to know if there are any plans for completing the Pytorch code, and would you accept merge requests from non-class members?

What's version of the Python that the course uses?

Has anyone run those scripts? the performances are really bad... before uploaded to the repo, is there anyone verify those code?

Hi, Thanks for sharing the source code. I learned a lot. But, I have a quick question regarding "Reinforce score", my question is why the Reinforce score is "Score*reward" as calculated here :

line 126 of bilstm-tagger.py

#then calculate the reinforce scores using reinforce
    reinforce_scores = [r_s*score for r_s, score in zip(rewards_over_baseline, scores)]```

There are some error in function generate(sent)

I've got a few questions about the code "16-reinforce" , so I will just list them out. And hope you can help.

1. line 106 throws an error '_dynet.Expression' object has no attribute 'as_array'
   2)line 109 throws an error '_dynet.Expression' object has no attribute 'len(x)'

Can you please help me the solution for this.

Both language model examples in 06-rnn are lacking generation capability and usage examples. I think those should be added, and the output format changed to be closer to that of the 02-lm examples.

This is a bug and should be removed:
https://github.com/neubig/nn4nlp-code/blob/master/02-lm/loglin-lm.py#L97

Other files should be checked as well.