cgpotts / cs224u Goto Github PK
View Code? Open in Web Editor NEWCode for Stanford CS224u
License: Apache License 2.0
Code for Stanford CS224u
License: Apache License 2.0
In line 35 of tf_model_base.py, self.hidden_activation is initialized hardcoded to tf.nn.tanh instead of taking the parameter value.
The video at the following page is not loading. The page is also shooting a warning, since https is not implemented: Video: High-level goals and guiding hypotheses [slides]
the data.zip might be broken since when I try to open it, my machine told me this and I could not find the csv.gz file as a result..
I believe get_feature_names() is deprecated (and now removed). get_feature_names_out() is the replacement.
Hi,
I was working on the hw_colors notebook to create the updated EncoderDecoder model following the instructions. I thought I got all my updates correct until I received an error in the last test.
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-217-f204130de1b0> in <module>
----> 1 test_full_system(ColorizedInputDescriber)
<ipython-input-216-df38441535ee> in test_full_system(describer_class)
8 toy_mod = describer_class(toy_vocab)
9
---> 10 _ = toy_mod.fit(toy_color_seqs_train, toy_word_seqs_train)
11
12 acc = toy_mod.listener_accuracy(toy_color_seqs_test, toy_word_seqs_test)
~/Downloads/Stanford-CS224U/codebase/torch_model_base.py in fit(self, *args)
359 y_batch = batch[-1]
360
--> 361 batch_preds = self.model(*X_batch)
362
363 err = self.loss(batch_preds, y_batch)
/usr/local/lib/python3.7/site-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs)
548 result = self._slow_forward(*input, **kwargs)
549 else:
--> 550 result = self.forward(*input, **kwargs)
551 for hook in self._forward_hooks.values():
552 hook_result = hook(self, input, result)
<ipython-input-214-6cb1cf97c969> in forward(self, color_seqs, word_seqs, seq_lengths, hidden, targets)
18 output, hidden = self.decoder(
19 word_seqs,
---> 20 target_colors=color_seqs[:,-1,:])
21
22 # Your decoder will return `output, hidden` pairs; the
/usr/local/lib/python3.7/site-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs)
548 result = self._slow_forward(*input, **kwargs)
549 else:
--> 550 result = self.forward(*input, **kwargs)
551 for hook in self._forward_hooks.values():
552 hook_result = hook(self, input, result)
~/Downloads/Stanford-CS224U/codebase/torch_color_describer.py in forward(self, word_seqs, seq_lengths, hidden, target_colors)
232 batch_first=True,
233 lengths=seq_lengths,
--> 234 enforce_sorted=False)
235 # RNN forward:
236 output, hidden = self.rnn(embs, hidden)
/usr/local/lib/python3.7/site-packages/torch/nn/utils/rnn.py in pack_padded_sequence(input, lengths, batch_first, enforce_sorted)
232 'the trace incorrect for any other combination of lengths.',
233 stacklevel=2)
--> 234 lengths = torch.as_tensor(lengths, dtype=torch.int64)
235 if enforce_sorted:
236 sorted_indices = None
TypeError: an integer is required (got type NoneType)
I was having a hard time understanding this error because I thought we didn't need to input hidden and seq_lengths in the forward step.
I am defining my decoder call as:
output, hidden = self.decoder(
word_seqs,
target_colors=color_seqs[:,-1,:])
Is it a mistake in my code or do I have to include an additional length variable?
I know this is part of the homework and not necessarily related to the code base, but any help would be appreciated.
Thank you
Executing the first cell of the hw_sentiment.ipynb
cell returns the error partially initialized module 'charset_normalizer' has no attribute 'md__mypyc'
on a newly install nlu environment using miniconda.
This was resolved by running this command in the conda environment:
pip install -U --force-reinstall charset-normalizer
(based on the post on pip install -U --force-reinstall charset-normalizer .
While working on the hw_wordentail notebook, I have found an error in test_TorchDeepNueralClassifier
function.
The test aims to check whether the nn.module
return value from class TorchDeepNeuralClassifier
has been successfully implemented. However, the test function extracts the graph by using define_graph()
function which is not implemented.
This should be changed to build_graph()
. I can make the change and create a pull request if needed.
The code base thoroughly uses in
idiom to test environment variable presence as a flag, like
if 'IS_GRADESCOPE_ENV' not in os.environ:
pass
To enable above condition, one has to unset IS_GRADESCOPE_ENV
. Flipping the value has not effect, such as IS_GRADESCOPE_ENV=0
or IS_GRADESCOPE_ENV=1
.
However, it is usually to use os.environ.get()
as a better alternative in production systems. For example,
if not os.environ.get('IS_GRADESCOPE_ENV', False):
pass
# or
if not os.environ.get('IS_GRADESCOPE_ENV', None):
pass
# or strictly checking against pre-defined value
if not os.environ.get('IS_GRADESCOPE_ENV', '0') == '1':
pass
Because in shell script, we usually test if an environment variable flag is on by checking if it is present or if it is non-empty (i.e., [[ ! -z "$VAR" ]]
). In this way, either of following will be evaluated as false as expected and follows the convention, just in case users forgot to unset
the value from the environment.
IS_GRADESCOPE_ENV=
unset IS_GRADESCOPE_ENV
export IS_GRADESCOPE_ENV=''
Just a minor issue, feel free to ignore. ๐
Hi @cgpotts,
I've just done all of the assignments (except the original system) in hw_wordrelatedness.ipynb
and related notebooks and have a couple of thoughts that may be useful to share.
run_knn_score_model
that students are asked to write is not tested at all. It could be helpful to check the output score so that students know they have written it correctly. You could make train_test_split
deterministic by setting shuffle=False
. Another option would be to add a note about what approximate score one should expect.vsm_01_distributional.ipynb
your proper_cosine
function looks to be returning the angular distance. Calling it proper_cosine
may be confusing to some unless that's a standard name for it.Looking forward to going through the next notebooks!
In the second homework question for hw_wordsim, we are provided with the following instructions:
Gigaword with LSA at different dimensions [0.5 points]
We might expect PPMI and LSA to form a solid pipeline that combines the strengths of PPMI with those of dimensionality reduction. However, LSA has a hyper-parameter ๐ โ the dimensionality of the final representations โ that will impact performance. For this problem, write a wrapper function run_ppmi_lsa_pipeline that does the following:
1. Takes as input a count pd.DataFrame and an LSA parameter k.
2. Reweights the count matrix with PPMI.
3. Applies LSA with dimensionality k.
4. Evaluates this reweighted matrix using full_word_similarity_evaluation. The return value of run_ppmi_lsa_pipeline should be the return value of this call to full_word_similarity_evaluation.
The goal of this question is to help you get a feel for how much LSA alone can contribute to this problem.
The function test_run_ppmi_lsa_pipeline will test your function on the count matrix in data/vsmdata/giga_window20-flat.csv.gz.
When I construct run_ppmi_lsa_pipeline
with the following steps, I get the wrong output:
count_df
using: ppmi_df = vsm.pmi(count_df, positive=True)
vsm.lsa(ppmi_df, k)
This leads to a similarity evaluation for men
of 0.57. Not the expected 0.16
However, when I construct run_ppmi_lsa_pipeline
using the following steps (without applying PPMI), I get the correct output:
vsm.lsa(count_df, k)
This leads to the expected similarity for men
of 0.16.
Is there a mistake in the instructions/expected results? Perhaps I did something wrong? Any and all help would be appreciated.
I wanted to create PR but I do not have create branch permission.
Minor thing, but could improve understanding. In hw_wordentail.ipynb:
"That is, if a word w appears in a training pair, it does not occur in any text pair. "
should read as:
"That is, if a word w appears in a training pair, it does not occur in any test pair. "
Hi there,
I noticed that this line should be changed to
reps = bert_model(X_example, attention_mask=X_example_mask, return_dict=True)
for the following cells to work.
Thank you,
Wen
in hw2_sst.ipynb
I think you meant amazing
instead of enlightening
in expected
(BTW: Thank you so much for your amazing work)
def test_op_unigrams_phi(func):
tree = Tree.fromstring("""(4 (2 NLU) (4 (2 is) (4 amazing)))""")
expected = {"enlightening": 1}
result = func(tree)
assert result == expected, \
("Error for `op_unigrams_phi`: "
"Got `{}` which differs from `expected` "
"in `test_op_unigrams_phi`".format(result))
I downloaded data.zip from "the course data" link on this notebook https://github.com/cgpotts/cs224u/blob/spring-2019/vsm_01_distributional.ipynb
System:
macOS High Sierra 10.13.6
java 12.0.2 2019-07-16
Java(TM) SE Runtime Environment (build 12.0.2+10)
Java HotSpot(TM) 64-Bit Server VM (build 12.0.2+10, mixed mode, sharing)
I run jar xvf data.zip
and get:
created: data/
inflated: data/.DS_Store
created: __MACOSX/
created: __MACOSX/data/
inflated: __MACOSX/data/._.DS_Store
created: data/glove.6B/
inflated: data/glove.6B/glove.6B.100d.txt
created: __MACOSX/data/glove.6B/
inflated: __MACOSX/data/glove.6B/._glove.6B.100d.txt
inflated: data/glove.6B/glove.6B.200d.txt
inflated: __MACOSX/data/glove.6B/._glove.6B.200d.txt
inflated: data/glove.6B/glove.6B.300d.txt
inflated: __MACOSX/data/glove.6B/._glove.6B.300d.txt
inflated: data/glove.6B/glove.6B.50d.txt
inflated: __MACOSX/data/glove.6B/._glove.6B.50d.txt
created: data/negotiate/
inflated: data/negotiate/data.txt
created: __MACOSX/data/negotiate/
inflated: __MACOSX/data/negotiate/._data.txt
inflated: data/negotiate/selfplay.txt
inflated: __MACOSX/data/negotiate/._selfplay.txt
inflated: data/negotiate/test.txt
inflated: __MACOSX/data/negotiate/._test.txt
inflated: data/negotiate/train.txt
inflated: __MACOSX/data/negotiate/._train.txt
inflated: data/negotiate/val.txt
inflated: __MACOSX/data/negotiate/._val.txt
inflated: __MACOSX/data/._negotiate
created: data/nlidata/
created: data/nlidata/.ipynb_checkpoints/
inflated: data/nlidata/.ipynb_checkpoints/prep_wordentail_data-checkpoint.ipynb
inflated: data/nlidata/.ipynb_checkpoints/prep_wordentail_data-Copy1-checkpoint.ipynb
created: data/nlidata/multinli_1.0/
extracted: data/nlidata/multinli_1.0/Icon
created: __MACOSX/data/nlidata/
created: __MACOSX/data/nlidata/multinli_1.0/
inflated: __MACOSX/data/nlidata/multinli_1.0/._Icon
inflated: data/nlidata/multinli_1.0/manuscript.pdf
inflated: __MACOSX/data/nlidata/multinli_1.0/._manuscript.pdf
inflated: data/nlidata/multinli_1.0/multinli_1.0_dev_matched.jsonl
inflated: __MACOSX/data/nlidata/multinli_1.0/._multinli_1.0_dev_matched.jsonl
inflated: data/nlidata/multinli_1.0/multinli_1.0_dev_matched.txt
inflated: __MACOSX/data/nlidata/multinli_1.0/._multinli_1.0_dev_matched.txt
inflated: data/nlidata/multinli_1.0/multinli_1.0_dev_mismatched.jsonl
inflated: __MACOSX/data/nlidata/multinli_1.0/._multinli_1.0_dev_mismatched.jsonl
inflated: data/nlidata/multinli_1.0/multinli_1.0_dev_mismatched.txt
inflated: __MACOSX/data/nlidata/multinli_1.0/._multinli_1.0_dev_mismatched.txt
inflated: data/nlidata/multinli_1.0/multinli_1.0_train.jsonl
inflated: __MACOSX/data/nlidata/multinli_1.0/._multinli_1.0_train.jsonl
java.io.EOFException: Unexpected end of ZLIB input stream
at java.base/java.util.zip.InflaterInputStream.fill(InflaterInputStream.java:245)
at java.base/java.util.zip.InflaterInputStream.read(InflaterInputStream.java:159)
at java.base/java.util.zip.ZipInputStream.read(ZipInputStream.java:195)
at java.base/java.util.zip.ZipInputStream.closeEntry(ZipInputStream.java:141)
at jdk.jartool/sun.tools.jar.Main.extractFile(Main.java:1457)
at jdk.jartool/sun.tools.jar.Main.extract(Main.java:1364)
at jdk.jartool/sun.tools.jar.Main.run(Main.java:409)
at jdk.jartool/sun.tools.jar.Main.main(Main.java:1681)
I run unzip data.zip
and get:
Archive: data.zip
End-of-central-directory signature not found. Either this file is not
a zipfile, or it constitutes one disk of a multi-part archive. In the
latter case the central directory and zipfile comment will be found on
the last disk(s) of this archive.
unzip: cannot find zipfile directory in one of data.zip or
data.zip.zip, and cannot find data.zip.ZIP, period.
The autograder being used in XCS224U when converting the notebook hw_colors.ipynb
to a python script adds get_ipython()
command to the script. This won't work unless you import it - from IPython import get_ipython
. something like this:
So it would be good to change these lines of code from:
to this:
I already have the permission to make changes to the repo. Just wanted to run by you once before making changes and committing it. Thanks @cgpotts
Thank you for the great course! The course lectures and the other materials are really valuable to learn more about NLU.
I am not an enrolled student, but I've decided ask here a minor question related to the first lecture about "Contextual Word Representation".
In slide 5 (https://web.stanford.edu/class/cs224u/slides/cs224u-contextualreps-part1-handout.pdf), the "context vector" is evaluated as
My question: Is it really necessary to do the "mean" operation instead of a "sum" ?
The attention weights
What I see often is to scale the dot products (before the softmax)
Thanks again!
Hi,
I have been trying to run the codes in the notebook. However, as I try to run the code that utilizes torch_model, I keep getting an error when I fit the model. How can I resolve this issue?
Thank you,
Joey
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-60-77ee602003f0> in <module>
1 giga_ae = TorchAutoencoder(max_iter=1000,
2 hidden_dim=100,
----> 3 eta=0.03).fit(giga5_svd500)
~/Downloads/Stanford-CS224U/codebase/torch_autoencoder.py in fit(self, X)
124
125 """
--> 126 super().fit(X, X)
127 # Hidden representations:
128 with torch.no_grad():
~/Downloads/Stanford-CS224U/codebase/torch_model_base.py in fit(self, *args)
351 epoch_error = 0.0
352
--> 353 for batch_num, batch in enumerate(dataloader, start=1):
354
355 batch = [x.to(self.device, non_blocking=True) for x in batch]
/usr/local/lib/python3.7/site-packages/torch/utils/data/dataloader.py in __next__(self)
613 if self.num_workers == 0: # same-process loading
614 indices = next(self.sample_iter) # may raise StopIteration
--> 615 batch = self.collate_fn([self.dataset[i] for i in indices])
616 if self.pin_memory:
617 batch = pin_memory_batch(batch)
TypeError: 'NoneType' object is not callable
For the main interface, we can just subclass TorchRNNClassifier and change the build_graph method to use TorchVecAvgModel. (For more details on the code and logic here, see the notebook: tutorial_torch_models.ipynb
This should be:
tutorial_pytorch_models.ipynb
Thank you for the wonderful course resources you've made available! Will the videos for the later lectures such as grounded language understanding, semantic parsing, evaluation metrics, and contextual word embeddings ever make their way to YouTube?
Hi - I've followed the instructions to setup an environment for the course on my machine. The only difference I am aware of is that miniconda was pre-installed.
When following the instructions, the version of pytorch installed was cpu only. To fix that, I ran the following command:
conda install pytorch=1.13.1 pytorch-cuda=11.7 -c pytorch -c nvidia
This seems to have resolved it for me.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.