guilhermevarela / deep_srlbr Goto Github PK

Importing a Dual Layer - Bi Directional LSTM is more work than benefit ( due to implicit declared variables)

Make models persist the result of calibrated models
Make models loadable
Make a script for prediction

Moving an easier task -- R closer to the features might aliviate the vanishing gradient problem

Computations using PropBank English -- using up to 40GB of RAM.

• Avoid loading full development set and evaluation set into memory
• Avoid storing embedded representations and compute embeddings from ids using tf.nn.embedding_lookup -- Using up less memory but increasing computing overhead

• Add SrlAgent(*args, **kwargs)
• Add fit() -- train
• Add eval(X, T) -- avaliate a dataset
• Add pred(X) -- predict tags

Make hyperparameter comparison easy by leveraing on jupyternotebooks

Decouple DBLSTM using the composition design pattern, some associated tasks:

• gather word embeddings
• propagate the features thru stacked LSTM layers --> interleaving LSTM*? GRU? BILSTM?
• make the last prediction --> CRF*? SOFTMAX? A*?

Previous works show that the argument recognition subtask is important to the development of srl --

Instead of having
B-A0, I-A0, B-V, B-A1, I-A1, I-A1

Have:
(B, A0), (I, A0), (B, V), (B, A1), (I, A1), (I, A1)

Save binary information within a file -- especially column dimensions which are needed to declare graph tensors and also pull information from the binaries

Besides PoS tagging another linguistic feature that may contribute to the model's performance is the inclusion of Semantic Chunks -- This new feature can be computed by extracting sub-trees from the Syntactic Tree included in the golden set.

Beware: Some phrases are more important than others.

• Besides interleaving propagator (Dual Bi-directional LSTM DBLSTM) add serial propagator (Bidirectional BLSTM)

Not all parameters are being stored such as:

*input_features
*output_target
*target: T, IOB etc

Embeddings are currently fixed -- make them trainable

*Change on the pipelines
*Change scripts

• move tensor decoding functions to propbank encoder

integrate system with english variant of propbank

GPU -- training machine has tensorflow version 1.2.1 installed
development machine has tensorflow version 1.9 installed

There's a function in the latter which is absent from the former and it helps decode the CRF tags from the predictions.

tf 1.9 --> has crf_decode
tf 1.2.1 --> has viterbi_decode ( which is computed outside tensorflow ) How to port crf_decode to the training machine?

Add softamax predictor besides CRF

Which is the best having the recognition task (R) + classes (T) or recognition task (R) + classes (IOB)