lifu-tu / engine Goto Github PK
View Code? Open in Web Editor NEWENGINE: Energy-Based Inference Networks for Non-Autoregressive Machine Translation
License: Other
ENGINE: Energy-Based Inference Networks for Non-Autoregressive Machine Translation
License: Other
Is it saying that, the energy is equivalent to the cross entropy loss of a pretrained model_E
where target side input replaced by O1(Y) weighted embedding, and target distribution is O2(Y_nograd)?
Hi Lifu,
Thanks for you code and paper. It's very nice work but I am a little confused about the pretrained models, I didn't find the one for IWSLT, facebook only provided the wmt version.
Hi,
Why you replace the softmax function with the exp function in SX?
ENGINE/fairseq/models/transformer.py
Line 735 in 73a9d38
ENGINE/fairseq/criterions/inf_energy_loss.py
Line 163 in 73a9d38
Did you use raw data or distilled data for training the CMLM model(for initializing the ENGINE model)? And which type of data do you use to train ENGINE?
In your ENGINE_WMT16ROEN.zip
file, I found that the pre-trained CMLM model got 28.20 BLEU scores on WMT16 RO-EN test dataset, which means the CMLM the trained using the distilled dataset. But you preprocess the raw data in your preprocess_wmtRO2EN.sh
file.
So, you trained the CMLM model using the distilled data, and then trained the ENGINE model using the raw data. Am I understanding it correctly?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.