Comments (1)
Hi, thank you for your question! I have to admit that we made a mistake on that statement. We will remove this in our later versions.
Nevertheless, we think the comparison is fair. We re-implemented D3PM with PyTorch. Besides, we replaced their backbone with the architecture of bert-base-uncased and used the same tokenizer (so that both methods are lower-cased). We obtained the baseline results based on such re-implementation.
It is also worth noting that our reported results of D3PM-absorbing are only slightly worse than that in their paper due to limitation of computational resources, indicating the correctness of our implementation. But we trained DiffusionBert for even less time.
Hope this helps! Please feel free to contact with me if you have any other questions. We will also include the cased results in the final version. :)
from diffusion-bert.
Related Issues (20)
- Conditional Generation HOT 1
- checkpoint HOT 1
- Function missed HOT 2
- Here are some of my problems, please advise
- 关于时间表 HOT 5
- Missing key(s) in state_dict when testing using predict_downstream_condition.py HOT 7
- unfinished codebase? HOT 2
- No module named 'perplexity
- No module named 'perplexity HOT 2
- FileNotFoundError: [Errno 2] No such file or directory: '/remote-home/zfhe/projects/diffusion_torch/D3PM_new_timestep_ckpts/best(1799999).th' HOT 1
- How to fine-tune it HOT 1
- Do you plan to release the checkpoints?
- Missing key(s) in state_dict for unconditional
- self is not defined for discrete_diffusion_predict_fn() HOT 1
- What's the meaning of the parameter 'load_step'?
- why TypeError? HOT 2
- Can you please publish some details of the Diffusion-LM training phase? Learning steps, batch_size, etc.
- RuntimeError: Error(s) in loading state_dict for RobertaForMaskedLM:
- Resuming training via `--load_step`
- How did you calculate the perplexity of DiffusionLM
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from diffusion-bert.