codecaution / awesome-mixture-of-experts-papers Goto Github PK
View Code? Open in Web Editor NEWA curated reading list of research in Mixture-of-Experts(MoE).
License: Apache License 2.0
A curated reading list of research in Mixture-of-Experts(MoE).
License: Apache License 2.0
Hi authors,
Thank you for your repo! I also created one awesome MoE repo recently. I will update your new work into my repo.
https://github.com/XueFuzhao/awesome-mixture-of-experts
Also, I think a few of my papers are missing.
Go Wider Instead of Deeper [AAAI2022]
Cross-token Modeling with Conditional Computation [5 Sep 2021]
One Student Knows All Experts Know: From Sparse to Dense [26 Jan 2022]
Thank you so much!
There is a new distributed system FasterMoE: modeling and optimizing training of large-scale dynamic pre-trained models published on PPoPP'22. Please kindly consider including this paper in your list.
FYI, we have also included your MoE systems and paper collections on FastMoE's homepage
Dear authors,
Thank you for contributing such a well-organized GitHub repo.
However, since this repo has not been updated since 2022, I created a new repo to summarize state-of-the-art papers of MoE.
For everyone, kindly feel free to look at it: https://github.com/Oliver-FutureAI/Awesome-MoE.
And please star it if it is helpful for you.
Hey! Thank you for your work.
Could you add our MoE work for generalist models in NIPS 2022๏ผ
Uni-Perceiver-MoE: Learning Sparse Generalist Models with Conditional MoEs. [paper] [code]
This work uses MoE to mitigate task interference in multitask training and proposes routing strategies to make MoE more efficient.
Thank you so much!
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.