shigangli / wagma-sgd Goto Github PK
View Code? Open in Web Editor NEWWAGMA-SGD is a decentralized asynchronous SGD based on wait-avoiding group model averaging. The synchronization is relaxed by making the collectives externally-triggerable, namely, a collective can be initiated without requiring that all the processes enter it. It partially reduces the data within non-overlapping groups of process, improving the parallel scalability.
License: GNU General Public License v3.0