Guest
Race Result Details |
| Racer | clu0 (clu0) |
| Race Number | 1 |
| Date | Wed, 27 Aug 2025 0:07:09 |
| Universe | magic |
| Speed |
72 WPM
 Try to beat?
|
| Accuracy | 95.8% |
| Rank | 7th place (out of 8) |
| Opponents | cielmaster (6th place) emef (4th place) foresterdaniel (3rd place) seanmor5 (1st place) sonniko (9th place) vig0 (2nd place) xskape (8th place) zomglings (5th place) |
Text typed:
|
We validate the performance of Loss-Free Balancing on MoE models with up to 3B parameters trained on up to 200B tokens. Experimental results show that Loss-Free Balancing achieves both better performance and better load balance compared with traditional auxiliary-loss-controlled load balancing strategies.
— (other)
by AUXILIARY-LOSS-FREE LOAD BALANCING STRATEGY FOR MIXTURE-OF-EXPERTS
(see stats)
|