view article Article makeMoE: Implement a Sparse Mixture of Experts Language Model from Scratch By AviSoori1x • May 7, 2024 • 88
view article Article Perceiver IO: a scalable, fully-attentional model that works on any modality By nielsr • Dec 15, 2021 • 11
Running 2.84k 2.84k The Ultra-Scale Playbook 🌌 The ultimate guide to training LLM on large GPU Clusters