Post
154
Pangu Pro MoE 🔥 Huawei's first open model!
Paper:
Pangu Pro MoE: Mixture of Grouped Experts for Efficient Sparsity (2505.21411)
Model:
https://gitcode.com/ascend-tribe/pangu-pro-moe-model
✨ MoGE: Mixture of Grouped Experts
✨ 16B activated params - 48 layers
✨ Trained on 15T tokens
✨ Natively optimized for Ascend hardware
Paper:
Pangu Pro MoE: Mixture of Grouped Experts for Efficient Sparsity (2505.21411)
Model:
https://gitcode.com/ascend-tribe/pangu-pro-moe-model
✨ MoGE: Mixture of Grouped Experts
✨ 16B activated params - 48 layers
✨ Trained on 15T tokens
✨ Natively optimized for Ascend hardware