Model Architecture - a zk67 Collection

zk67 's Collections

Foundation Models and AGI

Model Architecture

Instruction Tuning

inference optimization

LLM Reasoning Papers

LLM Tech Report

LLM Post Training

Model Architecture

updated Jan 20

Sparse Upcycling: Training Mixture-of-Experts from Dense Checkpoints

Paper • 2212.05055 • Published Dec 9, 2022 • 5