SMELT: Scaling Laws for Compute-Matched MoE Looped Transformers
Paper • 2609.01343 • Published • 92
Djellal Mohamed Aniss
dmaniss
P(doom) <0.001%
AI & ML interests
SLM, distillation, synthetic data and reasoning.
Recent Activity
updated a collection 4 days ago
wazn published a model 4 days ago
numidlabs/wazn-2b-v0.1 updated a model 4 days ago
numidlabs/wazn-2b-v0.1Organizations
upvoted a paper about 1 month ago
upvoted an article 3 months ago
Article
Beyond LoRA: Can you beat the most popular fine-tuning technique?
- +2
BenjaminB, sayakpaul, hubnemo, kashif
• • 104 upvoted an article 5 months ago
Article
⚡ nano-vLLM: Lightweight, Low-Latency LLM Inference from Scratch
zamal
• • 48 upvoted an article 7 months ago
Article
Finally, a Replacement for BERT: Introducing ModernBERT
- +13
bwarner, NohTow, bclavie, orionweller, ohallstrom, staghado, alexisgallagher, rbiswasfc, fladhak, tomaarsen, ncoop57, griffin, jph00, johnowhitaker, iacolippo
• • 754 upvoted an article 9 months ago
Article
Accelerate ND-Parallel: A guide to Efficient Multi-GPU Training
- +3
smohammadi, siro1, winglian, marcsun13, djsaunde
• • 101 upvoted 2
papers 11 months ago
upvoted 2
papers 12 months ago
upvoted 2
papers about 1 year ago
upvoted an article about 1 year ago
Article
Training and Finetuning Reranker Models with Sentence Transformers
tomaarsen
• • 197 upvoted 4
papers over 1 year ago
LongWriter-Zero: Mastering Ultra-Long Text Generation via Reinforcement Learning
Paper • 2506.18841 • Published • 57
GenRecal: Generation after Recalibration from Large to Small Vision-Language Models
Paper • 2506.15681 • Published • 43
Phi-4-Mini-Reasoning: Exploring the Limits of Small Reasoning Language Models in Math
Paper • 2504.21233 • Published • 50
Distilling LLM Agent into Small Models with Retrieval and Code Tools
Paper • 2505.17612 • Published • 82
upvoted an article over 1 year ago
Article
An Introduction to AI Model Optimization Techniques
PrunaAI
• • 30 upvoted a collection over 1 year ago
upvoted 2
papers almost 2 years ago