Hymba: A Hybrid-head Architecture for Small Language Models
Paper
•
2411.13676
•
Published
•
45
•
3
Pavlo Molchanov PRO
pmolchanov
AI & ML interests
Efficiency in Multi-Modal LLMs
Recent Activity
authored
a paper
17 days ago
PHI-S: Distribution Balancing for Label-Free Multi-Teacher Distillation
authored
a paper
17 days ago
EoRA: Training-free Compensation for Compressed LLM with Eigenspace
Low-Rank Approximation
authored
a paper
17 days ago
Hymba: A Hybrid-head Architecture for Small Language Models
Organizations
pmolchanov's activity
Update LMFlow support
#6 opened 5 months ago
by
shizhediao2
Update LMFlow support
#10 opened 5 months ago
by
shizhediao2
when will the Hymba-125M-Instruct be released? thanks
1
#6 opened 5 months ago
by
nemonameless
How good would this be for RAG?
2
#2 opened 5 months ago
by
Unomagan
How well this model can inference on cpu only devices?
2
#5 opened 5 months ago
by
mhdaw
A collection of observations in the code
5
2
#1 opened 6 months ago
by
AntonV
Add base model metadata
#7 opened 5 months ago
by
davanstrien
Add link to paper, update tags
#3 opened 5 months ago
by
nielsr
Update README.md
1
#4 opened 5 months ago
by
mishig
commented
a
paper
6 months ago
commented
a
paper
7 months ago
Mistral-NeMo-Minitron-8B-Chat
1
5
#5 opened 8 months ago
by
rasyosef
Awesome model, instruct version as well possibly?
3
#2 opened 9 months ago
by
OPPEYRADY
Context Length
6
1
#4 opened 9 months ago
by
Sao10K
commented
a
paper
9 months ago
Where is Minitron-4B-Instruct?
1
#2 opened 10 months ago
by
mgoin
Import error
5
#1 opened about 1 year ago
by
halidziya1