Hugging Face's logo
Sylvain Delahaies's picture
3 2

Sylvain Delahaies

sylvain471
·
https://sdelahaies.fr

AI & ML interests

None yet

Recent Activity

reacted to Jaward's post with 👍 2 days ago
minimal single script implementation of knowledge distillation in LLMs. In this implementation, we use GPT-2 (124M) as student model and GPT-2 Medium (340M) as teacher via reverse Kullback-Leibler (KL) divergence, trained on a small chunk of openwebtext. Code: https://github.com/Jaykef/ai-algorithms/blob/main/llm_knowledge_distillation.ipynb
upvoted an article 4 days ago
The LASER technique: Evaluating SVD compression
new activity 5 days ago
HuggingFaceTB/SmolLM-135M:SmolLm and mergekit_moe: is lm_head missing ?
View all activity

Organizations

sdata's profile picture q71's profile picture

sylvain471's activity

upvoted an article 4 days ago