Sylvain Delahaies's picture

3 2

Sylvain Delahaies

sylvain471

·

https://sdelahaies.fr

sylvain471

AI & ML interests

None yet

Recent Activity

reacted to Jaward's post with 👍 2 days ago

minimal single script implementation of knowledge distillation in LLMs. In this implementation, we use GPT-2 (124M) as student model and GPT-2 Medium (340M) as teacher via reverse Kullback-Leibler (KL) divergence, trained on a small chunk of openwebtext. Code: https://github.com/Jaykef/ai-algorithms/blob/main/llm_knowledge_distillation.ipynb

upvoted an article 4 days ago

The LASER technique: Evaluating SVD compression

new activity 5 days ago

HuggingFaceTB/SmolLM-135M:SmolLm and mergekit_moe: is lm_head missing ?

View all activity

Organizations

spaces 1

Drug Classification

models 8

sylvain471/llama3.1-8B-beges-instruct-v2

Updated Nov 2, 2024

sylvain471/llama-base-carbone

Updated Oct 25, 2024

sylvain471/llama3.1-8b-beges-adapter

Updated Sep 23, 2024

sylvain471/Florence2-fuel

Updated Aug 2, 2024

sylvain471/distillBEiT

Image Classification • Updated Jan 19, 2024 • 9

sylvain471/beit_doc_classification

Image Classification • Updated Dec 1, 2023 • 5

sylvain471/tatr_structure_ft

Object Detection • Updated Nov 17, 2023 • 6

sylvain471/tatr_detection_ft

Object Detection • Updated Nov 17, 2023 • 3

datasets 2

sylvain471/beges_qas_fr

Viewer • Updated Oct 31, 2024 • 37.2k • 32

sylvain471/climate_qas_fr

Viewer • Updated Oct 29, 2024 • 8.79k • 32