SmolVLM: Redefining small and efficient multimodal models
Paper
โข
2504.05299
โข
Published
โข
158
Code Llama
company
AI & ML interests
None defined yet.
Recent Activity
View all activity
codellama's activity
pcuenqย
authored
a
paper
7 days ago
lvwerraย
authored
a
paper
7 days ago
loubnabnlย
authored
a
paper
7 days ago
loubnabnlย
authored
a
paper
2 months ago
lvwerraย
authored
a
paper
2 months ago
lvwerraย
authored
a
paper
3 months ago
andrewrreedย
posted
an
update
3 months ago
Post
2852
๐ Supercharge your LLM apps with Langfuse on Hugging Face Spaces!
Langfuse brings end-to-end observability and tooling to accelerate your dev workflow from experiments through production
Now available as a Docker Space directly on the HF Hub! ๐ค
๐ Trace everything: monitor LLM calls, retrieval, and agent actions with popular frameworks
1โฃ One-click deployment: on Spaces with persistent storage and integrated OAuth
๐ Simple Prompt Management: Version, edit, and update without redeployment
โ Intuitive Evals: Collect user feedback, run model/prompt evaluations, and improve quality
๐ Dataset Creation: Build datasets directly from production data to enhance future performance
Kudos to the Langfuse team for this collab and the awesome, open-first product theyโre building! ๐ @marcklingen @Clemo @MJannik
๐ Space: langfuse/langfuse-template-space
๐ Docs: https://huggingface.co/docs/hub/spaces-sdks-docker-langfuse
Langfuse brings end-to-end observability and tooling to accelerate your dev workflow from experiments through production
Now available as a Docker Space directly on the HF Hub! ๐ค
๐ Trace everything: monitor LLM calls, retrieval, and agent actions with popular frameworks
1โฃ One-click deployment: on Spaces with persistent storage and integrated OAuth
๐ Simple Prompt Management: Version, edit, and update without redeployment
โ Intuitive Evals: Collect user feedback, run model/prompt evaluations, and improve quality
๐ Dataset Creation: Build datasets directly from production data to enhance future performance
Kudos to the Langfuse team for this collab and the awesome, open-first product theyโre building! ๐ @marcklingen @Clemo @MJannik
๐ Space: langfuse/langfuse-template-space
๐ Docs: https://huggingface.co/docs/hub/spaces-sdks-docker-langfuse
- 1 reply
Post
1537
Performance leap: TGI v3 is out. Processes 3x more tokens, 13x faster than vLLM on long prompts. Zero config !
3x more tokens.
By reducing our memory footprint, weโre able to ingest many more tokens and more dynamically than before. A single L4 (24GB) can handle 30k tokens on llama 3.1-8B, while vLLM gets barely 10k. A lot of work went into reducing the footprint of the runtime and its effect are best seen on smaller constrained environments.
13x faster
On long prompts (200k+ tokens) conversation replies take 27.5s in vLLM, while it takes only 2s in TGI. How so ? We keep the initial conversation around, so when a new reply comes in, we can answer almost instantly. The overhead of the lookup is ~5us. Thanks @Dani รซl de Kok for the beast data structure.
Zero config
Thatโs it. Remove all the flags your are using and youโre likely to get the best performance. By evaluating the hardware and model, TGI carefully selects automatic values to give best performance. In production, we donโt have any flags anymore in our deployments. We kept all existing flags around, they may come in handy in niche scenarios.
Read more: https://huggingface.co/docs/text-generation-inference/conceptual/chunking
3x more tokens.
By reducing our memory footprint, weโre able to ingest many more tokens and more dynamically than before. A single L4 (24GB) can handle 30k tokens on llama 3.1-8B, while vLLM gets barely 10k. A lot of work went into reducing the footprint of the runtime and its effect are best seen on smaller constrained environments.
13x faster
On long prompts (200k+ tokens) conversation replies take 27.5s in vLLM, while it takes only 2s in TGI. How so ? We keep the initial conversation around, so when a new reply comes in, we can answer almost instantly. The overhead of the lookup is ~5us. Thanks @Dani รซl de Kok for the beast data structure.
Zero config
Thatโs it. Remove all the flags your are using and youโre likely to get the best performance. By evaluating the hardware and model, TGI carefully selects automatic values to give best performance. In production, we donโt have any flags anymore in our deployments. We kept all existing flags around, they may come in handy in niche scenarios.
Read more: https://huggingface.co/docs/text-generation-inference/conceptual/chunking
Post
3424
Making SmolLM2 reproducible: open-sourcing our training & evaluation toolkit ๐ ๏ธ https://github.com/huggingface/smollm/
- Pre-training code with nanotron
- Evaluation suite with lighteval
- Synthetic data generation using distilabel (powers our new SFT dataset HuggingFaceTB/smoltalk)
- Post-training scripts with TRL & the alignment handbook
- On-device tools with llama.cpp for summarization, rewriting & agents
Apache 2.0 licensed. V2 pre-training data mix coming soon!
Which other tools should we add next?
- Pre-training code with nanotron
- Evaluation suite with lighteval
- Synthetic data generation using distilabel (powers our new SFT dataset HuggingFaceTB/smoltalk)
- Post-training scripts with TRL & the alignment handbook
- On-device tools with llama.cpp for summarization, rewriting & agents
Apache 2.0 licensed. V2 pre-training data mix coming soon!
Which other tools should we add next?
andrewrreedย
posted
an
update
5 months ago
Post
1064
Trace LLM calls with Arize AI's Phoenix observability dashboards on Hugging Face Spaces! ๐
โจ I just added a new recipe to the Open-Source AI Cookbook that shows you how to:
1๏ธโฃ Deploy Phoenix on HF Spaces with persistent storage in a few clicks
2๏ธโฃ Configure LLM tracing with the ๐ฆ๐ฒ๐ฟ๐๐ฒ๐ฟ๐น๐ฒ๐๐ ๐๐ป๐ณ๐ฒ๐ฟ๐ฒ๐ป๐ฐ๐ฒ ๐๐ฃ๐
3๏ธโฃ Observe multi-agent application runs with the CrewAI integration
๐ข๐ฏ๐๐ฒ๐ฟ๐๐ฎ๐ฏ๐ถ๐น๐ถ๐๐ ๐ถ๐ ๐ฐ๐ฟ๐๐ฐ๐ถ๐ฎ๐น for building robust LLM apps.
Phoenix makes it easy to visualize trace data, evaluate performance, and track down issues. Give it a try!
๐ Cookbook recipe: https://huggingface.co/learn/cookbook/en/phoenix_observability_on_hf_spaces
๐ Phoenix docs: https://docs.arize.com/phoenix
โจ I just added a new recipe to the Open-Source AI Cookbook that shows you how to:
1๏ธโฃ Deploy Phoenix on HF Spaces with persistent storage in a few clicks
2๏ธโฃ Configure LLM tracing with the ๐ฆ๐ฒ๐ฟ๐๐ฒ๐ฟ๐น๐ฒ๐๐ ๐๐ป๐ณ๐ฒ๐ฟ๐ฒ๐ป๐ฐ๐ฒ ๐๐ฃ๐
3๏ธโฃ Observe multi-agent application runs with the CrewAI integration
๐ข๐ฏ๐๐ฒ๐ฟ๐๐ฎ๐ฏ๐ถ๐น๐ถ๐๐ ๐ถ๐ ๐ฐ๐ฟ๐๐ฐ๐ถ๐ฎ๐น for building robust LLM apps.
Phoenix makes it easy to visualize trace data, evaluate performance, and track down issues. Give it a try!
๐ Cookbook recipe: https://huggingface.co/learn/cookbook/en/phoenix_observability_on_hf_spaces
๐ Phoenix docs: https://docs.arize.com/phoenix
Post
3878
Native tensor parallel has landed in transformers!!! https://github.com/huggingface/transformers/pull/34184 thanks a lot to the torch team for their support!
Contributions are welcome to support more models! ๐ฅ
Contributions are welcome to support more models! ๐ฅ
lvwerraย
authored
a
paper
5 months ago
lvwerraย
authored
a
paper
10 months ago
loubnabnlย
authored
a
paper
10 months ago
Post
5885
๐ท FineWeb technical report is out and so is ๐ FineWeb-Edu, a 1.3 trillion tokens dataset that outperforms all other open web datasets, with remarkable improvements on educational benchmarksย such as MMLU, ARC, and OpenBookQA.
Technical report: HuggingFaceFW/blogpost-fineweb-v1
Dataset: HuggingFaceFW/fineweb-edu
We used Llama 3 generations to train an educational quality classifier, filtering the 15 trillion tokens of FineWeb to select only those with high educational value (an approach also used in Llama 3 and Phi-3 training datasets). We're releasing both FineWeb-Edu and the classifier, along with a larger, less heavily filtered version containing 5.4 trillion tokens.
You can find more details about the dataset and the experiments we ran in the FineWeb technical report, It's a 45-minute read but it contains all the secret sauce for building high quality web datasets.
Enjoy!
Technical report: HuggingFaceFW/blogpost-fineweb-v1
Dataset: HuggingFaceFW/fineweb-edu
We used Llama 3 generations to train an educational quality classifier, filtering the 15 trillion tokens of FineWeb to select only those with high educational value (an approach also used in Llama 3 and Phi-3 training datasets). We're releasing both FineWeb-Edu and the classifier, along with a larger, less heavily filtered version containing 5.4 trillion tokens.
You can find more details about the dataset and the experiments we ran in the FineWeb technical report, It's a 45-minute read but it contains all the secret sauce for building high quality web datasets.
Enjoy!
lvwerraย
authored
a
paper
11 months ago
loubnabnlย
authored
a
paper
11 months ago
Post
1975
text-generation-inference v2.0.3 is out.
Main new features:
- Falcon2 support
- PaliGemma support
- New faster speculation method from IBM !
https://github.com/huggingface/text-generation-inference/releases
Main new features:
- Falcon2 support
- PaliGemma support
- New faster speculation method from IBM !
https://github.com/huggingface/text-generation-inference/releases
andrewrreedย
posted
an
update
11 months ago
Post
2613
๐ฌ Open LLM Progress Tracker ๐ฌ
Inspired by the awesome work from @mlabonne , I created a Space to monitor the narrowing gap between open and proprietary LLMs as scored by the LMSYS Chatbot Arena ELO ratings ๐ค
The goal is to have a continuously updated place to easily visualize these rapidly evolving industry trends ๐
๐ Open LLM Progress Tracker: andrewrreed/closed-vs-open-arena-elo
๐ Source of Inspiration: https://www.linkedin.com/posts/maxime-labonne_arena-elo-graph-updated-with-new-models-activity-7187062633735368705-u2jB/
Inspired by the awesome work from @mlabonne , I created a Space to monitor the narrowing gap between open and proprietary LLMs as scored by the LMSYS Chatbot Arena ELO ratings ๐ค
The goal is to have a continuously updated place to easily visualize these rapidly evolving industry trends ๐
๐ Open LLM Progress Tracker: andrewrreed/closed-vs-open-arena-elo
๐ Source of Inspiration: https://www.linkedin.com/posts/maxime-labonne_arena-elo-graph-updated-with-new-models-activity-7187062633735368705-u2jB/
-
- 2 replies
Post
1295
text-generation-inference 2.0.2 is out.
- Native support for Idefics2, with much better efficiency than llava 1.6 (next) !
Phi3, Increase VLM support in the openai layer.
Release notes https://github.com/huggingface/text-generation-inference/releases/tag/v2.0.2
- Native support for Idefics2, with much better efficiency than llava 1.6 (next) !
Phi3, Increase VLM support in the openai layer.
Release notes https://github.com/huggingface/text-generation-inference/releases/tag/v2.0.2