Merge Crew

Activity Feed Request to join this org

AI & ML interests

Merging models

Recent Activity

birgermoell authored a paper 21 days ago

Voice Cloning for Dysarthric Speech Synthesis: Addressing Data Scarcity in Speech-Language Pathology

KennethEnevoldsen authored a paper about 1 month ago

TextDescriptives: A Python package for calculating a large variety of metrics from text

KennethEnevoldsen authored a paper about 1 month ago

MMTEB: Massive Multilingual Text Embedding Benchmark

View all activity

merge-crew's activity

mlabonne

posted an update 7 days ago

Post

6483

✂️ AutoAbliteration

I made a Colab notebook to automatically abliterate models.

It's quite general, so you can do interesting stuff like blocking a given language in the model outputs.

💻 Colab: https://colab.research.google.com/drive/1RmLv-pCMBBsQGXQIM8yF-OdCNyoylUR1?usp=sharing

mlabonne

posted an update 8 days ago

Post

5950

✂️ Gemma 3 Abliterated

I noticed that Gemma 3 was much more resilient to refusal removal than other models like Qwen 2.5.

I experimented with different recipes and improved the abliteration technique I wrote about last year.

It's still experimental but the refusal rate is super low in my tests. Enjoy!

mlabonne/gemma-3-4b-it-abliterated
mlabonne/gemma-3-12b-it-abliterated
mlabonne/gemma-3-27b-it-abliterated

1 reply

birgermoell

authored a paper 21 days ago

Voice Cloning for Dysarthric Speech Synthesis: Addressing Data Scarcity in Speech-Language Pathology

Paper • 2503.01266 • Published 22 days ago

KennethEnevoldsen

authored 2 papers about 1 month ago

TextDescriptives: A Python package for calculating a large variety of metrics from text

Paper • 2301.02057 • Published Jan 5, 2023

MMTEB: Massive Multilingual Text Embedding Benchmark

Paper • 2502.13595 • Published Feb 19 • 33

birgermoell

authored 2 papers about 1 month ago

Language Complexity Measurement as a Noisy Zero-Shot Proxy for Evaluating LLM Performance

Paper • 2502.11578 • Published Feb 17

Large Language Models and Mathematical Reasoning Failures

Paper • 2502.11574 • Published Feb 17 • 3

mlabonne

posted an update 2 months ago

Post

6261

🆕 LLM Course 2025 edition!

I updated the LLM Scientist roadmap and added a ton of new information and references. It covers training, datasets, evaluation, quantization, and new trends like test-time compute scaling.

The LLM Course has been incredibly popular (41.3k stars!) and I've been touched to receive many, many messages about how it helped people in their careers.

I know how difficult this stuff can be, so I'm super proud of the impact it had. I want to keep updating it in 2025, especially with the LLM Engineer roadmap.

Thanks everyone, hope you'll enjoy it!

💻 LLM Course: https://huggingface.co/blog/mlabonne/llm-course

mlabonne

authored a paper 4 months ago

Merging in a Bottle: Differentiable Adaptive Merging (DAM) and the Path from Averaging to Automation

Paper • 2410.08371 • Published Oct 10, 2024 • 1

SyedAbdul

authored a paper 5 months ago

SHAKTI: A 2.5 Billion Parameter Small Language Model Optimized for Edge AI and Low-Resource Environments

Paper • 2410.11331 • Published Oct 15, 2024 • 8

KennethEnevoldsen

authored 4 papers 7 months ago

Encoder vs Decoder: Comparative Analysis of Encoder and Decoder Language Models on Multilingual NLU Tasks

Paper • 2406.13469 • Published Jun 19, 2024

saattrupdan

authored a paper 8 months ago

Encoder vs Decoder: Comparative Analysis of Encoder and Decoder Language Models on Multilingual NLU Tasks

Paper • 2406.13469 • Published Jun 19, 2024

mlabonne

posted an update 9 months ago

Post

19594

Large models are surprisingly bad storytellers.

I asked 8 LLMs to "Tell me a bedtime story about bears and waffles."

Claude 3.5 Sonnet and GPT-4o gave me the worst stories: no conflict, no moral, zero creativity.

In contrast, smaller models were quite creative and wrote stories involving talking waffle trees and bears ostracized for their love of waffles.

Here you can see a comparison between Claude 3.5 Sonnet and NeuralDaredevil-8B-abliterated. They both start with a family of bears but quickly diverge in terms of personality, conflict, etc.

I mapped it to the hero's journey to have some kind of framework. Prompt engineering can definitely help here, but it's still disappointing that the larger models don't create better stories right off the bat.

Do you know why smaller models outperform the frontier models here?