GPT-X2.5-135M is finally released! 🚀 The new flagship from Axiomic Labs takes 3rd on the open SLM leaderboard trailing only the SmolLMs, check it out and follow us: AxiomicLabs/GPT-X2.5-135M
I recently released FlameF0X/TinyMoE-100m-2x8-retrained, a small Mixture of Experts language model trained on the Smollm-Corpus. Built on top of the Mixtral architecture, it’s fully compatible with 🤗 Transformers right out of the box!
The model can produce somewhat coherent text on its own, and for some reason, it generates even more coherent responses when given a ChatLM template.
I’m excited to see what you all come up with, and feel free to fine-tune it if you’d like. In the meantime, I’ll be working on developing the chat-trained version.