stas/ml-engineering-book
Updated
• 26
Stas Bekman
stas
AI & ML interests
Toolmaker. Software creator, optimizer and harmonizer.
Makes things work and fly at Snowflake AI Research
Training LLM/RAG/Generative AI/Machine Learning/Scalability
Recent Activity
posted an
update
about 3 hours ago
Good news! Ulysses Sequence Parallelism from the Snowflake AI Research and the Deepspeed teams has been integrated into
HuggingFace Trainer, Accelerate and TRL
For extensive details please see this writeup:
https://huggingface.co/blog/ulysses-sp
Thanks a lot to Kashif Rasul for helping make it happen. Also the others in the HF team who helped with integration. published an
article
2 days ago
Ulysses Sequence Parallelism: Training with Million-Token Contexts authored
a paper
about 1 month ago
Arctic Long Sequence Training: Scalable And Efficient Training For
Multi-Million Token Sequences Organizations
stas/tiny-random-llama-2
Text Generation • 104k • Updated
• 3.93k • 41
stas/tiny-m2m_100
Updated
• 3.35k
stas/tr8b-104B-debug3
Updated
stas/pegasus-cnn_dailymail-tiny-random
Updated
• 761
stas/mt5-tiny-random
Updated
• 116 • 2
stas/tiny-wmt19-en-de
Updated
• 105k • 1
stas/tiny-wmt19-en-ru
Updated
• 301
stas/t5-very-small-random
Updated
• 14 • 1