2.68k
The Ultra-Scale Playbook
🌌
The ultimate guide to training LLM on large GPU Clusters
Akim Mousterou
AkimfromParis
AI & ML interests
Natural Language Processing for French, English, and Japanese | - - - |
NLP in finance and economics, opinion mining, and quantitative research.
Recent Activity
new activity
5 days ago
mcp-course/README:No Submit button after Unit 1 Quiz
new activity
20 days ago
llm-jp/ja-vg-vqa-conversation:Which dataset is suitable for JA-VG-VQA conversion?
new activity
26 days ago
llm-jp/open-japanese-llm-leaderboard:Benchmarking Stability Japanese Stable LM variants?
Organizations
AkimfromParis's activity
No Submit button after Unit 1 Quiz
1
#9 opened 8 days ago
by
AkimfromParis
Which dataset is suitable for JA-VG-VQA conversion?
2
#1 opened 20 days ago
by
sabaridsnfuji
Benchmarking Stability Japanese Stable LM variants?
2
#2 opened 27 days ago
by
erclee2
Qwen2.5-72B-Instruct results?
1
#1 opened about 2 months ago
by
yellowtown
liked
a
Space
4 months ago
upvoted
a
paper
4 months ago
liked
a
Space
4 months ago
reacted to
lewtun's
post with 🔥
4 months ago
Post
10398
We are reproducing the full DeepSeek R1 data and training pipeline so everybody can use their recipe. Instead of doing it in secret we can do it together in the open!
🧪 Step 1: replicate the R1-Distill models by distilling a high-quality reasoning corpus from DeepSeek-R1.
🧠 Step 2: replicate the pure RL pipeline that DeepSeek used to create R1-Zero. This will involve curating new, large-scale datasets for math, reasoning, and code.
🔥 Step 3: show we can go from base model -> SFT -> RL via multi-stage training.
Follow along: https://github.com/huggingface/open-r1
🧪 Step 1: replicate the R1-Distill models by distilling a high-quality reasoning corpus from DeepSeek-R1.
🧠 Step 2: replicate the pure RL pipeline that DeepSeek used to create R1-Zero. This will involve curating new, large-scale datasets for math, reasoning, and code.
🔥 Step 3: show we can go from base model -> SFT -> RL via multi-stage training.
Follow along: https://github.com/huggingface/open-r1
- 5 replies
reacted to
burtenshaw's
post with 🤗
5 months ago
Post
4071
🚧 Work in Progress! 🚧
👷♀️ We're working hard on getting the official agents course ready for the 50,000 students that have signed up.
If you want to contribute to the discussion, I started these community posts. Looking forward to hearing from you:
- smolagents unit in the agents course - agents-course/README#7
- LlamaIndex Unit in the agents course - agents-course/README#6
- LangChain and LangGraph unit in the agents course - agents-course/README#5
- Real world use cases in the agents course - agents-course/README#8
👷♀️ We're working hard on getting the official agents course ready for the 50,000 students that have signed up.
If you want to contribute to the discussion, I started these community posts. Looking forward to hearing from you:
- smolagents unit in the agents course - agents-course/README#7
- LlamaIndex Unit in the agents course - agents-course/README#6
- LangChain and LangGraph unit in the agents course - agents-course/README#5
- Real world use cases in the agents course - agents-course/README#8
reacted to
davidberenstein1957's
post with 👀
5 months ago
Post
1266
You can now use the "Synthetic Data Generator" at a much larger scale with your preferred inference engine: Ollama, vLLM, TGI, and serverless inference! 🔥
Install, configure, launch!
Space: https://huggingface.co/spaces/argilla/synthetic-data-generator?duplicate=true
Examples: https://github.com/argilla-io/synthetic-data-generator/tree/main/examples
Install, configure, launch!
Space: https://huggingface.co/spaces/argilla/synthetic-data-generator?duplicate=true
Examples: https://github.com/argilla-io/synthetic-data-generator/tree/main/examples
replied to
MoritzLaurer's
post
5 months ago
OpenAI sales revenues forecasted at $11.6 billion for 2025. So they will probably be positive.
Maybe you can burn cash, when you have a valuation at $157B?! Numbers are really crazy, only history will tell…
How to use ModernBERT with the AutoModelForQuestionAnswering class?
➕
3
4
#15 opened 6 months ago
by
sraj
posted
an
update
5 months ago
Post
1803
💵 Polymarket is leveraging “Chatbot Arena LLM Leaderboard” on HuggingFace for online gambling on the “Top AI model on January 31?”. 🤗
As of January 3rd, 2025:
-1./ Gemini (83%) -2./ ChatGPT (13%) -3./ Other (2%) -4./ Claude (2%) -5./ Grok (1%) -6./ Llama (<1%)
🇺🇸 The market opinion is following historical data. It's clearly bias towards US historical AI giants, yet Polymarket is forbidden in the USA and for US citizens.
🇨🇳 In the “Other”, you might have Chinese AI labs that are probably the future AI leaders (Qwen, DeepSeek, Yi).
⚖️ In the market resolution, if two models are tied in the evaluation, they will take the alphabetical order. (e.g. if both were tied, “Google” would resolve to “Yes”, and “xAI” would resolve to “No”). 🙃
That might be illegal usage of the Chatbot Arena policy? And maybe HuggingFace? @clem
Or maybe authors and contributors should get a cut each month as “market markers”. @weichiang @angelopoulos
As of January 3rd, 2025:
-1./ Gemini (83%) -2./ ChatGPT (13%) -3./ Other (2%) -4./ Claude (2%) -5./ Grok (1%) -6./ Llama (<1%)
🇺🇸 The market opinion is following historical data. It's clearly bias towards US historical AI giants, yet Polymarket is forbidden in the USA and for US citizens.
🇨🇳 In the “Other”, you might have Chinese AI labs that are probably the future AI leaders (Qwen, DeepSeek, Yi).
⚖️ In the market resolution, if two models are tied in the evaluation, they will take the alphabetical order. (e.g. if both were tied, “Google” would resolve to “Yes”, and “xAI” would resolve to “No”). 🙃
That might be illegal usage of the Chatbot Arena policy? And maybe HuggingFace? @clem
Or maybe authors and contributors should get a cut each month as “market markers”. @weichiang @angelopoulos
- 1 reply
upvoted
an
article
5 months ago
Article
🇪🇺✍️ EU AI Act: Systemic Risks in the First CoP Draft Comments ✍️🇪🇺
By
and 1 other
•
•
14
liked
a
Space
5 months ago
replied to
their
post
5 months ago
Kievan Rus' was coined by modern Russian historian. You are supporting the idea of the Old Russian empire, and forgetting the history of Ukraine, and Kyiv. When Kyiv was at the center of eastern salvic and finnic tribes breakdown in multiple states and principalities.
Please don't patronize me. I will not "carry on", or "close this discussion" at your wish. But you can always delete your messages. : )
replied to
their
post
6 months ago
Origin of Ukraine... "Kievan Rus', also known as Kyivan Rus', was the first East Slavic state and later an amalgam of principalities in Eastern Europe from the late 9th to the mid-13th century.*
In 1965, it's Ukrainian Soviet Socialist Republic. : )
You don't talk to LLM. "Talk" is an active verb that involves a very human action. You are rude to your fellow human but anthropomorphizing LLM.
Best,
replied to
their
post
6 months ago
Everything I wrote is a fact... You cannot be impolite and play the victim in the same time.
replied to
their
post
6 months ago
A polite human would use "In my humble opinion..." or "Please not that..."... And not use "No." or even "Carry on..."
Ivanenkho famously refused busy Moscow for the quietness of Kyiv because he was criticized by the party for being too abstract.
I wish I could used the USSR flag but US big brother doesn't have it. Maybe I will change the Ukrainian flag to Ukrainian Soviet Socialist Republic...
replied to
their
post
6 months ago
I wrote few points based on the Deep Learning history of Prof. Schmidhuber.
I wrote USSR, and I even put the little flag with hammer and chisel. : ) Ivaknenko is born in March 1913 in Kobeliaky and die in Kyiv, Ukraine back in 2007. He did most of his research in Kyiv. You can be Soviet and Ukrainian...
Ivaknenko did also his primary school, high-school, and Phd in Ukraine. But yes, he did his Master's Degree in St Petersburg. You don't get your nationality from MSc. That's non sequitur.
Ps: You can disagree with my post, but please be more polite when you reply. : )