7 15 4

Lee Park

gogo8232

AI & ML interests

None yet

Recent Activity

upvoted a collection 19 days ago

Gemma 3 Release

upvoted a collection about 1 month ago

Qwen3

new activity 2 months ago

google/gemma-3-4b-it:Please open-source Gemini 1.5 Flash

View all activity

Organizations

gogo8232's activity

upvoted a collection 19 days ago

Gemma 3 Release

Collection

24 items • Updated 13 days ago • 384

upvoted a collection about 1 month ago

Qwen3

Collection

40 items • Updated 22 days ago • 749

New activity in google/gemma-3-4b-it 2 months ago

Please open-source Gemini 1.5 Flash

👍 2

#42 opened 2 months ago by

drguolai

New activity in yuhuili/EAGLE-Qwen2-7B-Instruct 3 months ago

Is there a plan to add draft model of Qwen2.5 7B / 32B & Chiese support ?

#2 opened 7 months ago by

dcknightdc

upvoted a paper 7 months ago

HtmlRAG: HTML is Better Than Plain Text for Modeling Retrieved Knowledge in RAG Systems

Paper • 2411.02959 • Published Nov 5, 2024 • 71

upvoted an article 8 months ago

Article

From DeepSpeed to FSDP and Back Again with Hugging Face Accelerate

and 3 others •

Jun 13, 2024

• 54

updated 6 datasets 11 months ago

upvoted an article 12 months ago

Article

Our Transformers Code Agent beats the GAIA benchmark!

and 1 other •

Jul 1, 2024

• 88

reacted to yushun0410's post with 🔥 12 months ago

Post

4645

Hi Huggingfacers!

Thrilled to introduce Adam-mini, an optimizer that achieves on-par or better performance than AdamW with 45% to 50% less memory footprint. Adam-mini can also achieve 49.5% higher throughput than AdamW on Llama2-7B pre-training.

The design of Adam-mini is inspired by certain Hessian structures we observed on Transformers.

Feel free to try it out! Try switching to Adam-mini with the same hyperparams of AdamW, it would work with only half memory. Hope Adam-mini can help save time, cost, and energy in your tasks!

Paper: "Adam-mini: Use Fewer Learning Rates To Gain More" https://arxiv.org/abs/2406.16793

Code: https://github.com/zyushun/Adam-mini