HtmlRAG: HTML is Better Than Plain Text for Modeling Retrieved Knowledge in RAG Systems
Paper
•
2411.02959
•
Published
•
70
Lee Park
gogo8232
AI & ML interests
None yet
Recent Activity
new activity
25 days ago
yuhuili/EAGLE-Qwen2-7B-Instruct:GQA config change
new activity
25 days ago
yuhuili/EAGLE-Qwen2-72B-Instruct:GQA configuration
Organizations
None yet
gogo8232's activity
Is there a plan to add draft model of Qwen2.5 7B / 32B & Chiese support ?
3
#2 opened 4 months ago
by
dcknightdc
GQA config change
#3 opened 25 days ago
by
gogo8232
GQA configuration
#1 opened 25 days ago
by
gogo8232
check your training metrics
#1 opened 3 months ago
by
gogo8232
upvoted
a
paper
5 months ago
upvoted
an
article
6 months ago
Article
From DeepSpeed to FSDP and Back Again with Hugging Face Accelerate
•
48
updated
a
dataset
8 months ago
updated
5
datasets
9 months ago
gogo8232/ucla-agi-data-mistral-7b-instruct-sppo-iter1
Viewer
•
Updated
•
19.8k
•
49
gogo8232/experiment_first_half
Viewer
•
Updated
•
17.4k
•
43
gogo8232/experiment_perplexity_instruction_llama3_8b_response
Viewer
•
Updated
•
34.9k
•
49
gogo8232/testing-dataset
Viewer
•
Updated
•
19.7k
•
42
gogo8232/experiment_perplexity_instruction
Viewer
•
Updated
•
34.9k
•
48
upvoted
an
article
9 months ago
Article
Our Transformers Code Agent beats the GAIA benchmark!
•
77
reacted to
yushun0410's
post with 🔥
9 months ago
Post
4633
Hi Huggingfacers!
Thrilled to introduce Adam-mini, an optimizer that achieves on-par or better performance than AdamW with 45% to 50% less memory footprint. Adam-mini can also achieve 49.5% higher throughput than AdamW on Llama2-7B pre-training.
The design of Adam-mini is inspired by certain Hessian structures we observed on Transformers.
Feel free to try it out! Try switching to Adam-mini with the same hyperparams of AdamW, it would work with only half memory. Hope Adam-mini can help save time, cost, and energy in your tasks!
Paper: "Adam-mini: Use Fewer Learning Rates To Gain More" https://arxiv.org/abs/2406.16793
Code: https://github.com/zyushun/Adam-mini
Thrilled to introduce Adam-mini, an optimizer that achieves on-par or better performance than AdamW with 45% to 50% less memory footprint. Adam-mini can also achieve 49.5% higher throughput than AdamW on Llama2-7B pre-training.
The design of Adam-mini is inspired by certain Hessian structures we observed on Transformers.
Feel free to try it out! Try switching to Adam-mini with the same hyperparams of AdamW, it would work with only half memory. Hope Adam-mini can help save time, cost, and energy in your tasks!
Paper: "Adam-mini: Use Fewer Learning Rates To Gain More" https://arxiv.org/abs/2406.16793
Code: https://github.com/zyushun/Adam-mini
- 1 reply
upvoted
a
paper
9 months ago