Towards Best Practices for Open Datasets for LLM Training
Paper
β’
2501.08365
β’
Published
β’
40
CompVis
university
AI & ML interests
Generative approaches for visual synthesis, Invertible deep models for explainable AI, Deep metric and representation learning, self-supervised learning paradigms
Recent Activity
View all activity
CompVis's activity
lvwerraΒ
authored
a
paper
2 days ago
stellaathenaΒ
authored
a
paper
2 days ago
Post
5419
Google drops Gemini 2.0 Flash Thinking
a new experimental model that unlocks stronger reasoning capabilities and shows its thoughts. The model plans (with thoughts visible), can solve complex problems with Flash speeds, and more
now available in anychat, try it out: akhaliq/anychat
a new experimental model that unlocks stronger reasoning capabilities and shows its thoughts. The model plans (with thoughts visible), can solve complex problems with Flash speeds, and more
now available in anychat, try it out: akhaliq/anychat
- 1 reply
YfZΒ
authored
7
papers
30 days ago
Towards Aligned Layout Generation via Diffusion Model with Aesthetic Constraints
Paper
β’
2402.04754
β’
Published
LLaVA-Read: Enhancing Reading Ability of Multimodal Language Models
Paper
β’
2407.19185
β’
Published
β’
1
ARTIST: Improving the Generation of Text-rich Images by Disentanglement
Paper
β’
2406.12044
β’
Published
MMR: Evaluating Reading Ability of Large Multimodal Models
Paper
β’
2408.14594
β’
Published
TextLap: Customizing Language Models for Text-to-Layout Planning
Paper
β’
2410.12844
β’
Published
LoRA-Contextualizing Adaptation of Large Multimodal Models for Long Document Understanding
Paper
β’
2411.01106
β’
Published
β’
4
SUGAR: Subject-Driven Video Customization in a Zero-Shot Manner
Paper
β’
2412.10533
β’
Published
β’
5
Post
2235
Introducing ππ
π’π§πππππ‘: the best public math pre-training dataset with 50B+ tokens!
HuggingFaceTB/finemath
Math remains challenging for LLMs and by training on FineMath we see considerable gains over other math datasets, especially on GSM8K and MATH.
We build the dataset by:
π οΈ carefully extracting math data from Common Crawl;
π iteratively filtering and recalling high quality math pages using a classifier trained on synthetic annotations to identify math reasoning and deduction.
We conducted a series of ablations comparing the performance of Llama-3.2-3B-Base after continued pre-training on FineMath and observe notable gains compared to the baseline model and other public math datasets.
We hope this helps advance the performance of LLMs on math and reasoning! π
Weβre also releasing all the ablation models as well as the evaluation code.
HuggingFaceTB/finemath-6763fb8f71b6439b653482c2
HuggingFaceTB/finemath
Math remains challenging for LLMs and by training on FineMath we see considerable gains over other math datasets, especially on GSM8K and MATH.
We build the dataset by:
π οΈ carefully extracting math data from Common Crawl;
π iteratively filtering and recalling high quality math pages using a classifier trained on synthetic annotations to identify math reasoning and deduction.
We conducted a series of ablations comparing the performance of Llama-3.2-3B-Base after continued pre-training on FineMath and observe notable gains compared to the baseline model and other public math datasets.
We hope this helps advance the performance of LLMs on math and reasoning! π
Weβre also releasing all the ablation models as well as the evaluation code.
HuggingFaceTB/finemath-6763fb8f71b6439b653482c2
fix website link
1
#2 opened about 1 month ago
by
qwertyforce
Add pipeline tag, enable download stats
3
#1 opened about 1 month ago
by
nielsr
kliyerΒ
updated
a
model
about 1 month ago
Add pipeline tag, enable download stats
3
#1 opened about 1 month ago
by
nielsr
kolja-bΒ
updated
a
model
about 1 month ago
kolja-bΒ
authored
a
paper
about 1 month ago
stefan-baumannΒ
authored
a
paper
about 1 month ago
kliyerΒ
authored
a
paper
about 1 month ago
Post
6530
QwQ-32B-Preview is now available in anychat
A reasoning model that is competitive with OpenAI o1-mini and o1-preview
try it out: akhaliq/anychat
A reasoning model that is competitive with OpenAI o1-mini and o1-preview
try it out: akhaliq/anychat
- 1 reply