Maurice Weber

mauriceweber's activity

• All
• Models
• Datasets
• Spaces
• Papers
• Collections
• Community
• Posts
• Upvotes
• Likes

authored a paper 2 days ago

Towards Best Practices for Open Datasets for LLM Training

Paper • 2501.08365 • Published 4 days ago • 40

New activity in togethercomputer/RedPajama-Data-V2 about 2 months ago

Add paper citation

1

#30 opened about 2 months ago by

davanstrien

authored a paper about 2 months ago

RedPajama: an Open Dataset for Training Large Language Models

Paper • 2411.12372 • Published Nov 19, 2024 • 48

New activity in togethercomputer/RedPajama-Data-V2 5 months ago

RPV2 ccnet preprocessing

1

#29 opened 5 months ago by

bpwl0121

New activity in togethercomputer/RedPajama-Data-1T-Sample 6 months ago

sample split details

3

#4 opened about 1 year ago by

sujantkumarkv

New activity in togethercomputer/RedPajama-Data-V2 7 months ago

How can I download the sample-10B fastestly?

1

#28 opened 7 months ago by

zgxiao

updated a dataset 7 months ago

togethercomputer/RedPajama-Data-1T

Viewer • Updated Jun 17, 2024 • 1.73M • 814 • 1.07k

New activity in togethercomputer/RedPajama-Data-1T 9 months ago

defunct book subset

4

#28 opened about 1 year ago by

polinaeterna

New activity in togethercomputer/RedPajama-Data-V2 9 months ago

How much disk space would the whole HF dataset take?

1

#27 opened 9 months ago by

protossw512

updated a dataset 12 months ago

togethercomputer/RedPajama-Data-V2

Updated Nov 21, 2024 • 2.62k • 358

New activity in togethercomputer/RedPajama-Data-V2 12 months ago

rpv2-subsamples

1

#26 opened about 1 year ago by

mauriceweber

New activity in togethercomputer/RedPajama-Data-V2 about 1 year ago

The doc_id in duplicates is should contain?

3

#24 opened about 1 year ago by

newbietuan

Deduplication steps

23

#15 opened about 1 year ago by

ilyayudkovich

Here's a download script parallelized using Spark

1

#22 opened about 1 year ago by

srowen

what is the meaning of snapshots in redpajama-data-v2?

2

#21 opened about 1 year ago by

choidonghun

How to join documents and quality signals when downloading directly

3

#19 opened about 1 year ago by

tgshdyfuhuf

Missing duplicates parquet files

5

#18 opened about 1 year ago by

bebensee

Script to download all files of 1B sample data locally

2

#13 opened about 1 year ago by

ivanzhouyq

What is the total size, of the entirety of this dataset in TB?

1

#10 opened about 1 year ago by

Bayaz

What's the concept on partitions

2

#5 opened about 1 year ago by

SwatCat