Bridging the Data Provenance Gap Across Text, Speech and Video
Paper
•
2412.17847
•
Published
•
9
Flax Community
non-profit
AI & ML interests
JAX, Flax, TPU, 🤗
Recent Activity
View all activity
flax-community's activity
vumichien
authored
a
paper
9 days ago
Adding `safetensors` variant of this model
#1 opened about 1 month ago
by
SFconvertbot
gagan3012
authored
a
paper
2 months ago
Post
1529
My latest project is the outcome of the last 2+ years working with TPUs from the amazing TPU Research Cloud (TRC) program and training Encoder-only LMs with the TensorFlow Model Garden library.
👉 Link: https://github.com/stefan-it/model-garden-lms
An overview of some features:
- Cheatsheet for setting-up a TPU VM Pod (with all necessary dependencies) to pretrain LMs with TF Model Garden
- Conversion scripts that convert TF Model Garden weights to Hugging Face Transformers-compatible models
- Supported architectures include BERT, BERT with Token Dropping and TEAMS
I also released BERT-based models pretrained on the great Hugging Face FineWeb and FineWeb-Edu datasets (10BT subset). With more to come!
👉 Model Hub Link: https://huggingface.co/model-garden-lms
If you find these resources useful, please give them a like!
Made from Bavarian Oberland with ❤️ and 🥨.
👉 Link: https://github.com/stefan-it/model-garden-lms
An overview of some features:
- Cheatsheet for setting-up a TPU VM Pod (with all necessary dependencies) to pretrain LMs with TF Model Garden
- Conversion scripts that convert TF Model Garden weights to Hugging Face Transformers-compatible models
- Supported architectures include BERT, BERT with Token Dropping and TEAMS
I also released BERT-based models pretrained on the great Hugging Face FineWeb and FineWeb-Edu datasets (10BT subset). With more to come!
👉 Model Hub Link: https://huggingface.co/model-garden-lms
If you find these resources useful, please give them a like!
Made from Bavarian Oberland with ❤️ and 🥨.
christopher
posted
an
update
3 months ago
Post
1682
The folks at Foursquare released a dataset of 104.5 million places of interest (
foursquare/fsq-os-places) and here's all of them on a plot
-
- 4 replies
christopher
posted
an
update
3 months ago
Post
2401
The Lichess database of games, puzzles, and engine evaluations is now on the Hub: https://huggingface.co/Lichess
Billions of chess data points to download, query, and stream and we're excited to see what you'll build with it! ♟️ 🤗
- Lichess/positions-datasets-66f50837db5cd3287d60d489
- Lichess/games-datasets-66f508df78f4b43e1bb2d353
Billions of chess data points to download, query, and stream and we're excited to see what you'll build with it! ♟️ 🤗
- Lichess/positions-datasets-66f50837db5cd3287d60d489
- Lichess/games-datasets-66f508df78f4b43e1bb2d353
infinitylogesh
authored
a
paper
3 months ago
gagan3012
authored
a
paper
4 months ago
nbroad
authored
a
paper
4 months ago
mariagrandury
authored
a
paper
5 months ago
christopher
posted
an
update
6 months ago
Post
1329
4 million chess puzzles
gagan3012
authored
a
paper
7 months ago
vumichien
authored
a
paper
7 months ago
mariagrandury
authored
a
paper
7 months ago
Post
1304
Llama 3.1 405B Instruct beats GPT-4o on MixEval-Hard
Just ran MixEval for 405B, Sonnet-3.5 and 4o, with 405B landing right between the other two at 66.19
The GPT-4o result of 64.7 replicated locally but Sonnet-3.5 actually scored 70.25/69.45 in my replications 🤔 Still well ahead of the other 2 though.
Sammple of 1 of the eval calls here: https://wandb.ai/morgan/MixEval/weave/calls/07b05ae2-2ef5-4525-98a6-c59963b76fe1
Quick auto-logging tracing for openai-compatible clients and many more here: https://wandb.github.io/weave/quickstart/
Just ran MixEval for 405B, Sonnet-3.5 and 4o, with 405B landing right between the other two at 66.19
The GPT-4o result of 64.7 replicated locally but Sonnet-3.5 actually scored 70.25/69.45 in my replications 🤔 Still well ahead of the other 2 though.
Sammple of 1 of the eval calls here: https://wandb.ai/morgan/MixEval/weave/calls/07b05ae2-2ef5-4525-98a6-c59963b76fe1
Quick auto-logging tracing for openai-compatible clients and many more here: https://wandb.github.io/weave/quickstart/
gagan3012
authored
a
paper
7 months ago
mariagrandury
authored
a
paper
8 months ago
vumichien
authored
a
paper
8 months ago