File size: 604 Bytes
2351748 02ddd7f 410ca10 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 |
---
title: README
emoji: 👀
colorFrom: purple
colorTo: red
sdk: static
pinned: false
---
## What is this?
A dataprocessing pipeline that uses huggingface datsets as intermediate data store.
Metadata are designed to be updated like a DAG, where some depends on others.
Workflows are gradually being built over time and maybe we'll see hundreds of data repos one day.
## How do I use it?
To load files in local, Huggingface as well as S3 a tool is being developed in progress.
|