epfl-nlp/ConLID
Text Classification
•
Updated
•
32
Jakhongir Saydaliev
Jakh0103
AI & ML interests
None yet
Recent Activity
updated
a model
12 days ago
epfl-nlp/ConLID
published
a model
12 days ago
epfl-nlp/ConLID
updated
a collection
12 days ago
Visual-Intelligence
Organizations
Jakh0103's activity
updated
a
model
12 days ago
published
a
model
12 days ago
updated
a
collection
12 days ago
updated
a
model
18 days ago
published
a
model
18 days ago
updated
a
model
about 1 month ago
published
a
model
about 1 month ago
updated
a
model
about 1 month ago
published
a
model
about 1 month ago
reacted to
kargaranamir's
post with 👍
about 2 months ago
Post
1406
Introducing GlotCC: a new 2TB corpus based on an early 2024 CommonCrawl snapshot with data for 1000+ languages.
🤗 corpus v1: cis-lmu/GlotCC-V1
🐱 pipeline v3: https://github.com/cisnlp/GlotCC
More details? Stay tuned for our upcoming paper.
More data? In the next version, we plan to include additional snapshots of CommonCrawl.
Limitation: Due to the lower frequency of low-resource languages compared to others, there are sometimes only a few sentences available for very low-resource languages. However, the data volume for English in this version stands at 750GB, and the top 200 languages still have a strong presence in our data (see plot attached; we write the index for every 20 languages, meaning the 10th index is the 200th language).
🤗 corpus v1: cis-lmu/GlotCC-V1
🐱 pipeline v3: https://github.com/cisnlp/GlotCC
More details? Stay tuned for our upcoming paper.
More data? In the next version, we plan to include additional snapshots of CommonCrawl.
Limitation: Due to the lower frequency of low-resource languages compared to others, there are sometimes only a few sentences available for very low-resource languages. However, the data volume for English in this version stands at 750GB, and the top 200 languages still have a strong presence in our data (see plot attached; we write the index for every 20 languages, meaning the 10th index is the 200th language).
reacted to
kargaranamir's
post with 👍
about 2 months ago
Post
A Text Language Identification Model with Support for +2000 Labels:
space: cis-lmu/glotlid-space
model: cis-lmu/glotlid
github: https://github.com/cisnlp/GlotLID
paper: GlotLID: Language Identification for Low-Resource Languages (2310.16248)
space: cis-lmu/glotlid-space
model: cis-lmu/glotlid
github: https://github.com/cisnlp/GlotLID
paper: GlotLID: Language Identification for Low-Resource Languages (2310.16248)
updated
a
collection
2 months ago
liked
a
Space
3 months ago
updated
a
collection
4 months ago
upvoted
2
articles
4 months ago
Article
Vision Language Models Explained
By
and 1 other
•
•
377
Article
Assisted Generation: a new direction toward low-latency text generation
By
•
•
64
commented on
Open-R1: Update #1
4 months ago
Fixed, thanks!
The link in the below statement also seems to be broken:
"You can find the instructions to run these evaluations in the open-r1 repository."
upvoted
2
articles
4 months ago
Article
Open-R1: Update #1
By
and 7 others
•
•
305
Article
Open-R1: a fully open reproduction of DeepSeek-R1
By
and 2 others
•
•
867