Banerjee's picture

1 5

Banerjee

port8080

·

port8080

AI & ML interests

datasets

Recent Activity

reacted to jsulz's post with 🔥 6 days ago

At https://huggingface.co/xet-team we've been hard at work bringing a new generation of storage to the Hugging Face community, and we’ve crossed some major milestones: 👷 Over 2,000 builders and nearing 100 organizations with access to Xet 🚀 Over 70,000 model and dataset repositories are Xet-backed 🤯 1.4 petabytes managed by Xet As we move repos from LFS to Xet for everyone we onboard, we’re pushing our content-addressed store (CAS). Check out the chart below 👇 of CAS hitting up to 150 Gb/s throughput this past week. All of this growth is helping us build richer insights. We expanded our repo graph, which maps how Xet-backed repositories on the Hub share bytes with each other. Check out the current network in the image below (nodes are repositories, edges are where repos share bytes) and visit the space to see how different versions of Qwen, Llama, and Phi models are grouped together https://huggingface.co/spaces/xet-team/repo-graph Join the waitlist to get access! https://huggingface.co/join/xet

reacted to jsulz's post with 🔥 28 days ago

The Llama 4 release - https://huggingface.co/collections/meta-llama/llama-4-67f0c30d9fe03840bc9d0164 - was a big one for the https://huggingface.co/xet-team with every model backed by the storage infrastructure of the future for the Hub. It's been a wild few days, and especially 🤯 to see every tensor file with a Xet logo next to it instead of LFS. The attached graph shows requests per second to our content-addressed store (CAS) right as the release went live. yellow = GETs; dashed line = launch time. You can definitely tell when the community started downloading 👀 h/t to @rajatarya for the graph, the entire Xet crew to bring us to this point, and special shoutout to Rajat, @port8080, @brianronan , @seanses , and @znation who made sure the bytes kept flying all weekend ⚡️

reacted to jsulz's post with 🔥 about 1 month ago

Huge week for https://huggingface.co/xet-team as Llama 4 is the first major model on Hugging Face uploaded with Xet providing the backing! Every byte downloaded comes through our infrastructure. Using Xet on Hugging Face is the fastest way to download and iterate on open source models and we've proved it with Llama 4 giving a boost of ~25% across all models. We expect builders on the Hub to see even more improvements, helping power innovation across the community. With the models on our infrastructure, we can peer in and see how well our dedupe performs across the Llama 4 family. On average, we're seeing ~25% dedupe, providing huge savings to the community who iterate on these state-of-the-art models. The attached image shows a few selected models and how they perform on Xet. Thanks to the https://huggingface.co/meta-llama team for launching on Xet!

View all activity

Organizations

Articles 1

Article

46

Rearchitecting Hugging Face Uploads and Downloads

models 0

None public yet

datasets 1

port8080/test-xet-sagemaker

Viewer • Updated Jan 21 • 1 • 15