Authentication and Authorization

To invoke any API’s mentioned in this specification a client MUST first acquire a token (and the URL) to authenticate against the server which serves these API’s.

The Xet protocol server uses bearer authentication via a token generated by the Hugging Face Hub (https://huggingface.co).

The following section explains how to acquire such a token.

Token Request API Endpoints

URL Pattern:

https://huggingface.co/api/{repo_type}s/{repo_id}/xet-{token_type}-token/{revision}

Parameters:

All parameters are required to form the URL.

• repo_type: Type of repository - model, dataset, or space
• repo_id: Repository identifier in format namespace/repo-name
• token_type: Either read or write.
• revision: Git revision (branch, tag, or commit hash; default to using main if no specific ref is required)

To understand the distinction for between token_type values read onwards in this document to Token Scope.

Example URLs:

https://huggingface.co/api/models/sentence-transformers/all-MiniLM-L6-v2/xet-read-token/main
https://huggingface.co/api/datasets/HuggingFaceM4/the_cauldron/xet-write-token/v1.1
https://huggingface.co/api/spaces/jsulz/ready-xet-go/xet-read-token/main

HTTP Method: GET

Required Headers:

• Authorization: Bearer token for Hugging Face Hub authentication

Response Format

A JSON encoded object with the following format:

{
   "accessToken": string,
   "exp": number,
   "casUrl": string,
}

• accessToken is the token to be used when invoking API’s on the Xet CAS service (any Xet API denoted in this specification)
• exp is the unix timestamp of when this token expires
• casUrl is the API service endpoint URL

Users MAY assume the “accessToken” and “casUrl” fields lengths have an upper limit of 64000 characters.

Example Response Object

{
   "accessToken": "xet_xxxxxxxxxxx",
   "exp": 1848535668,
   "casUrl": "https://cas-server.xethub.hf.co"
}

Error Handling

HTTP Errors

• 401 Unauthorized: Invalid or missing Hub authentication token
• 403 Forbidden: Insufficient permissions for the requested token type
• 404 Not Found: Repository or revision does not exist

Implementation Example

Here’s a basic implementation flow:

1. Make the request:

   Copied

   GET /api/models/black-forest-labs/FLUX.1-dev/xet-read-token/main
   Host: huggingface.co
   Authorization: Bearer xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

2. Parse the response

   Copied

   endpoint = response_json["casUrl"]
   access_token = response_json["accessToken"]
   expiration = response_json["exp"]

3. Use the token with Xet service: Use Bearer authentication with the value for the accessToken key to authenticate with the Xet service at endpoint until expiration time.

4. Token refresh (when needed): Use the same API to generate a new token.

   In xet-core we SHOULD add 30 seconds of buffer time before the provided expiration time to refresh the token.

Token Scope

Xet tokens can have either a read or a write scope. write scope supersedes read scope and all read scope API’s can be invoked when using a write scope token. The type of token issued is determined on the token_type URI path component when requesting the token from the Hugging Face Hub (see above).

Check API specification for what scope level is necessary to invoke each API (briefly, only POST /shard and POST /xorb/* API’s require write scope).

The scope of the Xet tokens is limited to the repository and ref for which they were issued. To upload or download from different repositories or refs (different branches) clients MUST be issued different tokens.

Token Scope Relative to Hugging Face Hub Authentication Token

When requesting a Xet token from the Hugging Face Hub, you will only receive a Xet token matching the requested parameters if you actually have access to them, based on the access afforded to your Hub authentication token.

If you require a write scope Xet token, then you MUST request it using a Hugging Face Hub token that has write access to the particular repository and ref that you want to access.

If you request a read scope Xet token, then you MUST request it using a Hugging Face Hub token that has at least read access to the particular repository and ref you want to access.

If you are using Fine-grained Hugging Face Hub Access Tokens, your tokens MUST have read or write access to the contents of repositories to be issues read or write Xet tokens respectively.

Security Considerations

• Xet tokens are time-limited and SHOULD be refreshed/swapped before expiration
• Store tokens securely and SHOULD NOT log them (both Hub authentication tokens and Xet tokens)
• SHOULD use read tokens when possible; only request write tokens when necessary

Diagram