How to use facebook/rag-token-nq with Transformers:
# Load model directly from transformers import AutoTokenizer, RagTokenForGeneration tokenizer = AutoTokenizer.from_pretrained("facebook/rag-token-nq") model = RagTokenForGeneration.from_pretrained("facebook/rag-token-nq", device_map="auto")
https://huggingface.co/facebook/rag-token-nq/resolve/af32fa164f774a532dfb63c94b2e898e80434643/generator_tokenizer/tokenizer_config.json
hf download hf://facebook/rag-token-nq@af32fa164f774a532dfb63c94b2e898e80434643/generator_tokenizer/tokenizer_config.json
curl -L -o tokenizer_config.json https://huggingface.co/facebook/rag-token-nq/resolve/af32fa164f774a532dfb63c94b2e898e80434643/generator_tokenizer/tokenizer_config.json
