How to use facebook/rag-token-nq with Transformers:
# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, RagTokenForGeneration tokenizer = AutoTokenizer.from_pretrained("facebook/rag-token-nq") model = RagTokenForGeneration.from_pretrained("facebook/rag-token-nq", device_map="auto")
https://huggingface.co/facebook/rag-token-nq/resolve/3cd8133e26e4656e45b5393398b229fa12b95c56/generator_tokenizer/vocab.json
hf download hf://facebook/rag-token-nq@3cd8133e26e4656e45b5393398b229fa12b95c56/generator_tokenizer/vocab.json
curl -L -o vocab.json https://huggingface.co/facebook/rag-token-nq/resolve/3cd8133e26e4656e45b5393398b229fa12b95c56/generator_tokenizer/vocab.json
