cerpengen-500words / added_tokens.json
diamantrsd's picture
Upload tokenizer
3d87f63
raw
history blame contribute delete
122 Bytes
{
"<CLS>": 50261,
"<CONTENT>": 50262,
"<pad>": 50259,
"<|BOS|>": 50258,
"<|EOS|>": 50257,
"<|body|>": 50260
}