Qwen

/

Qwen2.5-Math-PRM-7B

like 36

Follow

Qwen 9.85k

tokenizer.decode([34139]) =  ']<'
tokenizer.decode([27]) =  '<'

is bug

suggest use

 "\n<extra_0>"

or

".<extra_0>"

Which tokenizer are you using? In the tokenizer config of this file, '<extra_0>' is a special token.

"151651": {
      "content": "<extra_0>",
      "lstrip": false,
      "normalized": false,
      "rstrip": false,
      "single_word": false,
      "special": true
    }

yeah, my mistake

suggest use

 "\n<extra_0>"

or
``