Add a goddamn max length parameter to your tokenizer. (#4)

- Add a goddamn max length parameter to your tokenizer. (e0c812c9087e51e818b0e38041acb1319e7abd6d)
- Update tokenizer_config.json (d7157fd86585afa6695d24daac23c39d5cf65a98)


Co-authored-by: Michael <[email protected]>

tokenizer_config.json

@@ -46,7 +46,7 @@
   "cls_token": "<s>",
   "eos_token": "</s>",
   "mask_token": "<mask>",
-  "model_max_length": 1000000000000000019884624838656,
+  "model_max_length": 512,
   "pad_token": "<pad>",
   "sep_token": "</s>",
   "sp_model_kwargs": {},