turboderp
/

pixtral-12b-exl2

turboderp commited on Nov 20, 2024

Commit

555970b

verified ·

1 Parent(s): d673e84

Fix max_position_embeddings

Files changed (1) hide show

config.json CHANGED Viewed

@@ -15,7 +15,7 @@
         "head_dim": 128,
         "intermediate_size": 14336,
         "is_composition": true,
-        "max_position_embeddings": 1024000,
         "model_type": "mistral",
         "num_hidden_layers": 40,
         "num_attention_heads": 32,

         "head_dim": 128,
         "intermediate_size": 14336,
         "is_composition": true,
+        "max_position_embeddings": 131072,
         "model_type": "mistral",
         "num_hidden_layers": 40,
         "num_attention_heads": 32,