aws-neuron
/

optimum-neuron-cache

dacorvo HF Staff commited on Sep 17

Commit

6266253

verified ·

1 Parent(s): 9726c79

Update inference-cache-config/qwen-moe.json

Files changed (1) hide show

inference-cache-config/qwen-moe.json CHANGED Viewed

@@ -15,7 +15,7 @@
     {
       "batch_size": 8,
       "sequence_length": 4096,
-      "num_cores": 8,
       "auto_cast_type": "bf16"
     }
   ]

     {
       "batch_size": 8,
       "sequence_length": 4096,
+      "num_cores": 16,
       "auto_cast_type": "bf16"
     }
   ]