deepseek-ai
/

DeepSeek-V3

Text Generation

Model card Files Files and versions Community

kylesayrs commited on Feb 24

Commit

db2fa7b

·

1 Parent(s): 8651896

fix cache seq length call

Signed-off-by: Kyle Sayers <[email protected]>

Files changed (1) hide show

modeling_deepseek.py +1 -1

modeling_deepseek.py CHANGED Viewed

@@ -1652,7 +1652,7 @@ class DeepseekV3ForCausalLM(DeepseekV3PreTrainedModel):
             if isinstance(past_key_values, Cache):
                 cache_length = past_key_values.get_seq_length()
                 past_length = past_key_values.seen_tokens
-                max_cache_length = past_key_values.get_max_length()
             else:
                 cache_length = past_length = past_key_values[0][0].shape[2]
                 max_cache_length = None

             if isinstance(past_key_values, Cache):
                 cache_length = past_key_values.get_seq_length()
                 past_length = past_key_values.seen_tokens
+                max_cache_length = past_key_values.get_max_cache_shape()
             else:
                 cache_length = past_length = past_key_values[0][0].shape[2]
                 max_cache_length = None