yahavb
/

inf2-bs32-tp16-mml16k-llama-31-8b-vllm

Model card Files Files and versions Community

inf2-bs32-tp16-mml16k-llama-31-8b-vllm

Ctrl+K

Ctrl+K

1 contributor

History: 2 commits

yahavb's picture

Add NxD compiled model yahavb/inf2-bs32-tp16-mml16k-llama-31-8b-vllm for vLLM; after converting checkpoints

afe0afe verified about 9 hours ago

weights
Add NxD compiled model yahavb/inf2-bs32-tp16-mml16k-llama-31-8b-vllm for vLLM; after converting checkpoints about 9 hours ago
.gitattributes

1.52 kB

initial commit about 9 hours ago
model.pt
Detected Pickle imports (16)
- "__torch__.neuronx_distributed.trace.spmd.NxDModel",
- "__torch__.neuronx_distributed.trace.spmd.NxDModelExecutor",
- "torch.jit._pickle.restore_type_tag",
- "__torch__.torch.classes.neuron.LayoutTransformation",
- "torch.jit._pickle.build_intlist",
- "torch.jit._pickle.build_intlist",
- "torch.jit._pickle.build_intlist",
- "torch.jit._pickle.build_intlist",
- "torch.jit._pickle.build_intlist",
- "torch.jit._pickle.build_intlist",
- "torch.jit._pickle.build_intlist",
- "torch._utils._rebuild_tensor_v2",
- "torch.IntStorage",
- "torch.jit._pickle.restore_type_tag",
- "collections.OrderedDict",
- "torch.FloatStorage"
How to fix it?
63.7 MB
LFS

Add NxD compiled model yahavb/inf2-bs32-tp16-mml16k-llama-31-8b-vllm for vLLM; after converting checkpoints about 9 hours ago
neuron_config.json

5.24 kB

Add NxD compiled model yahavb/inf2-bs32-tp16-mml16k-llama-31-8b-vllm for vLLM; after converting checkpoints about 9 hours ago