Hugging Face's logo

google
/
paligemma-3b-pt-224

Image-Text-to-Text
Transformers
Safetensors
paligemma
text-generation-inference
Inference Endpoints
Model card Files Community
28

Avoiding RuntimeError when loading in 4-bit / 8-bit

#28
by mumeranwaar - opened
base: refs/heads/main
←
from: refs/pr/28
Discussion Files changed
+4
-0
mumeranwaar

Convert to HalfTensor as the model expects it

model_inputs["pixel_values"] = model_inputs["pixel_values"].half()

This avoid this error: RuntimeError: Input type (torch.cuda.FloatTensor) and weight type (torch.cuda.HalfTensor) should be the same

Avoiding RuntimeError when loading in 4-bit / 8-bitf1f76c1d
Ready to merge
This branch is ready to get merged automatically.

Sign up or log in to comment