SeanScripts/pixtral-12b-nf4 · working python packages

Oct 2

I got this working with the following python packages:
accelerate==0.34.2
bitsandbytes==0.44.1
certifi==2024.8.30
charset-normalizer==3.3.2
colorama==0.4.6
filelock==3.16.1
fsspec==2024.9.0
huggingface-hub==0.25.1
idna==3.10
Jinja2==3.1.3
MarkupSafe==2.1.5
mpmath==1.3.0
networkx==3.2.1
numpy==2.1.1
packaging==24.1
pillow==10.2.0
psutil==6.0.0
PyYAML==6.0.2
regex==2024.9.11
requests==2.32.3
safetensors==0.4.5
sympy==1.12
tokenizers==0.20.0
torch==2.4.1+cu121
torchvision==0.19.1+cu121
tqdm==4.66.5
transformers==4.45.1
typing_extensions==4.12.2
urllib3==2.2.3

fusi0n

Oct 7

•

edited Oct 7

going from 4.40 to 4.45.1 with transformers fixed my issues.

5t/s on a 4060ti 16gb.

info-int

Oct 8

@fusi0n same card here, how did you find the outputs? I found the inference to be totally hit and miss though. I think a bigger parameter model is needed.

fusi0n

29 days ago

@info-int output was not great, i perfer moondream2 for my use case, but it sucks losing the true multimodel aspect