No package metadata was found for bitsandbytes
I try to run falcon-7b on local
pip install -q -U bitsandbytes
import torch
from transformers import BitsAndBytesConfig
quantization_config = BitsAndBytesConfig(
load_in_4bit=True,
bnb_4bit_compute_dtype=torch.float16,
bnb_4bit_quant_type="nf4",
bnb_4bit_use_double_quant=True,
)
Traceback (most recent call last):
File "...", line 4, in
quantization_config = BitsAndBytesConfig(
File "...\Programs\Python\Python310\lib\site-packages\transformers\utils\quantization_config.py", line 212, in init
self.post_init()
File "...\Programs\Python\Python310\lib\site-packages\transformers\utils\quantization_config.py", line 238, in post_init
if self.load_in_4bit and not version.parse(importlib.metadata.version("bitsandbytes")) >= version.parse(
File "...\Programs\Python\Python310\lib\importlib\metadata_init.py", line 984, in version
return distribution(distribution_name).version
File "...\Programs\Python\Python310\lib\importlib\metadata_init.py", line 957, in distribution
return Distribution.from_name(distribution_name)
File "...\Local\Programs\Python\Python310\lib\importlib\metadata_init_.py", line 548, in from_name
raise PackageNotFoundError(name)
importlib.metadata.PackageNotFoundError: No package metadata was found for bitsandbytes
It seems that bitsandbytes is not working on WIN10. Does anyone know how to fix this?
I've got very much the same problem with python 3.9.
transformers package is from git+https://github.com/huggingface/transformers and says version 4.37.0.dev as a version number.
what is funny that the import statement says it loads BitsAndBytes from transformers, but then there exists a package with the same name.
Does one have to install that package additionally?
I am getting similar error .How is this resolved
hey guys is there anyone who resolved the issue?
I'm also looking at the same issue.
to fix:pip install bitsandbytes
quick fix:
pip install bitsandbytes
Hello,
I tried by installing bitsandbytes usingpip install bitsandbytes but still getting the same error
ImportError Traceback (most recent call last)
in <cell line: 1>()
----> 1 model= load_quantized_model(model_name)
3 frames
/usr/local/lib/python3.10/dist-packages/transformers/quantizers/quantizer_bnb_4bit.py in validate_environment(self, *args, **kwargs)
64 raise ImportError("Using bitsandbytes 4-bit quantization requires Accelerate: pip install accelerate")
65 if not is_bitsandbytes_available():
---> 66 raise ImportError(
67 "Using bitsandbytes 4-bit quantization requires the latest version of bitsandbytes: pip install -U bitsandbytes"
68 )
ImportError: Using bitsandbytes 4-bit quantization requires the latest version of bitsandbytes: pip install -U bitsandbytes
What kernel? I restarted the linux kernel, and it didn't fix anything.
@heinzketchup i have tried in the colab and restarted the google colab kernel it has worked for me