This is the microsoft/Phi-3.5-vision-instruct model, converted to OpenVINO, with fp16 weights.

Use OpenVINO GenAI to run inference on this model:

  • Install OpenVINO GenAI nightly and pillow:
pip install --upgrade --pre pillow openvino-genai openvino openvino-tokenizers --extra-index-url https://storage.openvinotoolkit.org/simple/wheels/nightly
  • Download a test image: curl -O "https://storage.openvinotoolkit.org/test_data/images/dog.jpg"
  • Run inference:
import numpy as np
import openvino as ov
import openvino_genai
from PIL import Image

# Choose GPU instead of CPU in the line below to run the model on Intel integrated or discrete GPU
pipe = openvino_genai.VLMPipeline("Phi-3.5-vision-instruct-ov-fp16", "CPU")
pipe.start_chat()

# Setting eos_token_id to tokenizer's eos token id is necessary for Phi-3.5-vision-instruct
config = openvino_genai.GenerationConfig()
config.set_eos_token_id(pipe.get_tokenizer().get_eos_token_id())
config.max_new_tokens = 100

image = Image.open("dog.jpg")
image_data = np.array(image.getdata()).reshape(1, image.size[1], image.size[0], 3).astype(np.uint8)
image_data = ov.Tensor(image_data)  

prompt = "Can you describe the image?"
result = pipe.generate(prompt, image=image_data, generation_config=config)
print(result.texts[0])

See OpenVINO GenAI repository

Downloads last month
12
Inference Providers NEW
This model is not currently available via any of the supported Inference Providers.
The model cannot be deployed to the HF Inference API: The model has no library tag.

Model tree for helenai/Phi-3.5-vision-instruct-ov-fp16

Finetuned
(16)
this model