Intel
/

gpt-j-6B-int8-dynamic-inc

Batch inference

by zqcawzs - opened Apr 18, 2023

Apr 18, 2023

Hi,

Could you give an example about how to do batch inference using this gpt-j-6B-int8-dynamic model?

Thanks!

Intel org Apr 19, 2023

Hi,

I have added batch inference code into the evaluation.ipynb, please check it!

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment