What are the minimal requirements to run this model independently in an on-prem mode (not in the cloud) ?
#1
by
Yeay
- opened
Hi Team!
What are the minimal requirements to run this model independently in an on-prem mode (not in the cloud) ?
Depends mostly on the task and needed context left on the GPU. Can you tell us a bit more about it? Will you use the whole output tokens available or only some of them?