comaniac
/

Meta-Llama-3-70B-Instruct-FP8-v1

Text Generation

text-generation-inference

Inference Endpoints

Model card Files Files and versions Community

Meta-Llama-3-70B-Instruct-FP8-v1 / README.md

comaniac's picture

Create README.md

3e9d5ca verified 8 months ago

|

history blame contribute delete

241 Bytes

Llama-3-70B-Instruct-FP8-v1

Weights and activations are per-tensor quantized to float8_e4m3.
Quantization with AutoFP8.
Calibration dataset: Ultrachat (mgoin/ultrachat_2k)
Samples: 1024
Sequence length: 4096

Evaluation

TBA