qaihm-bot commited on
Commit
95b371d
·
verified ·
1 Parent(s): f76c27f

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +5 -2
README.md CHANGED
@@ -44,8 +44,11 @@ This model is an implementation of Llama-v3.1-8B-Chat found [here](https://githu
44
  - TTFT: Time To First Token is the time it takes to generate the first response token. This is expressed as a range because it varies based on the length of the prompt. The lower bound is for a short prompt (up to 128 tokens, i.e., one iteration of the prompt processor) and the upper bound is for a prompt using the full context length (4096 tokens).
45
  - Response Rate: Rate of response generation after the first response token.
46
 
47
- | Model | Precision | Device | Chipset | Target Runtime | Inference Time (ms) | Peak Memory Range (MB) | Primary Compute Unit | Target Model
48
- |---|---|---|---|---|---|---|---|---|
 
 
 
49
 
50
  ## Deploying Llama 3.1 on-device
51
 
 
44
  - TTFT: Time To First Token is the time it takes to generate the first response token. This is expressed as a range because it varies based on the length of the prompt. The lower bound is for a short prompt (up to 128 tokens, i.e., one iteration of the prompt processor) and the upper bound is for a prompt using the full context length (4096 tokens).
45
  - Response Rate: Rate of response generation after the first response token.
46
 
47
+ | Model | Precision | Device | Chipset | Target Runtime | Response Rate (tokens per second) | Time To First Token (range, seconds)
48
+ |---|---|---|---|---|---|
49
+ | Llama-v3.1-8B-Chat | w4a16 | Snapdragon 8 Elite QRD | Snapdragon® 8 Elite Mobile | QNN | 13.0546 | 0.154517 - 4.944544 | -- | -- |
50
+ | Llama-v3.1-8B-Chat | w4a16 | Snapdragon X Elite CRD | Snapdragon® X Elite | QNN | 9.357 | 0.207727 - 6.647264 | -- | -- |
51
+ | Llama-v3.1-8B-Chat | w4a16 | SA8255P ADP | Qualcomm® SA8255P | QNN | 7.93185 | 0.3297069 - 10.5506208 | -- | -- |
52
 
53
  ## Deploying Llama 3.1 on-device
54