File size: 193 Bytes
952d35f |
1 2 3 4 5 6 7 8 9 10 11 12 13 |
type: llamathree
context_length: 2048
emb_dim: 256
n_heads: 4
n_layers: 6
hidden_dim: 128
hidden_activation: silu
n_kv_groups: 1
rope_base: 50000
rope_freq: null
dtype: float32
vocab_size: 269
|