based
Collection
These language model checkpoints are trained at the 360M and 1.3Bn parameter scales for up to 50Bn tokens on the Pile corpus, for research purposes.
•
15 items
•
Updated
•
9
Configuration Parsing
Warning:
In config.json: "architectures" must be an array
Based model but uses layernorm instead of QK.sum(-1) for the normalization, for better hardware efficiency.
- Downloads last month
- 16
Inference Providers
NEW
This model is not currently available via any of the supported Inference Providers.
The model cannot be deployed to the HF Inference API:
The model has no library tag.