---
# This modelcard aims to be a base template for new models. It has been generated using [this raw template](https://github.com/huggingface/huggingface_hub/blob/main/src/huggingface_hub/templates/modelcard_template.md?plain=1).
license: llama3
language:
- en
base_model:
- m3rg-iitd/llamat-3
tags:
- material science
- large language model
- domain adaptation
- scientific domain adaptation
- crystal generation
- materials copilot
- information extraction
- table understanding
- table data parsing
---

# Model Card for llamat-3-chat

<!-- Provide a quick summary of what the model is/does. -->
LLaMat-3-chat is a materials research copilot.
  
## Model Details
 foundational model that is finetuned from LLaMat-3, which is made by continued pretraining of LLaMA-3 on material science tokens. It has instruction following abilities and can be used as a copilot for information extraction from material science textual or tabular data.
### Model Description

<!-- Provide a longer summary of what this model is. -->

- **Developed by:** M3RG, IIT Delhi
- **Model type:** Large Language Model based on LLaMA-3 architecture
- **Language(s) (NLP):** English
- **License:** LLaMA-3
- **Finetuned from model [optional]:** m3rg-iitd/llamat-3

### Model Sources [optional]

<!-- Provide the basic links for the model. -->

- **Repository:** https://github.com/M3RG-IITD/llamat
<!-- - **Paper [optional]:** [More Information Needed] -->
<!-- - **Demo [optional]:** [More Information Needed] -->


### Compute Infrastructure
This work was supported by the Edinburgh International Data Facility (EIDF) and the Data-Driven Innovation Programme at the University of Edinburgh. The EIDF provided access to Cerebras CS2 clusters for pretraining the language models.
Link - https://edinburgh-international-data-facility.ed.ac.uk/services/computing/cerebras-cs

This work is also supported by High Performance Computing cluster and Yardi School of AI at IIT Delhi. 

#### Hardware
Pretraining: 2 CS-2 Cerebras Wafer-Scale Engine (WSE-2)
Finetuning:  8 NVIDIA-A100 80GB GPUs 
Inferencing: 1 NVIDIA-A100 80GB GPU
#### Software
PyTorch, HuggingFace, Transformers