|
--- |
|
language: |
|
- en |
|
base_model: |
|
- meta-llama/Llama-3.1-70B-Instruct |
|
tags: |
|
- finance |
|
- Llama3.1 |
|
--- |
|
# Llama-3.1-Omni-FinAI-70B Model Card |
|
|
|
## Model Overview (Built with Llama) |
|
Llama-3.1-Omni-FinAI-70B is a pre-trained large language model optimized for finance-specific fine-tuning applications. Based on the LLaMA 3.1 70B architecture, this model was pre-trained on 143 billion tokens of high-quality financial texts. Llama-3.1-Omni-FinAI-70B provides a foundation for further fine-tuning in specialized financial analysis tasks. |
|
|
|
## Model Details |
|
- **Base Model**: Llama-3.1-70B-Instruct |
|
- **Training Data**: |
|
- SEC 10-K, 10-Q, and 8-K filings |
|
- Reuters News data (RCV1, TRC2) |
|
- Finance-specific papers from Arxiv |
|
- Financial discussions from Reddit |
|
- Wikipedia |
|
- **Primary Use Case**: Pre-training for finance-specific fine-tuning, allowing users to leverage Llama-3.1-Omni-FinAI-70B's foundational financial language understanding. |
|
|
|
## Use Cases |
|
Llama-3.1-Omni-FinAI-70B is designed as a base model for finance-specific fine-tuning tasks, supporting applications such as: |
|
- Sentiment Analysis |
|
- Stock Movement Prediction |
|
- QA Instruction |
|
- Summarization |
|
- Predictive Financial Analysis |
|
|
|
## Training Process |
|
Llama-3.1-Omni-FinAI-70B was trained using the NVIDIA NeMo framework on 64 H100 GPUs, utilizing a diverse dataset that ensures robust performance for fine-tuning in finance-related applications. |
|
|
|
## Limitations |
|
This model is pre-trained for finance-specific fine-tuning tasks and may require additional fine-tuning for specialized applications. Due to its large size, substantial computational resources are recommended for deployment. |
|
|
|
## License |
|
This model is licensed under the Llama 3.1 Community License. |
|
|
|
## Citation |
|
If you use the Llama-3.1-Omni-FinAI-70B model, please cite as follows: |
|
> Chiu, I-Chan and Hung, Mao-Wei and Chen, Zih-Ching and Chiu, Jun-wei and Lin, Yang-Hsien and Lee, Cheng-Kuang and Huang, Eddie TC and See, Simon, Omni-FinAI: Unlocking Financial Disclosure Insights (October 30, 2024). Available at SSRN: https://ssrn.com/abstract=5004298 |
|
|
