|
|
--- |
|
|
license: llama3.3 |
|
|
base_model: SicariusSicariiStuff/Negative_LLAMA_70B |
|
|
tags: |
|
|
- llama-3.3 |
|
|
- finetune |
|
|
- roleplay |
|
|
- chat |
|
|
- wings-of-fire |
|
|
datasets: |
|
|
- Darkhn/WOF_QA_V2 |
|
|
- Darkhn/WOF_Pretraining |
|
|
- Darkhn/WOF_V2_All_Good_Stories |
|
|
- Darkhn/WOF_V1_All_Incredible_Stories |
|
|
--- |
|
|
|
|
|
# Model Name - L3.3-70B-Animus-V1 |
|
|
|
|
|
<img src="special-edition-box-set-poster-v0-e7a67voh505f1.webp" alt="Wings_of_Fire" width="700"/> |
|
|
|
|
|
## Model Description |
|
|
|
|
|
This is a fine-tuned version of `SicariusSicariiStuff/Negative_LLAMA_70B` specialized for roleplaying and instruction-following within the *Wings of Fire* universe. |
|
|
|
|
|
The model was trained on a 3-million-token dataset extracted from the *Wings of Fire* book series to build a strong foundation of domain knowledge. It was fine-tuned for 2 epochs on a curated dataset of conversational examples. |
|
|
|
|
|
The goal of this model is to provide a high-quality, immersive, and lore-accurate conversational experience for fans of the series. It can adopt character personas, answer questions about the world, and engage in creative storytelling. |
|
|
|
|
|
## Training Details |
|
|
|
|
|
### Training Hardware |
|
|
The model was trained on a single NVIDIA H200 GPU. |
|
|
|
|
|
### Training Procedure |
|
|
A QLoRA (Quantized Low-Rank Adaptation) approach was used for efficient fine-tuning. |
|
|
|
|
|
### Training Data |
|
|
The training process involved two main stages: |
|
|
|
|
|
1. **Domain training:** The base model was adapted to the *Wings of Fire* universe using a custom dataset of **3 million tokens** compiled directly from the book series. This step saturated the model with the specific lore, characters, and writing style of the source material. |
|
|
|
|
|
2. **Instruction & Chat Fine-tuning:** The model was fine-tuned on a mixed dataset of **2,200 examples**: |
|
|
* **1,400 Roleplay Conversations:** Multi-turn conversational examples designed to teach the model how to adopt and maintain character personas from the series. |
|
|
* **800 Assistant Examples:** Instruction-response pairs focused on answering lore questions and following commands within the context of the *Wings of Fire* world. |
|
|
|
|
|
## Intended Use & Limitations |
|
|
|
|
|
* **Intended Use:** This model is intended for creative and roleplaying purposes within the *Wings of Fire* universe. It is designed for fans of the series and is not a general-purpose chatbot. |
|
|
|
|
|
* **Limitations & Quirks:** |
|
|
* Performance on tasks outside of its training domain (general knowledge, coding, etc.) is not guaranteed and will likely be poor. |
|
|
* The model may "hallucinate" or generate plausible but non-canonical information. |
|
|
* **Content:** The base model (`Negative_LLAMA_70B`) has a slight NSFW inclination. While this finetune was **not** trained for NSFW content, some of that behavior may bleed through. It is not the intended purpose of this model. |
|
|
* **Formatting:** The model has a strong tendency to use asterisks (`*...*`) for actions and emphasis. This is a learned behavior from its training data. |
|
|
* **Safety:** This model has not undergone additional safety alignment beyond what was included in its base. Standard responsible AI practices should be followed. |
|
|
|
|
|
## Acknowledgements |
|
|
* Credit to Meta for the powerful Llama 3 architecture. |
|
|
* Credit to `SicariusSicariiStuff` for the `Negative_LLAMA_70B` base model. |
