|
--- |
|
license: mit |
|
license_name: xt-aurora-license |
|
license_link: LICENSE |
|
language: |
|
- en |
|
- es |
|
tags: |
|
- conversational |
|
- chat |
|
- roleplay |
|
library_name: GGUF |
|
pipeline_tag: text-generation |
|
base_model: TinyLlama/TinyLlama-1.1B-intermediate-step-715k-1.5T |
|
datasets: |
|
- XeTute/Small-Medium-Conversation-Multilingual |
|
- XeTute/Conversational-Small |
|
--- |
|
|
|
 |
|
|
|
**Note**<br> |
|
With the release of Meta's LLaMA 3.2 1B, this model got outperformed significantly. Since we don't have a lot of GPU power or money to furter train this or another model to even come close to Meta's models, we recommend you to use theirs over ours. |
|
|
|
We, XeTute, introduce AURORA V1.0 - a humerous, efficient, smart(for its size) and mostly unbiased(consider it a virtual child with a bunch of knowledge =), biases were largely removed after training through some easy techniques) Language Model. |
|
|
|
**Intended usecases:** |
|
- Next-Word prediction for mobile devices: |
|
- - This Model can be reliably packaged into a keyboard-app to help make Next-Word suggestions more accurate (for performance, INT4 or less might be smart) |
|
- Conversations: |
|
- - AURORA can engage in conversations using the Vicuna format, remember to replace "ASSISTANT" with "AURORA" though. |
|
- - AURORA can engage in SFW roleplay with simple character definitions. It wasn't trained on NSFW. |
|
- - AURORA can engage in simple, short Q&A. It was trained on factual data too, which means it performs well for its size. |
|
|
|
**Training:** |
|
- Trained for two months. |
|
- Dataset created by XeTute, and translated using different free-lancing services. |
|
- Dataset included: |
|
- - Mathematic Q&A |
|
- - Logic Q&A |
|
- - One-Page stories and roleplays with very brief character definitions |
|
- ADAM as an optimizer. |
|
Alltogether, the model was trained on additional 20B tokens. |
|
|
|
<a href='https://ko-fi.com/C0C2ZXNON' target='_blank'><img height='36' style='border:0px;height:36px;' src='https://storage.ko-fi.com/cdn/kofi3.png?v=3' border='0' alt='Buy Me a Coffee at ko-fi.com' /></a> |
|
Note: |
|
- All previous beta versions of this series of SLMs were deleted, because almost no downloads were made. |
|
- V1.0 is the last model in this series which will be published, because of too little community activity. |
|
|
|
Recommended settings: |
|
- Temperature 0.1 - 0.4 is stable. |
|
- Context Length of 2048(base) to 4096(RoPE) will work well for story-telling, role-playing and simple conversations. |
|
- Output Length: 256 will work very stable, but you can extent to 512. Anything beyond that point is risky, text might become repetitous. |
|
- A system prompt which works well can be found at "Files at Versions" => "chat_template". Just copy and paste this into the system prompt or add it before your first message. |
|
- Chat Format: |
|
```For roleplay: |
|
{name of your roleplay}: {input} |
|
{name of AURORA's character}: {output} |
|
``` |
|
or, |
|
```For normal chatting: |
|
USER: {input} |
|
AURORA: {output} |
|
``` |
|
|
|
Chat examples using KoboldCPP and the settings recommended above: |
|
|
|
 |
|
|
|
 |
|
Note, a roleplay where you directly pass character definitions and a starting scenario will work way better, this is just an example. |
|
|
|
We wish you a friendly chat with AURORA. |
