File size: 2,561 Bytes
edd892f 2ef669e e16c2f3 2ef669e 12930a8 edd892f 8cb0727 edd892f 8cb0727 edd892f e16c2f3 8cb0727 e16c2f3 edd892f 3e65bc2 edd892f 5d7455e edd892f 518e5d5 edd892f e16c2f3 edd892f 8cb0727 3e65bc2 8cb0727 3e65bc2 8cb0727 3e65bc2 8cb0727 3e65bc2 8cb0727 3e65bc2 8cb0727 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 |
---
language:
- en
tags:
- FantasyGPT
widget:
- "Legolas and Gimli advanced on the orcs, raising their weapons with a harrowing war cry."
- "Hermione smiled at Harry."
- "Ghost bared his teeth."
- "Geralt drew his sword"
- "Galadriel drew her sword"
- "Daenerys kissed Gandalf, as the witcher hacked off Lord Voldemort's head with a brutal swing of Longclaw."
- "Harry leapt forward, dodging Fingolfin's wildfire and reaching for the ring."
---
GPT-2 (including tokenizer) trained **from scratch** on some of my favorite books (about 31M words in total).
It's only trained on an RTX 3090 for three hours, so don't take it seriously, just have fun!
- peak lr: 4e-4
- global batch size: 32
- weight decay: 0.01
- training steps: 25k
- warmup steps: 1k
- lr decay: cosine
Example usage:
```python
from transformers import AutoTokenizer, GPT2LMHeadModel
tokenizer = AutoTokenizer.from_pretrained('Geralt-Targaryen/FantasyGPT')
model = GPT2LMHeadModel.from_pretrained('Geralt-Targaryen/FantasyGPT')
input_text = ["Daenerys kissed Gandalf, as the witcher hacked off Lord Voldemort's head with a brutal swing of Longclaw."]
input_tokenized = tokenizer(input_text, return_tensors='pt')
output = model.generate(inputs=input_tokenized.input_ids, max_new_tokens=256, do_sample=True, top_p=0.9, temperature=1)
print(tokenizer.decode(output[0]))
```
Sample output:
> Daenerys kissed Gandalf, as the witcher hacked off Lord Voldemort's head with a brutal swing of Longclaw. “Do you know what a warrior like that will do?”
>
> “I am a knight of seven,” Geralt said. “And how is this knight?”
>
> “Prince, it is known,” replied the witcher, “I am a knight of Solamnia, not a Knight. A knight of Solamnia in the name of Reorx, with elven armies at the head of his knights and knights. You do not even remember my name, elf. It was a good call.”
>
> “That name,” said the witcher, “what does it mean, elf?”
>
> “Some story,” said the old knight. “A good story from the Cataclysm.”
>
> The witcher snorted and looked at the witcher. “That is how you feel, elf. You don't understand why the knights won't be executed for them.”
>
> “That's because the knight is a king,” Chireadan said finally, “the king's elder son. Because he is, the knights would also be honorable with the title of knighthood. But a warrior would be better suited to have the honor to win.”
>
> “But…”
>
> “It's true,” said the old knight, “that, isn't entirely what you want. The lance of your sword is one of the best
|