|
|
--- |
|
|
language: en |
|
|
tags: |
|
|
- shakespeare |
|
|
- gpt2 |
|
|
- text-generation |
|
|
- english |
|
|
license: mit |
|
|
datasets: |
|
|
- shakespeare |
|
|
--- |
|
|
|
|
|
# Shakespeare GPT-2 |
|
|
|
|
|
A GPT-2 model fine-tuned on Shakespeare's complete works to generate Shakespeare-style text. |
|
|
|
|
|
## Model Description |
|
|
|
|
|
This model is a fine-tuned version of GPT-2 (124M parameters) trained on Shakespeare's complete works. It can generate text in Shakespeare's distinctive style, including dialogue, soliloquies, and dramatic prose. |
|
|
|
|
|
### Model Architecture |
|
|
|
|
|
- Base Model: GPT-2 (124M parameters) |
|
|
- Layers: 12 |
|
|
- Heads: 12 |
|
|
- Embedding Dimension: 768 |
|
|
- Context Length: 1024 tokens |
|
|
- Total Parameters: ~124M |
|
|
|
|
|
### Training Details |
|
|
|
|
|
- Dataset: Complete works of Shakespeare |
|
|
- Training Steps: 100,000 |
|
|
- Batch Size: 4 |
|
|
- Sequence Length: 32 |
|
|
- Learning Rate: 3e-4 |
|
|
- Optimizer: AdamW |
|
|
- Device: MPS/CUDA/CPU |
|
|
|
|
|
## Intended Use |
|
|
|
|
|
This model is intended for: |
|
|
- Generating Shakespeare-style text |
|
|
- Creative writing assistance |
|
|
- Educational purposes in literature |
|
|
- Entertainment and artistic projects |
|
|
|
|
|
## Limitations |
|
|
|
|
|
- May generate text that mimics but doesn't perfectly replicate Shakespeare's style |
|
|
- Limited by training data to Shakespeare's vocabulary and themes |
|
|
- Can produce anachronistic or inconsistent content |
|
|
- Maximum context length of 1024 tokens |
|
|
|
|
|
## Training Data |
|
|
|
|
|
The model was trained on Shakespeare's complete works, including: |
|
|
- All plays (comedies, tragedies, histories) |
|
|
- Sonnets and poems |
|
|
- Total training tokens: [Insert number of tokens] |
|
|
|
|
|
## Performance |
|
|
|
|
|
The model achieves: |
|
|
- Training Loss: [Insert final training loss] |
|
|
- Best Loss: [Insert best loss achieved] |
|
|
|
|
|
## Example Usage |
|
|
python |
|
|
from transformers import GPT2LMHeadModel, GPT2Tokenizer |
|
|
Load model and tokenizer |
|
|
model_name = "your-username/shakespeare-gpt" |
|
|
tokenizer = GPT2Tokenizer.from_pretrained(model_name) |
|
|
model = GPT2LMHeadModel.from_pretrained(model_name) |
|
|
Generate text |
|
|
prompt = "To be, or not to be," |
|
|
input_ids = tokenizer.encode(prompt, return_tensors='pt') |
|
|
output = model.generate( |
|
|
input_ids, |
|
|
max_length=500, |
|
|
temperature=0.8, |
|
|
top_k=40, |
|
|
do_sample=True |
|
|
) |
|
|
generated_text = tokenizer.decode(output[0], skip_special_tokens=True) |
|
|
print(generated_text) |
|
|
|
|
|
## Sample Outputs |
|
|
Prompt: "To be, or not to be," |
|
|
Output: [Insert sample generation] |
|
|
Prompt: "Friends, Romans, countrymen," |
|
|
Output: [Insert sample generation] |
|
|
|
