File size: 2,307 Bytes
c827178 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 |
---
language: en
tags:
- shakespeare
- gpt2
- text-generation
- english
license: mit
datasets:
- shakespeare
---
# Shakespeare GPT-2
A GPT-2 model fine-tuned on Shakespeare's complete works to generate Shakespeare-style text.
## Model Description
This model is a fine-tuned version of GPT-2 (124M parameters) trained on Shakespeare's complete works. It can generate text in Shakespeare's distinctive style, including dialogue, soliloquies, and dramatic prose.
### Model Architecture
- Base Model: GPT-2 (124M parameters)
- Layers: 12
- Heads: 12
- Embedding Dimension: 768
- Context Length: 1024 tokens
- Total Parameters: ~124M
### Training Details
- Dataset: Complete works of Shakespeare
- Training Steps: 100,000
- Batch Size: 4
- Sequence Length: 32
- Learning Rate: 3e-4
- Optimizer: AdamW
- Device: MPS/CUDA/CPU
## Intended Use
This model is intended for:
- Generating Shakespeare-style text
- Creative writing assistance
- Educational purposes in literature
- Entertainment and artistic projects
## Limitations
- May generate text that mimics but doesn't perfectly replicate Shakespeare's style
- Limited by training data to Shakespeare's vocabulary and themes
- Can produce anachronistic or inconsistent content
- Maximum context length of 1024 tokens
## Training Data
The model was trained on Shakespeare's complete works, including:
- All plays (comedies, tragedies, histories)
- Sonnets and poems
- Total training tokens: [Insert number of tokens]
## Performance
The model achieves:
- Training Loss: [Insert final training loss]
- Best Loss: [Insert best loss achieved]
## Example Usage
python
from transformers import GPT2LMHeadModel, GPT2Tokenizer
Load model and tokenizer
model_name = "your-username/shakespeare-gpt"
tokenizer = GPT2Tokenizer.from_pretrained(model_name)
model = GPT2LMHeadModel.from_pretrained(model_name)
Generate text
prompt = "To be, or not to be,"
input_ids = tokenizer.encode(prompt, return_tensors='pt')
output = model.generate(
input_ids,
max_length=500,
temperature=0.8,
top_k=40,
do_sample=True
)
generated_text = tokenizer.decode(output[0], skip_special_tokens=True)
print(generated_text)
## Sample Outputs
Prompt: "To be, or not to be,"
Output: [Insert sample generation]
Prompt: "Friends, Romans, countrymen,"
Output: [Insert sample generation]
|