aayushraina commited on
Commit
c827178
·
verified ·
1 Parent(s): 3c9a4c3

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +92 -3
README.md CHANGED
@@ -1,3 +1,92 @@
1
- ---
2
- license: mit
3
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language: en
3
+ tags:
4
+ - shakespeare
5
+ - gpt2
6
+ - text-generation
7
+ - english
8
+ license: mit
9
+ datasets:
10
+ - shakespeare
11
+ ---
12
+
13
+ # Shakespeare GPT-2
14
+
15
+ A GPT-2 model fine-tuned on Shakespeare's complete works to generate Shakespeare-style text.
16
+
17
+ ## Model Description
18
+
19
+ This model is a fine-tuned version of GPT-2 (124M parameters) trained on Shakespeare's complete works. It can generate text in Shakespeare's distinctive style, including dialogue, soliloquies, and dramatic prose.
20
+
21
+ ### Model Architecture
22
+
23
+ - Base Model: GPT-2 (124M parameters)
24
+ - Layers: 12
25
+ - Heads: 12
26
+ - Embedding Dimension: 768
27
+ - Context Length: 1024 tokens
28
+ - Total Parameters: ~124M
29
+
30
+ ### Training Details
31
+
32
+ - Dataset: Complete works of Shakespeare
33
+ - Training Steps: 100,000
34
+ - Batch Size: 4
35
+ - Sequence Length: 32
36
+ - Learning Rate: 3e-4
37
+ - Optimizer: AdamW
38
+ - Device: MPS/CUDA/CPU
39
+
40
+ ## Intended Use
41
+
42
+ This model is intended for:
43
+ - Generating Shakespeare-style text
44
+ - Creative writing assistance
45
+ - Educational purposes in literature
46
+ - Entertainment and artistic projects
47
+
48
+ ## Limitations
49
+
50
+ - May generate text that mimics but doesn't perfectly replicate Shakespeare's style
51
+ - Limited by training data to Shakespeare's vocabulary and themes
52
+ - Can produce anachronistic or inconsistent content
53
+ - Maximum context length of 1024 tokens
54
+
55
+ ## Training Data
56
+
57
+ The model was trained on Shakespeare's complete works, including:
58
+ - All plays (comedies, tragedies, histories)
59
+ - Sonnets and poems
60
+ - Total training tokens: [Insert number of tokens]
61
+
62
+ ## Performance
63
+
64
+ The model achieves:
65
+ - Training Loss: [Insert final training loss]
66
+ - Best Loss: [Insert best loss achieved]
67
+
68
+ ## Example Usage
69
+ python
70
+ from transformers import GPT2LMHeadModel, GPT2Tokenizer
71
+ Load model and tokenizer
72
+ model_name = "your-username/shakespeare-gpt"
73
+ tokenizer = GPT2Tokenizer.from_pretrained(model_name)
74
+ model = GPT2LMHeadModel.from_pretrained(model_name)
75
+ Generate text
76
+ prompt = "To be, or not to be,"
77
+ input_ids = tokenizer.encode(prompt, return_tensors='pt')
78
+ output = model.generate(
79
+ input_ids,
80
+ max_length=500,
81
+ temperature=0.8,
82
+ top_k=40,
83
+ do_sample=True
84
+ )
85
+ generated_text = tokenizer.decode(output[0], skip_special_tokens=True)
86
+ print(generated_text)
87
+
88
+ ## Sample Outputs
89
+ Prompt: "To be, or not to be,"
90
+ Output: [Insert sample generation]
91
+ Prompt: "Friends, Romans, countrymen,"
92
+ Output: [Insert sample generation]