add doc for gpt2 embellish model
Browse files
README.md
CHANGED
@@ -32,8 +32,12 @@ Generates **melody and chord progression** from scratch.
|
|
32 |
- Training sequence length: 2400
|
33 |
### Stage 2: "Embellish" model
|
34 |
Generates **accompaniment, timing and dynamics** conditioned on Stage 1 outputs.
|
35 |
-
-
|
36 |
-
|
|
|
|
|
|
|
|
|
37 |
- Token vocabulary: [Revamped MIDI-derived events](https://arxiv.org/abs/2002.00212) (**REMI**) w/ slight modifications
|
38 |
- Training dataset: [AILabs.tw Pop1K7](https://github.com/YatingMusic/compound-word-transformer) (**Pop1K7**), 1747 songs
|
39 |
- Training sequence length: 3072
|
|
|
32 |
- Training sequence length: 2400
|
33 |
### Stage 2: "Embellish" model
|
34 |
Generates **accompaniment, timing and dynamics** conditioned on Stage 1 outputs.
|
35 |
+
- `embellish_model_gpt2_pop1k7_loss0.398.bin`
|
36 |
+
- Model backbone: 12-layer **GPT-2 Transformer** ([implementation](https://huggingface.co/docs/transformers/en/model_doc/gpt2))
|
37 |
+
- Num trainable params: 38.2M
|
38 |
+
- `embellish_model_pop1k7_loss0.399.bin` (requires `fast-transformers` package, which is outdated as of Jul. 2024)
|
39 |
+
- Model backbone: 12-layer **Performer** ([paper](https://arxiv.org/abs/2009.14794), [implementation](https://github.com/idiap/fast-transformers))
|
40 |
+
- Num trainable params: 38.2M
|
41 |
- Token vocabulary: [Revamped MIDI-derived events](https://arxiv.org/abs/2002.00212) (**REMI**) w/ slight modifications
|
42 |
- Training dataset: [AILabs.tw Pop1K7](https://github.com/YatingMusic/compound-word-transformer) (**Pop1K7**), 1747 songs
|
43 |
- Training sequence length: 3072
|