slseanwu commited on
Commit
fc173db
1 Parent(s): 345846f

add doc for gpt2 embellish model

Browse files
Files changed (1) hide show
  1. README.md +6 -2
README.md CHANGED
@@ -32,8 +32,12 @@ Generates **melody and chord progression** from scratch.
32
  - Training sequence length: 2400
33
  ### Stage 2: "Embellish" model
34
  Generates **accompaniment, timing and dynamics** conditioned on Stage 1 outputs.
35
- - Model backbone: 12-layer **Performer** ([paper](https://arxiv.org/abs/2009.14794), [implementation](https://github.com/idiap/fast-transformers))
36
- - Num trainable params: 38.2M
 
 
 
 
37
  - Token vocabulary: [Revamped MIDI-derived events](https://arxiv.org/abs/2002.00212) (**REMI**) w/ slight modifications
38
  - Training dataset: [AILabs.tw Pop1K7](https://github.com/YatingMusic/compound-word-transformer) (**Pop1K7**), 1747 songs
39
  - Training sequence length: 3072
 
32
  - Training sequence length: 2400
33
  ### Stage 2: "Embellish" model
34
  Generates **accompaniment, timing and dynamics** conditioned on Stage 1 outputs.
35
+ - `embellish_model_gpt2_pop1k7_loss0.398.bin`
36
+ - Model backbone: 12-layer **GPT-2 Transformer** ([implementation](https://huggingface.co/docs/transformers/en/model_doc/gpt2))
37
+ - Num trainable params: 38.2M
38
+ - `embellish_model_pop1k7_loss0.399.bin` (requires `fast-transformers` package, which is outdated as of Jul. 2024)
39
+ - Model backbone: 12-layer **Performer** ([paper](https://arxiv.org/abs/2009.14794), [implementation](https://github.com/idiap/fast-transformers))
40
+ - Num trainable params: 38.2M
41
  - Token vocabulary: [Revamped MIDI-derived events](https://arxiv.org/abs/2002.00212) (**REMI**) w/ slight modifications
42
  - Training dataset: [AILabs.tw Pop1K7](https://github.com/YatingMusic/compound-word-transformer) (**Pop1K7**), 1747 songs
43
  - Training sequence length: 3072