Update README.md

README.md

@@ -11,15 +11,49 @@ General-purpose Swich transformer based Japanese language model
 ## Text Generation
 
 ```python
->>> from transformers import AutoModel, AutoTokenizer
+>>> from transformers import AutoModel, AutoTokenizer, trainer_utils
+>>> 
+>>> device = "cuda"
+>>> model = AutoModel.from_pretrained("Tanrei/GPTSAN-japanese").to(device)
+>>> tokenizer = AutoTokenizer.from_pretrained("Tanrei/GPTSAN-japanese")
+>>> x_token = tokenizer.encode("織田信長は、", return_tensors="pt").to(device)
+>>> trainer_utils.set_seed(30)
+>>> gen_token = model.generate(x_token, max_new_tokens=50)
+>>> tokenizer.decode(gen_token[0])
+"織田信長は、政治・軍事の中枢まで掌握した政治家であり、日本史上類を見ない驚異的な軍事侵攻を続け..."
+```
+
+
 
->>> model = AutoModel.from_pretrained("Tanrei/GPTSAN-japanese")
+## Text Generation with Prefix-LM model
+
+```python
+>>> from transformers import AutoModel, AutoTokenizer, trainer_utils
+>>> 
+>>> device = "cuda"
+>>> model = AutoModel.from_pretrained("Tanrei/GPTSAN-japanese").to(device)
+>>> tokenizer = AutoTokenizer.from_pretrained("Tanrei/GPTSAN-japanese")
+>>> x_token = tokenizer.encode("", prefix_text="織田信長は、", return_tensors="pt").to(device)
+>>> trainer_utils.set_seed(30)
+>>> gen_token = model.generate(x_token, max_new_tokens=50)
+>>> tokenizer.decode(gen_token[0])
+"織田信長は、政治・外交で数々の戦果を上げるが、1568年からは、いわゆる本能寺の変で細川晴元に暗殺される..."
+```
+
+
+
+## Masked Language Model
+
+```python
+>>> from transformers import AutoModel, AutoTokenizer, trainer_utils
+>>> 
+>>> device = "cuda"
+>>> model = AutoModel.from_pretrained("Tanrei/GPTSAN-japanese").to(device)
 >>> tokenizer = AutoTokenizer.from_pretrained("Tanrei/GPTSAN-japanese")
->>> x_tok = tokenizer.encode("武田信玄は、", return_tensors="pt")
->>> model = model.cuda()
->>> c = model.generate(x_tok.cuda(), max_new_tokens=50, random_seed=63)
->>> tokenizer.decode(c[0])
-'武田信玄は、戦国の頃より「智勇兼備」した英雄として織田信長に比されてきた戦国武将であり、...'
+>>> x_token = tokenizer.encode("", prefix_text="武田信玄は、<|inputmask|>時代ファンならぜひ押さえ<|inputmask|>きたい名将の一人。", return_tensors="pt").to(device)
+>>> out_token = model(x_token)
+>>> tokenizer.decode(out_token[0].argmax(axis=-1)[0])
+"武田信玄は、戦国時代ファンならぜひ押さえておきたい名将の一人。"
 ```