NeelNanda
/

Attn-Only-2L512W-Shortformer-6B-big-lr

Model card Files Files and versions Community

NeelNanda commited on Oct 29, 2022

Commit

438f8e1

·

1 Parent(s): 41bb72c

Update config.json

Files changed (1) hide show

config.json +1 -1

config.json CHANGED Viewed

@@ -17,7 +17,7 @@
   "version": 188,
   "use_bfloat16_matmul": true,
   "n_ctx": 1024,
-  "d_vocab": 50278,
   "tokenizer_name": "EleutherAI/gpt-neox-20b",
   "betas": [
     0.9,

   "version": 188,
   "use_bfloat16_matmul": true,
   "n_ctx": 1024,
+  "d_vocab": 50277,
   "tokenizer_name": "EleutherAI/gpt-neox-20b",
   "betas": [
     0.9,