Update README.md
Browse files
README.md
CHANGED
@@ -62,7 +62,7 @@ If you want to use the lora model from guanaco-7b-leh-v2-adapter/ , remember to
|
|
62 |
|
63 |
## Training Setup
|
64 |
* 2x3090 with model parallel
|
65 |
-
* batch size = bsz
|
66 |
* ctx cut off length = 1024
|
67 |
* only train on output (with loss mask)
|
68 |
* enable group of len
|
|
|
62 |
|
63 |
## Training Setup
|
64 |
* 2x3090 with model parallel
|
65 |
+
* batch size = bsz 8 * grad acc 16 = 128
|
66 |
* ctx cut off length = 1024
|
67 |
* only train on output (with loss mask)
|
68 |
* enable group of len
|