lemonilia commited on
Commit
7fe7e61
1 Parent(s): cdb3ec3

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +7 -5
README.md CHANGED
@@ -91,13 +91,15 @@ your desired response length:
91
 
92
  ## Training procedure
93
  [Axolotl](https://github.com/OpenAccess-AI-Collective/axolotl) was used for training
94
- on a 4x NVidia A40 GPU cluster. The model has been trained as an 8-bit LoRA adapter, and
 
 
 
 
95
  it's so large because a LoRA rank of 256 was also used. The reasoning was that this
96
  might have helped the model internalize any newly acquired information, making the
97
- training process closer to a full finetune.
98
-
99
- It's suggested to merge the adapter to the base Llama2-7B model (or other Llama2-based
100
- models).
101
 
102
  ### Training hyperparameters
103
  For the first pass these settings were used:
 
91
 
92
  ## Training procedure
93
  [Axolotl](https://github.com/OpenAccess-AI-Collective/axolotl) was used for training
94
+ on a 4x NVidia A40 GPU cluster.
95
+
96
+ The A40 GPU cluster has been graciously provided by [Arc Compute](https://www.arccompute.io/).
97
+
98
+ The model has been trained as an 8-bit LoRA adapter, and
99
  it's so large because a LoRA rank of 256 was also used. The reasoning was that this
100
  might have helped the model internalize any newly acquired information, making the
101
+ training process closer to a full finetune. It's suggested to merge the adapter to
102
+ the base Llama2-7B model (or other Llama2-based models).
 
 
103
 
104
  ### Training hyperparameters
105
  For the first pass these settings were used: