sasha HF staff commited on
Commit
a583f4d
1 Parent(s): a2b0422

adding 9b and 9b instruct

Browse files
Files changed (1) hide show
  1. README.md +21 -6
README.md CHANGED
@@ -306,9 +306,9 @@ Similarly to the base IDEFICS models, we performed checkpoint selection to stop
306
 
307
  ## Hardware
308
 
309
- The IDEFICS models were trained on an AWS SageMaker cluster with 8x80GB A100 GPUs nodes and EFA network.
310
 
311
- - IDEFICS-80B took ~28 days of training on 64 nodes (512 GPUs).
312
  - IDEFICS-80b-instruct finetuned the base model for ~3 days on 48 nodes (384 GPUs).
313
 
314
 
@@ -328,20 +328,35 @@ We distinguish the 3 phases of the creation of IDEFICS and report our carbon emi
328
  - **Compute Region:** France (57g CO2eq/kWh)
329
  - **Carbon Emitted:** 16,714 kgs of CO2eq
330
 
331
- *IDEFICS-80B pretraining*
 
 
 
 
 
 
 
 
 
 
 
 
 
 
332
  - **Hardware Type:** 512 NVIDIA A100 GPUs
333
  - **Hours used:** 672 hours (28 days)
334
  - **Cloud Provider:** AWS
335
  - **Compute Region:** US-West 2 (288g CO2eq/kWh)
336
- - **Carbon Emitted:** 39,498 kgs of CO2eq
337
 
338
  *IDEFICS-80b-instruct finetuning*
339
  - **Hardware Type:** 384 NVIDIA A100 GPUs
340
  - **Hours used:** 72 hours (3 days)
341
  - **Cloud Provider:** AWS
342
  - **Compute Region:** US-West 2 (288g CO2eq/kWh)
343
- - **Carbon Emitted:** 3,174 kgs of CO2eq
344
 
 
345
 
346
  # Bias, Risks, and Limitations
347
 
@@ -465,4 +480,4 @@ Stas Bekman*, Léo Tronchon*, Hugo Laurençon*, Lucile Saulnier*, Amanpreet Sing
465
 
466
  # Model Card Contact
467
 
468
- Please open a discussion on the Community tab!
 
306
 
307
  ## Hardware
308
 
309
+ The IDEFICS models were trained on an AWS SageMaker cluster with 8x80GB A100 GPUs nodes and EFA network.
310
 
311
+ - IDEFICS-80B took ~28 days of training on 64 nodes (512 GPUs).
312
  - IDEFICS-80b-instruct finetuned the base model for ~3 days on 48 nodes (384 GPUs).
313
 
314
 
 
328
  - **Compute Region:** France (57g CO2eq/kWh)
329
  - **Carbon Emitted:** 16,714 kgs of CO2eq
330
 
331
+ *IDEFICS-9b pretraining*
332
+ - **Hardware Type:** 128 NVIDIA A100 GPUs
333
+ - **Hours used:** 350 hours
334
+ - **Cloud Provider:** AWS
335
+ - **Compute Region:** US-West 2 (288g CO2eq/kWh)
336
+ - **Carbon Emitted:** 5,160 kg of CO2eq
337
+
338
+ *IDEFICS-9b-instruct finetuning*
339
+ - **Hardware Type:** 128 NVIDIA A100 GPUs
340
+ - **Hours used:** 70 hours
341
+ - **Cloud Provider:** AWS
342
+ - **Compute Region:** US-West 2 (288g CO2eq/kWh)
343
+ - **Carbon Emitted:** 1,032 kg of CO2eq
344
+
345
+ *IDEFICS-80b pretraining*
346
  - **Hardware Type:** 512 NVIDIA A100 GPUs
347
  - **Hours used:** 672 hours (28 days)
348
  - **Cloud Provider:** AWS
349
  - **Compute Region:** US-West 2 (288g CO2eq/kWh)
350
+ - **Carbon Emitted:** 39,498 kg of CO2eq
351
 
352
  *IDEFICS-80b-instruct finetuning*
353
  - **Hardware Type:** 384 NVIDIA A100 GPUs
354
  - **Hours used:** 72 hours (3 days)
355
  - **Cloud Provider:** AWS
356
  - **Compute Region:** US-West 2 (288g CO2eq/kWh)
357
+ - **Carbon Emitted:** 3,174 kg of CO2eq
358
 
359
+ This means that the total carbon footprint of the entire IDEFICS project can be estimated at **65.57 tons of CO2eq**, which is roughly equal to 168,092 miles driven by an average gasoline-powered car or 8.3 homes' energy use for one year, according to the [US Environmental Protection Agency](https://www.epa.gov/energy/greenhouse-gas-equivalencies-calculator).
360
 
361
  # Bias, Risks, and Limitations
362
 
 
480
 
481
  # Model Card Contact
482
 
483
+ Please open a discussion on the Community tab!