davidpomerenke commited on
Commit
3f60023
·
verified ·
1 Parent(s): 34983b9

Upload from GitHub Actions: Make community links work, add CONTRIBUTING

Browse files
Files changed (2) hide show
  1. CONTRIBUTING.md +16 -0
  2. frontend/src/App.js +5 -2
CONTRIBUTING.md ADDED
@@ -0,0 +1,16 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Contributing
2
+
3
+ ## Adding a new benchmark dataset
4
+
5
+ - Create a new file in `evals/datasets/` for loading the dataset(s) for your task, ideally via HuggingFace's `datasets` library.
6
+ - Define the task and its evaluation metric(s) in `evals/tasks.py`.
7
+ - Update `evals/backend.py` and `frontend/src/components/ScoreColumns.js` to include the new task and its metrics.
8
+ - Submit a pull request.
9
+
10
+ ## Adding a new model
11
+
12
+ You can submit requests [here](https://forms.gle/ckvY9pS7XLcHYnaV8), and we will take care of the rest.
13
+
14
+ ## Adding a new language
15
+
16
+ We believe that we already have all of the languages. If this is not the case, open an issue!
frontend/src/App.js CHANGED
@@ -80,11 +80,14 @@ function App () {
80
  <li><b>Policymakers and funders</b> can identify and prioritize neglected languages.</li>
81
  <li><b>Model developers</b> can compete on our <i>AI Language Proficiency</i> metric.</li>
82
  </ul>
83
- <p>We invite the community to <a href="#">submit</a> their custom finetuned models, and to <a href="#">integrate</a> benchmarks for more languages and tasks.</p>
84
  <p>Benchmark results automatically refresh every night and include the most popular models on <a href="https://openrouter.ai" target="_blank" rel="noopener noreferrer">OpenRouter</a>, plus community-listed models.</p>
85
- <p>For a detailed methodlogy, see <a href="#">XXX</a>.</p>
86
  <p>The AI Language Proficiency Monitor is a collaboration between BMZ's <a href="https://www.bmz-digital.global/en/overview-of-initiatives/the-bmz-data-lab/" target="_blank" rel="noopener noreferrer">Data Lab</a>, GIZ's <a href="https://www.giz.de/expertise/html/61982.html" target="_blank" rel="noopener noreferrer">FairForward</a> initiative, and the <a href="https://www.dfki.de/en/web/research/research-departments/multilinguality-and-language-technology/ee-team" target="_blank" rel="noopener noreferrer">E&E group</a> of DFKI's Multilinguality and Language Technology Lab.</p>
87
  </div>
 
 
 
88
  </header>
89
  <main
90
  style={{
 
80
  <li><b>Policymakers and funders</b> can identify and prioritize neglected languages.</li>
81
  <li><b>Model developers</b> can compete on our <i>AI Language Proficiency</i> metric.</li>
82
  </ul>
83
+ <p>We invite the community to <a href="https://forms.gle/ckvY9pS7XLcHYnaV8" target="_blank" rel="noopener noreferrer">submit</a> their custom finetuned models, and to <a href="https://github.com/datenlabor-bmz/ai-language-monitor/blob/main/CONTRIBUTING.md" target="_blank" rel="noopener noreferrer">integrate</a> benchmarks for more languages and tasks.</p>
84
  <p>Benchmark results automatically refresh every night and include the most popular models on <a href="https://openrouter.ai" target="_blank" rel="noopener noreferrer">OpenRouter</a>, plus community-listed models.</p>
85
+ {/* <p>For a detailed methodlogy, see <a href="#">XXX</a>.</p> */}
86
  <p>The AI Language Proficiency Monitor is a collaboration between BMZ's <a href="https://www.bmz-digital.global/en/overview-of-initiatives/the-bmz-data-lab/" target="_blank" rel="noopener noreferrer">Data Lab</a>, GIZ's <a href="https://www.giz.de/expertise/html/61982.html" target="_blank" rel="noopener noreferrer">FairForward</a> initiative, and the <a href="https://www.dfki.de/en/web/research/research-departments/multilinguality-and-language-technology/ee-team" target="_blank" rel="noopener noreferrer">E&E group</a> of DFKI's Multilinguality and Language Technology Lab.</p>
87
  </div>
88
+ <a href="https://github.com/datenlabor-bmz/ai-language-monitor" target="_blank" rel="noopener noreferrer" style={{ textDecoration: 'none', color: 'inherit', marginTop: '2rem' }}>
89
+ <i className="pi pi-github" style={{ fontSize: '1.5rem' }} />
90
+ </a>
91
  </header>
92
  <main
93
  style={{