Spaces:
Running
on
CPU Upgrade
Update model size filter intervals
Browse filesHello!
We have cloned this leaderboard to run a [Open Korean LLM Leaderboard](https://huggingface.co/spaces/upstage/open-ko-llm-leaderboard). We have over 100 models in the past couple of weeks so far!
As discussed with
@clefourrier
, I'm proposing this change to make the model size filter behavior more clear.
Currently the notation like `~13B` means `around 13B` and shows models between 12B and 15B. Although individuals may interpret the notation differently, we thought this could be improved and be more intuitive. Moreover, with `~3B` and `~7B` shows `1.5B~5B` and `6B~11B` respectively, there is no way to filter for models with size between 5B and 6B, such as 5.8B. We actually had a request from one of participants for size 5.8B. (See [discussion](https://huggingface.co/spaces/upstage/open-ko-llm-leaderboard/discussions/5))
So the proposal is to change the notation to `1.5~3B` and filter for models with size between 1.5B and 3B.
![image.png](https://cdn-uploads.huggingface.co/production/uploads/64dd2f5ba437abbfbe962773/QxCIWL4zy7j_qOSFij7eI.png)
I hope this makes sense!
@@ -254,12 +254,13 @@ def select_columns(df: pd.DataFrame, columns: list) -> pd.DataFrame:
|
|
254 |
|
255 |
NUMERIC_INTERVALS = {
|
256 |
"Unknown": pd.Interval(-1, 0, closed="right"),
|
257 |
-
"
|
258 |
-
"~3B": pd.Interval(1.5,
|
259 |
-
"~7B": pd.Interval(
|
260 |
-
"~13B": pd.Interval(
|
261 |
-
"~35B": pd.Interval(
|
262 |
-
"60B
|
|
|
263 |
}
|
264 |
|
265 |
|
|
|
254 |
|
255 |
NUMERIC_INTERVALS = {
|
256 |
"Unknown": pd.Interval(-1, 0, closed="right"),
|
257 |
+
"0~1.5B": pd.Interval(0, 1.5, closed="right"),
|
258 |
+
"1.5~3B": pd.Interval(1.5, 3, closed="right"),
|
259 |
+
"3~7B": pd.Interval(3, 7, closed="right"),
|
260 |
+
"7~13B": pd.Interval(7, 13, closed="right"),
|
261 |
+
"13~35B": pd.Interval(13, 35, closed="right"),
|
262 |
+
"35~60B": pd.Interval(35, 60, closed="right"),
|
263 |
+
"60B+": pd.Interval(60, 10000, closed="right"),
|
264 |
}
|
265 |
|
266 |
|