Update README.md

README.md

@@ -1,7 +1,9 @@
 ---
 base_model: unsloth/Qwen2.5-14B-Instruct
 language:
-- en
+- de
+- fr
+- it
 license: apache-2.0
 tags:
 - text-generation-inference
@@ -9,14 +11,121 @@ tags:
 - unsloth
 - qwen2
 - trl
+datasets:
+- ipst/slds
+metrics:
+- bertscore
+- bleu
+- rouge
 ---
+# Model Card for Qwen2.5-7B-Instruct-SLDS
 
-# Uploaded  model
+## Model Summary
 
-- **Developed by:** rcds
-- **License:** apache-2.0
-- **Finetuned from model :** unsloth/Qwen2.5-14B-Instruct
+This model is a **Qwen2.5-7B-Instruct fine-tuned on the Swiss Landmark Decisions Summarization (SLDS) dataset**.  
+SLDS is a multilingual dataset of **20,000 Swiss Federal Supreme Court decisions** (1954–2024), each paired with **headnotes in German, French, and Italian**, resulting in ~60,000 decision–headnote pairs.  
 
-This qwen2 model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.
+The model is optimized for **legal abstractive summarization** and is capable of producing **concise, legally structured headnotes**.  
+It can be used for both **monolingual** and **cross-lingual summarization** tasks.
+
+This model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.
 
 [<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)
+
+---
+
+## Intended Use
+
+- **Primary Task**: Judicial summarization (decision → headnote generation).  
+- **Languages**: German (`de`), French (`fr`), Italian (`it`).  
+- **Scenarios**:  
+  - Monolingual summarization: e.g., German decision → German headnote.  
+  - Cross-lingual summarization: e.g., German decision → French headnote.  
+  - Legal research support: assisting in retrieval and navigation of court decisions.  
+
+**Not intended for**:  
+- Replacing human legal expertise.  
+- Serving as an authoritative legal source.  
+- Automated legal advice or decision-making.
+
+---
+
+## Training Data
+
+- **Dataset**: [Swiss Landmark Decisions Summarization (SLDS)](https://huggingface.co/datasets/ipst/slds).  
+- **Size**: ~20K decisions, ~60K decision–headnote pairs.  
+- **Splits**: Train (1954–2021), Validation (2022), Test (2023–2024).  
+- **Source**: [Swiss Federal Supreme Court](https://www.bger.ch).  
+
+---
+
+## Training Procedure
+
+- **Base Models**:  
+  - Qwen2.5 family (0.5B–14B)  
+  - Llama 3.2 (3B)  
+  - Phi-3.5-mini  
+
+- **Fine-tuning Objective**: Conditional generation (decision → headnote).  
+- **Evaluation Metrics**:  
+  - Lexical: ROUGE-1/2/L, BLEU, BERTScore.  
+  - Domain-specific: LLM-as-a-Judge framework (DeepSeek V3) assessing five rubrics: accuracy, completeness, clarity, legal citations, and considerations.  
+
+---
+
+## Model Performance
+
+On the SLDS test set (2023–2024):
+
+| Model | Setting | BERTScore ↑ | BLEU ↑ | ROUGE-1 ↑ | ROUGE-2 ↑ | ROUGE-L ↑ | JUDGE ↑ |
+|:--- |:--- |:--- |:--- |:--- |:--- |:--- |:--- |
+| [Phi-3.5-mini](https://huggingface.co/ipst/Phi-3.5-mini-instruct-SLDS) | fine-tuned | 11.24 ± 3.82 | 34.84 ± 0.41 | 31.20 ± 2.08 | 14.11 ± 1.27 | 20.96 ± 1.35 | 15.25 ± 2.32 |
+| [Llama 3.2B](https://huggingface.co/ipst/Llama-3.2-3B-Instruct-SLDS) | fine-tuned | 15.20 ± 4.40 | 21.89 ± 0.42 | 31.89 ± 2.34 | 14.87 ± 1.61 | 22.49 ± 1.60 | 18.47 ± 2.99 |
+| [Qwen2.5 0.5B](https://huggingface.co/ipst/Qwen2.5-0.5B-Instruct-SLDS) | fine-tuned | -1.37 ± 3.85 | 32.20 ± 0.35 | 23.87 ± 1.68 | 9.46 ± 0.94 | 17.37 ± 1.09 | 5.80 ± 1.26 |
+| [Qwen2.5 1.5B](https://huggingface.co/ipst/Qwen2.5-1.5B-Instruct-SLDS) | fine-tuned | 19.81 ± 2.72 | 36.79 ± 0.34 | 33.03 ± 1.73 | 14.14 ± 1.08 | 22.67 ± 1.13 | 15.92 ± 2.27 |
+| [Qwen2.5 3B](https://huggingface.co/ipst/Qwen2.5-3B-Instruct-SLDS) | fine-tuned | 23.23 ± 2.80 | 38.42 ± 0.34 | 35.18 ± 1.79 | 15.66 ± 1.23 | 24.10 ± 1.17 | 20.31 ± 2.66 |
+| [Qwen2.5 7B](https://huggingface.co/ipst/Qwen2.5-7B-Instruct-SLDS) | fine-tuned | 29.59 ± 1.97 | 41.40 ± 0.34 | 39.24 ± 1.59 | 18.26 ± 1.25 | 26.44 ± 1.15 | 28.37 ± 3.07 |
+| [Qwen2.5 14B](https://huggingface.co/ipst/Qwen2.5-14B-Instruct-SLDS) | fine-tuned | **32.48 ± 1.98** | **41.80 ± 0.37** | 40.04 ± 1.74 | **19.99 ± 1.41** | **28.00 ± 1.28** | 31.38 ± 3.19 |
+| GPT-4o | one-shot | 30.44 ± 1.74 | 31.89 ± 0.25 | **42.12 ± 1.79** | 18.92 ± 1.22 | 25.92 ± 1.05 | 39.70 ± 2.66 |
+| Claude 3.5 Sonnet | one-shot | 5.53 ± 2.00 | 21.88 ± 0.25 | 41.86 ± 1.64 | 19.23 ± 1.19 | 27.67 ± 1.20 | 41.25 ± 2.90 |
+| DeepSeek-R1 | one-shot | 20.28 ± 1.45 | 22.37 ± 0.18 | 38.30 ± 1.82 | 15.97 ± 0.85 | 21.03 ± 0.84 | **42.28 ± 2.21** |
+| o3-mini | one-shot | 14.18 ± 1.31 | 20.55 ± 0.17 | 34.77 ± 1.43 | 11.92 ± 0.69 | 18.21 ± 0.67 | 34.82 ± 2.41 |
+
+- **Lexical metrics**: Fine-tuned models outperform in overlap-based scores.  
+- **LLM-judge scores**: Larger proprietary and reasoning models outperform in legal precision.  
+
+---
+
+## Limitations
+
+- **Language imbalance**: German decisions dominate, while Italian remains underrepresented.  
+- **Biases**: Headnotes reflect judicial style and conventions, not neutral summaries.  
+- **Evaluation mismatch**: ROUGE and BLEU may not fully capture legal accuracy.  
+- **Overfitting risk**: Models may overfit to formulaic headnote structures.  
+- **Cross-lingual difficulty**: Some models struggle with non-monolingual headnote generation.  
+
+---
+
+## Ethical Considerations
+
+- **Sensitive information**: All data is anonymized by the Swiss Federal Supreme Court before publication.  
+- **Legal risk**: Generated headnotes must not be used as official legal advice.  
+- **Fair use**: Ensure attribution when reusing outputs.  
+
+---
+
+## How to Cite
+
+If you use this model, please cite the dataset paper:
+
+```bibtex
+@article{rolshoven2025slds,
+      title={Unlocking Legal Knowledge: A Multilingual Dataset for Judicial Summarization in Switzerland}, 
+      author={Luca Rolshoven and Vishvaksenan Rasiah and Srinanda Brügger Bose and Sarah Hostettler and Lara Burkhalter and Matthias Stürmer and Joel Niklaus},
+      year={2025},
+      eprint={2410.13456},
+      archivePrefix={arXiv},
+      primaryClass={cs.CL},
+      url={https://arxiv.org/abs/2410.13456}, 
+}
+```