Update README.md
Browse files
README.md
CHANGED
|
@@ -31,24 +31,6 @@ This is the model card for **phi-2-dialogsum**, a dialogue summarization model b
|
|
| 31 |
### Direct Use
|
| 32 |
This model can be used directly for **dialogue summarization** tasks. For example, given a multi-turn conversation, the model will produce a succinct summary capturing the key information and context.
|
| 33 |
|
| 34 |
-
### Downstream Use [optional]
|
| 35 |
-
Could be fine-tuned or adapted for other text summarization tasks, especially conversation-like data (customer service transcripts, chat logs, interviews, etc.).
|
| 36 |
-
|
| 37 |
-
### Out-of-Scope Use
|
| 38 |
-
- Generating harmful or misleading content.
|
| 39 |
-
- Deploying in high-stakes scenarios without proper validation (e.g., medical or legal advice).
|
| 40 |
-
|
| 41 |
-
## Bias, Risks, and Limitations
|
| 42 |
-
|
| 43 |
-
- **Biases:** The model may reflect biases present in the data used to train or fine-tune it.
|
| 44 |
-
- **Risks:** Summaries could omit critical context or misrepresent the conversation.
|
| 45 |
-
- **Limitations:** The model’s performance may degrade on conversations with specialized jargon, code-switching, or extremely long contexts.
|
| 46 |
-
|
| 47 |
-
### Recommendations
|
| 48 |
-
- Always review generated summaries for accuracy.
|
| 49 |
-
- Be mindful of potential biases or omissions.
|
| 50 |
-
- Avoid using the model as the sole source of truth in sensitive domains.
|
| 51 |
-
|
| 52 |
## How to Get Started with the Model
|
| 53 |
|
| 54 |
Below is a quick code snippet to load and run inference with this model:
|
|
@@ -69,6 +51,25 @@ inputs = tokenizer([input_text], max_length=512, truncation=True, return_tensors
|
|
| 69 |
summary_ids = model.generate(**inputs, max_length=60, num_beams=4, early_stopping=True)
|
| 70 |
summary = tokenizer.decode(summary_ids[0], skip_special_tokens=True)
|
| 71 |
|
| 72 |
-
print("Summary:", summary)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 73 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 74 |
|
|
|
|
| 31 |
### Direct Use
|
| 32 |
This model can be used directly for **dialogue summarization** tasks. For example, given a multi-turn conversation, the model will produce a succinct summary capturing the key information and context.
|
| 33 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 34 |
## How to Get Started with the Model
|
| 35 |
|
| 36 |
Below is a quick code snippet to load and run inference with this model:
|
|
|
|
| 51 |
summary_ids = model.generate(**inputs, max_length=60, num_beams=4, early_stopping=True)
|
| 52 |
summary = tokenizer.decode(summary_ids[0], skip_special_tokens=True)
|
| 53 |
|
| 54 |
+
print("Summary:", summary)
|
| 55 |
+
```
|
| 56 |
+
|
| 57 |
+
## Training Details
|
| 58 |
+
|
| 59 |
+
Training dataset (Dialogsum)[https://huggingface.co/datasets/neil-code/dialogsum-test]
|
| 60 |
+
|
| 61 |
+
## Evaluation
|
| 62 |
+
|
| 63 |
+
ORIGINAL MODEL:
|
| 64 |
+
{'rouge1': 0.2990526195120211, 'rouge2': 0.10874019046839419, 'rougeL': 0.21186900909813286, 'rougeLsum': 0.22342464591439556}
|
| 65 |
+
|
| 66 |
+
PEFT MODEL:
|
| 67 |
+
{'rouge1': 0.3132817683433486, 'rouge2': 0.1070363134080079, 'rougeL': 0.23226760188839027, 'rougeLsum': 0.25947902747914586}
|
| 68 |
+
|
| 69 |
+
## Absolute percentage improvement of PEFT MODEL over ORIGINAL MODEL
|
| 70 |
|
| 71 |
+
rouge1: 1.42%
|
| 72 |
+
rouge2: -0.17%
|
| 73 |
+
rougeL: 2.04%
|
| 74 |
+
rougeLsum: 3.61%
|
| 75 |
|