Improve model card: Add paper link and refine description (#2)
Browse files- Improve model card: Add paper link and refine description (2c8c0ba506f7c5576909d1fd19627d656b496a91)
Co-authored-by: Niels Rogge <[email protected]>
README.md
CHANGED
@@ -1,4 +1,11 @@
|
|
1 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2 |
library_name: transformers
|
3 |
pipeline_tag: text-generation
|
4 |
tags:
|
@@ -7,19 +14,11 @@ tags:
|
|
7 |
- RAG evaluation
|
8 |
- cognitive statements
|
9 |
- factual consistency
|
10 |
-
datasets:
|
11 |
-
- future7/CogniBench
|
12 |
-
- future7/CogniBench-L
|
13 |
-
language:
|
14 |
-
- en
|
15 |
-
base_model:
|
16 |
-
- meta-llama/Meta-Llama-3-8B
|
17 |
---
|
18 |
|
19 |
-
|
20 |
# CogniDet: Cognitive Faithfulness Detector for LLMs
|
21 |
|
22 |
-
**CogniDet** is a state-of-the-art model for detecting **both factual and cognitive hallucinations** in Large Language Model (LLM) outputs. Developed as part of the [CogniBench](https://github.com/FUTUREEEEEE/CogniBench) framework, it specifically addresses the challenge of evaluating inference-based statements beyond simple fact regurgitation.
|
23 |
|
24 |
## Key Features ✨
|
25 |
1. **Dual Detection Capability**
|
@@ -55,7 +54,9 @@ model = AutoModelForCausalLM.from_pretrained(model_id)
|
|
55 |
|
56 |
def detect_hallucinations(context, response):
|
57 |
inputs = tokenizer(
|
58 |
-
f"CONTEXT: {context}
|
|
|
|
|
59 |
return_tensors="pt"
|
60 |
)
|
61 |
outputs = model.generate(**inputs, max_new_tokens=100)
|
|
|
1 |
---
|
2 |
+
base_model:
|
3 |
+
- meta-llama/Meta-Llama-3-8B
|
4 |
+
datasets:
|
5 |
+
- future7/CogniBench
|
6 |
+
- future7/CogniBench-L
|
7 |
+
language:
|
8 |
+
- en
|
9 |
library_name: transformers
|
10 |
pipeline_tag: text-generation
|
11 |
tags:
|
|
|
14 |
- RAG evaluation
|
15 |
- cognitive statements
|
16 |
- factual consistency
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
17 |
---
|
18 |
|
|
|
19 |
# CogniDet: Cognitive Faithfulness Detector for LLMs
|
20 |
|
21 |
+
**CogniDet** is a state-of-the-art model for detecting **both factual and cognitive hallucinations** in Large Language Model (LLM) outputs. Developed as part of the [CogniBench](https://github.com/FUTUREEEEEE/CogniBench) framework, it specifically addresses the challenge of evaluating inference-based statements beyond simple fact regurgitation. The model is presented in the paper [CogniBench: A Legal-inspired Framework and Dataset for Assessing Cognitive Faithfulness of Large Language Models](https://huggingface.co/papers/2505.20767).
|
22 |
|
23 |
## Key Features ✨
|
24 |
1. **Dual Detection Capability**
|
|
|
54 |
|
55 |
def detect_hallucinations(context, response):
|
56 |
inputs = tokenizer(
|
57 |
+
f"CONTEXT: {context}
|
58 |
+
RESPONSE: {response}
|
59 |
+
HALLUCINATIONS:",
|
60 |
return_tensors="pt"
|
61 |
)
|
62 |
outputs = model.generate(**inputs, max_new_tokens=100)
|