File size: 2,913 Bytes
0712a97 d414210 94cf687 0712a97 94cf687 9516e66 0712a97 d414210 86b3b93 d414210 86b3b93 d414210 94cf687 d414210 94cf687 d414210 94cf687 0712a97 4cce56e 0712a97 94cf687 0712a97 a25b32d 0712a97 94cf687 d414210 94cf687 d414210 94cf687 d414210 94cf687 d414210 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 |
---
license: apache-2.0
language:
- en
base_model:
- Qwen/Qwen3-4B
pipeline_tag: text-ranking
tags:
- finance
- legal
- code
- stem
- medical
library_name: sentence-transformers
---
<img src="https://i.imgur.com/oxvhvQu.png"/>
# Releasing zeroentropy/zerank-1-small
In search enginers, [rerankers are crucial](https://www.zeroentropy.dev/blog/what-is-a-reranker-and-do-i-need-one) for improving the accuracy of your retrieval system.
This 1.7B reranker is the smaller version of our flagship model [zeroentropy/zerank-1](https://huggingface.co/zeroentropy/zerank-1). Though the model is over 2x smaller, it maintains nearly the same standard of performance, continuing to outperform other popular rerankers, and displaying massive accuracy gains over traditional vector search.
We release this model under the open-source Apache 2.0 license, in order to support the open-source community and push the frontier of what's possible with open-source models.
## How to Use
```python
from sentence_transformers import CrossEncoder
model = CrossEncoder("zeroentropy/zerank-1-small", trust_remote_code=True)
query_documents = [
("What is 2+2?", "4"),
("What is 2+2?", "The answer is definitely 1 million"),
]
scores = model.predict(query_documents)
print(scores)
```
The model can also be inferenced using ZeroEntropy's [/models/rerank](https://docs.zeroentropy.dev/api-reference/models/rerank) endpoint.
## Evaluations
NDCG@10 scores between `zerank-1-small` and competing closed-source proprietary rerankers. Since we are evaluating rerankers, OpenAI's `text-embedding-3-small` is used as an initial retriever for the Top 100 candidate documents.
| Task | Embedding | cohere-rerank-v3.5 | Salesforce/Llama-rank-v1 | **zerank-1-small** | zerank-1 |
|----------------|-----------|--------------------|--------------------------|----------------|----------|
| Code | 0.678 | 0.724 | 0.694 | **0.730** | 0.754 |
| Conversational | 0.250 | 0.571 | 0.484 | **0.556** | 0.596 |
| Finance | 0.839 | 0.824 | 0.828 | **0.861** | 0.894 |
| Legal | 0.703 | 0.804 | 0.767 | **0.817** | 0.821 |
| Medical | 0.619 | 0.750 | 0.719 | **0.773** | 0.796 |
| STEM | 0.401 | 0.510 | 0.595 | **0.680** | 0.694 |
Comparing BM25 and Hybrid Search without and with `zerank-1-small`:
<img src="https://cdn-uploads.huggingface.co/production/uploads/67776f9dcd9c9435499eafc8/2GPVHFrI39FspnSNklhsM.png" alt="Description" width="400"/> <img src="https://cdn-uploads.huggingface.co/production/uploads/67776f9dcd9c9435499eafc8/dwYo2D7hoL8QiE8u3yqr9.png" alt="Description" width="400"/> |