ahycl/ettin-cleaned-data-1-epoch-25000

PyLate model based on jhu-clsp/ettin-encoder-17m

This is a PyLate model finetuned from jhu-clsp/ettin-encoder-17m on the train dataset. It maps sentences & paragraphs to sequences of 128-dimensional dense vectors and can be used for semantic textual similarity using the MaxSim operator.

Model Details

Model Description

• Model Type: PyLate model
• Base model: jhu-clsp/ettin-encoder-17m
• Document Length: 300 tokens
• Query Length: 32 tokens
• Output Dimensionality: 128 tokens
• Similarity Function: MaxSim
• Training Dataset:
  • train

Model Sources

• Documentation: PyLate Documentation
• Repository: PyLate on GitHub
• Hugging Face: PyLate models on Hugging Face

Full Model Architecture

ColBERT(
  (0): Transformer({'max_seq_length': 299, 'do_lower_case': False}) with Transformer model: ModernBertModel 
  (1): Dense({'in_features': 256, 'out_features': 128, 'bias': False, 'activation_function': 'torch.nn.modules.linear.Identity'})
)

Usage

First install the PyLate library:

pip install -U pylate

Retrieval

PyLate provides a streamlined interface to index and retrieve documents using ColBERT models. The index leverages the Voyager HNSW index to efficiently handle document embeddings and enable fast retrieval.

Indexing documents

First, load the ColBERT model and initialize the Voyager index, then encode and index your documents:

from pylate import indexes, models, retrieve

# Step 1: Load the ColBERT model
model = models.ColBERT(
    model_name_or_path=pylate_model_id,
)

# Step 2: Initialize the Voyager index
index = indexes.Voyager(
    index_folder="pylate-index",
    index_name="index",
    override=True,  # This overwrites the existing index if any
)

# Step 3: Encode the documents
documents_ids = ["1", "2", "3"]
documents = ["document 1 text", "document 2 text", "document 3 text"]

documents_embeddings = model.encode(
    documents,
    batch_size=32,
    is_query=False,  # Ensure that it is set to False to indicate that these are documents, not queries
    show_progress_bar=True,
)

# Step 4: Add document embeddings to the index by providing embeddings and corresponding ids
index.add_documents(
    documents_ids=documents_ids,
    documents_embeddings=documents_embeddings,
)

Note that you do not have to recreate the index and encode the documents every time. Once you have created an index and added the documents, you can re-use the index later by loading it:

# To load an index, simply instantiate it with the correct folder/name and without overriding it
index = indexes.Voyager(
    index_folder="pylate-index",
    index_name="index",
)

Retrieving top-k documents for queries

Once the documents are indexed, you can retrieve the top-k most relevant documents for a given set of queries. To do so, initialize the ColBERT retriever with the index you want to search in, encode the queries and then retrieve the top-k documents to get the top matches ids and relevance scores:

# Step 1: Initialize the ColBERT retriever
retriever = retrieve.ColBERT(index=index)

# Step 2: Encode the queries
queries_embeddings = model.encode(
    ["query for document 3", "query for document 1"],
    batch_size=32,
    is_query=True,  #  # Ensure that it is set to False to indicate that these are queries
    show_progress_bar=True,
)

# Step 3: Retrieve top-k documents
scores = retriever.retrieve(
    queries_embeddings=queries_embeddings,
    k=10,  # Retrieve the top 10 matches for each query
)

Reranking

If you only want to use the ColBERT model to perform reranking on top of your first-stage retrieval pipeline without building an index, you can simply use rank function and pass the queries and documents to rerank:

from pylate import rank, models

queries = [
    "query A",
    "query B",
]

documents = [
    ["document A", "document B"],
    ["document 1", "document C", "document B"],
]

documents_ids = [
    [1, 2],
    [1, 3, 2],
]

model = models.ColBERT(
    model_name_or_path=pylate_model_id,
)

queries_embeddings = model.encode(
    queries,
    is_query=True,
)

documents_embeddings = model.encode(
    documents,
    is_query=False,
)

reranked_documents = rank.rerank(
    documents_ids=documents_ids,
    queries_embeddings=queries_embeddings,
    documents_embeddings=documents_embeddings,
)

Evaluation

Metrics

Py Late Information Retrieval

• Dataset: ['NanoClimateFEVER', 'NanoDBPedia', 'NanoFEVER', 'NanoFiQA2018', 'NanoHotpotQA', 'NanoMSMARCO', 'NanoNFCorpus', 'NanoNQ', 'NanoQuoraRetrieval', 'NanoSCIDOCS', 'NanoArguAna', 'NanoSciFact', 'NanoTouche2020']
• Evaluated with pylate.evaluation.pylate_information_retrieval_evaluator.PyLateInformationRetrievalEvaluator

Metric	NanoClimateFEVER	NanoDBPedia	NanoFEVER	NanoFiQA2018	NanoHotpotQA	NanoMSMARCO	NanoNFCorpus	NanoNQ	NanoQuoraRetrieval	NanoSCIDOCS	NanoArguAna	NanoSciFact	NanoTouche2020
MaxSim_accuracy@1	0.26	0.72	0.88	0.44	0.88	0.52	0.44	0.5	0.82	0.44	0.18	0.62	0.6735
MaxSim_accuracy@3	0.42	0.86	0.94	0.62	1.0	0.66	0.52	0.78	0.98	0.64	0.56	0.8	0.9796
MaxSim_accuracy@5	0.5	0.92	0.96	0.66	1.0	0.78	0.64	0.8	1.0	0.68	0.62	0.86	1.0
MaxSim_accuracy@10	0.74	0.94	0.98	0.74	1.0	0.82	0.66	0.84	1.0	0.8	0.76	0.88	1.0
MaxSim_precision@1	0.26	0.72	0.88	0.44	0.88	0.52	0.44	0.5	0.82	0.44	0.18	0.62	0.6735
MaxSim_precision@3	0.16	0.62	0.3267	0.2733	0.5467	0.22	0.3467	0.26	0.3867	0.3067	0.1867	0.28	0.6599
MaxSim_precision@5	0.116	0.556	0.204	0.208	0.344	0.156	0.328	0.164	0.248	0.24	0.124	0.188	0.6286
MaxSim_precision@10	0.092	0.486	0.108	0.132	0.18	0.082	0.258	0.088	0.126	0.16	0.076	0.1	0.5
MaxSim_recall@1	0.135	0.0732	0.8167	0.2409	0.44	0.52	0.0433	0.47	0.734	0.0917	0.18	0.595	0.0442
MaxSim_recall@3	0.215	0.1658	0.8933	0.3758	0.82	0.66	0.0766	0.72	0.9287	0.1897	0.56	0.77	0.1348
MaxSim_recall@5	0.2533	0.216	0.9233	0.4423	0.86	0.78	0.1177	0.75	0.9627	0.2457	0.62	0.845	0.2118
MaxSim_recall@10	0.3667	0.334	0.96	0.5565	0.9	0.82	0.1412	0.8	0.966	0.3267	0.76	0.88	0.3151
MaxSim_ndcg@10	0.2905	0.588	0.9048	0.464	0.8575	0.6645	0.3255	0.6545	0.9043	0.3302	0.4645	0.7485	0.5655
MaxSim_mrr@10	0.3754	0.7982	0.9112	0.5365	0.9333	0.6151	0.5032	0.6283	0.8973	0.5572	0.37	0.7097	0.8136
MaxSim_map@100	0.2288	0.4654	0.8781	0.3878	0.8	0.624	0.146	0.6016	0.8789	0.2551	0.3781	0.704	0.4122

Nano BEIR

• Dataset: NanoBEIR_mean
• Evaluated with pylate.evaluation.nano_beir_evaluator.NanoBEIREvaluator

Metric	Value
MaxSim_accuracy@1	0.5672
MaxSim_accuracy@3	0.7507
MaxSim_accuracy@5	0.8015
MaxSim_accuracy@10	0.8585
MaxSim_precision@1	0.5672
MaxSim_precision@3	0.3518
MaxSim_precision@5	0.2696
MaxSim_precision@10	0.1837
MaxSim_recall@1	0.3372
MaxSim_recall@3	0.5007
MaxSim_recall@5	0.556
MaxSim_recall@10	0.6251
MaxSim_ndcg@10	0.5971
MaxSim_mrr@10	0.6653
MaxSim_map@100	0.52

Training Details

Training Dataset

train

• Dataset: train at 63f51a3
• Size: 533,177 training samples
• Columns: query_id, document_ids, and scores
• Approximate statistics based on the first 1000 samples:

  	query_id	document_ids	scores
  type	int	list	list
  details	264: ~0.10% 836: ~0.10% 3582: ~0.10% 4599: ~0.10% 4645: ~0.10% 4853: ~0.10% 5154: ~0.10% 7504: ~0.10% 12254: ~0.10% 12283: ~0.10% 12916: ~0.10% 14049: ~0.10% 14587: ~0.10% 14828: ~0.10% 15674: ~0.10% 15813: ~0.10% 16728: ~0.10% 22006: ~0.10% 23675: ~0.10% 24199: ~0.10% 25323: ~0.10% 29213: ~0.10% 32344: ~0.10% 34604: ~0.10% 35424: ~0.10% 36148: ~0.10% 37826: ~0.10% 38185: ~0.10% 38242: ~0.10% 39839: ~0.10% 42077: ~0.10% 43614: ~0.10% 45073: ~0.10% 46289: ~0.10% 47507: ~0.10% 48005: ~0.10% 48785: ~0.10% 49216: ~0.10% 49636: ~0.10% 49970: ~0.10% 51145: ~0.10% 52075: ~0.10% 52094: ~0.10% 52725: ~0.10% 52788: ~0.10% 54210: ~0.10% 55032: ~0.10% 59546: ~0.10% 60941: ~0.10% 62649: ~0.10% 63333: ~0.10% 64197: ~0.10% 64879: ~0.10% 67608: ~0.10% 67627: ~0.10% 69463: ~0.10% 70002: ~0.10% 70429: ~0.10% 72166: ~0.10% 72518: ~0.10% 72607: ~0.10% 72791: ~0.10% 73325: ~0.10% 74078: ~0.10% 74857: ~0.10% 75323: ~0.10% 75816: ~0.10% 76929: ~0.10% 77306: ~0.10% 77845: ~0.10% 77889: ~0.10% 78077: ~0.10% 78256: ~0.10% 78401: ~0.10% 78798: ~0.10% 80329: ~0.10% 80871: ~0.10% 81089: ~0.10% 82179: ~0.10% 82883: ~0.10% 84168: ~0.10% 86891: ~0.10% 88535: ~0.10% 88953: ~0.10% 89346: ~0.10% 89386: ~0.10% 90699: ~0.10% 90795: ~0.10% 91367: ~0.10% 91795: ~0.10% 91834: ~0.10% 92070: ~0.10% 92523: ~0.10% 92753: ~0.10% 92787: ~0.10% 93317: ~0.10% 96382: ~0.10% 96455: ~0.10% 97274: ~0.10% 97603: ~0.10% 98140: ~0.10% 101205: ~0.10% 101305: ~0.10% 102707: ~0.10% 103074: ~0.10% 105437: ~0.10% 108207: ~0.10% 109776: ~0.10% 112056: ~0.10% 112955: ~0.10% 112977: ~0.10% 113635: ~0.10% 115551: ~0.10% 116098: ~0.10% 118057: ~0.10% 120298: ~0.10% 121703: ~0.10% 123429: ~0.10% 125043: ~0.10% 125979: ~0.10% 126851: ~0.10% 128218: ~0.10% 128804: ~0.10% 129598: ~0.10% 131299: ~0.10% 132114: ~0.10% 133553: ~0.10% 133696: ~0.10% 134460: ~0.10% 137602: ~0.10% 137679: ~0.10% 138121: ~0.10% 138260: ~0.10% 139039: ~0.10% 140651: ~0.10% 142305: ~0.10% 145653: ~0.10% 145683: ~0.10% 145763: ~0.10% 150202: ~0.10% 151135: ~0.10% 152307: ~0.10% 153693: ~0.10% 154470: ~0.10% 155587: ~0.10% 157602: ~0.10% 158565: ~0.10% 159177: ~0.10% 159224: ~0.10% 159341: ~0.10% 159892: ~0.10% 161881: ~0.10% 162414: ~0.10% 163765: ~0.10% 165888: ~0.10% 167756: ~0.10% 168048: ~0.10% 168894: ~0.10% 169991: ~0.10% 170731: ~0.10% 171705: ~0.10% 176165: ~0.10% 176798: ~0.10% 180259: ~0.10% 182660: ~0.10% 183930: ~0.10% 184045: ~0.10% 184676: ~0.10% 185294: ~0.10% 186475: ~0.10% 187155: ~0.10% 189571: ~0.10% 191383: ~0.10% 192165: ~0.10% 194207: ~0.10% 195056: ~0.10% 197377: ~0.10% 198546: ~0.10% 200183: ~0.10% 202122: ~0.10% 203519: ~0.10% 206220: ~0.10% 209739: ~0.10% 210554: ~0.10% 212638: ~0.10% 213096: ~0.10% 213410: ~0.10% 217541: ~0.10% 220993: ~0.10% 223913: ~0.10% 224657: ~0.10% 227497: ~0.10% 227726: ~0.10% 228099: ~0.10% 228451: ~0.10% 231416: ~0.10% 234348: ~0.10% 235869: ~0.10% 237784: ~0.10% 240739: ~0.10% 246495: ~0.10% 246821: ~0.10% 248675: ~0.10% 249798: ~0.10% 249962: ~0.10% 249977: ~0.10% 250019: ~0.10% 250548: ~0.10% 251089: ~0.10% 254878: ~0.10% 255183: ~0.10% 255727: ~0.10% 256321: ~0.10% 258276: ~0.10% 260993: ~0.10% 261225: ~0.10% 261247: ~0.10% 262123: ~0.10% 262508: ~0.10% 262950: ~0.10% 266047: ~0.10% 267089: ~0.10% 267192: ~0.10% 268642: ~0.10% 273171: ~0.10% 273864: ~0.10% 274521: ~0.10% 274586: ~0.10% 275037: ~0.10% 275643: ~0.10% 276744: ~0.10% 277212: ~0.10% 277990: ~0.10% 280012: ~0.10% 281699: ~0.10% 282128: ~0.10% 283913: ~0.10% 284268: ~0.10% 285905: ~0.10% 287456: ~0.10% 287506: ~0.10% 288154: ~0.10% 288286: ~0.10% 289046: ~0.10% 292211: ~0.10% 292588: ~0.10% 293357: ~0.10% 294123: ~0.10% 295125: ~0.10% 299287: ~0.10% 300622: ~0.10% 302135: ~0.10% 303224: ~0.10% 304353: ~0.10% 304820: ~0.10% 305508: ~0.10% 310215: ~0.10% 310236: ~0.10% 310409: ~0.10% 311231: ~0.10% 312821: ~0.10% 314244: ~0.10% 314415: ~0.10% 314745: ~0.10% 316385: ~0.10% 317442: ~0.10% 318639: ~0.10% 318652: ~0.10% 320855: ~0.10% 321983: ~0.10% 322114: ~0.10% 323196: ~0.10% 324868: ~0.10% 327581: ~0.10% 329337: ~0.10% 330641: ~0.10% 331572: ~0.10% 331650: ~0.10% 331993: ~0.10% 334757: ~0.10% 336561: ~0.10% 336791: ~0.10% 337002: ~0.10% 338332: ~0.10% 338456: ~0.10% 339065: ~0.10% 339556: ~0.10% 339755: ~0.10% 339870: ~0.10% 340599: ~0.20% 341156: ~0.10% 342121: ~0.10% 343411: ~0.10% 344860: ~0.10% 345924: ~0.10% 346421: ~0.10% 346425: ~0.10% 348157: ~0.10% 351281: ~0.10% 351858: ~0.10% 353008: ~0.10% 353748: ~0.10% 359787: ~0.10% 359893: ~0.10% 360168: ~0.10% 361127: ~0.10% 362220: ~0.10% 362560: ~0.10% 364559: ~0.10% 366835: ~0.10% 367185: ~0.10% 369045: ~0.10% 371113: ~0.10% 375630: ~0.10% 376044: ~0.10% 376524: ~0.10% 377735: ~0.10% 378574: ~0.10% 379749: ~0.10% 379953: ~0.10% 384364: ~0.10% 384398: ~0.10% 384751: ~0.10% 385758: ~0.10% 385893: ~0.10% 386098: ~0.10% 387205: ~0.10% 387374: ~0.10% 388450: ~0.10% 388589: ~0.10% 388593: ~0.10% 389571: ~0.10% 389572: ~0.10% 391531: ~0.10% 391857: ~0.10% 393174: ~0.10% 393426: ~0.10% 398132: ~0.10% 398721: ~0.10% 399016: ~0.10% 399589: ~0.10% 400259: ~0.10% 401489: ~0.10% 401601: ~0.10% 403876: ~0.10% 403897: ~0.10% 404830: ~0.10% 406102: ~0.10% 406397: ~0.10% 407151: ~0.10% 407608: ~0.10% 409373: ~0.10% 410084: ~0.10% 411693: ~0.10% 411984: ~0.10% 412214: ~0.10% 412560: ~0.10% 413117: ~0.10% 413563: ~0.10% 413660: ~0.10% 414337: ~0.10% 416391: ~0.10% 417066: ~0.10% 417198: ~0.10% 417751: ~0.10% 417778: ~0.10% 420257: ~0.10% 420787: ~0.10% 421001: ~0.10% 421045: ~0.10% 421354: ~0.10% 421738: ~0.10% 429057: ~0.10% 429459: ~0.10% 430319: ~0.10% 431215: ~0.10% 431332: ~0.10% 432097: ~0.10% 432283: ~0.10% 435353: ~0.10% 437793: ~0.10% 438297: ~0.10% 439016: ~0.10% 439129: ~0.10% 439217: ~0.10% 439755: ~0.10% 440343: ~0.10% 440506: ~0.10% 441030: ~0.10% 441509: ~0.10% 443408: ~0.10% 443686: ~0.10% 445516: ~0.10% 445999: ~0.10% 447039: ~0.10% 447219: ~0.10% 447298: ~0.10% 453040: ~0.10% 453745: ~0.10% 454869: ~0.10% 455323: ~0.10% 456224: ~0.10% 456251: ~0.10% 457065: ~0.10% 459890: ~0.10% 463716: ~0.10% 465235: ~0.10% 465388: ~0.10% 470344: ~0.10% 470470: ~0.10% 471875: ~0.10% 472462: ~0.10% 474016: ~0.10% 476794: ~0.10% 479266: ~0.10% 479360: ~0.10% 480621: ~0.10% 482154: ~0.10% 483014: ~0.10% 485031: ~0.10% 485828: ~0.10% 486664: ~0.10% 488266: ~0.10% 489488: ~0.10% 490249: ~0.10% 490992: ~0.10% 491894: ~0.10% 491983: ~0.10% 492620: ~0.10% 492904: ~0.10% 493035: ~0.10% 493461: ~0.10% 494255: ~0.10% 496473: ~0.10% 496474: ~0.10% 496516: ~0.10% 496813: ~0.10% 496853: ~0.10% 499553: ~0.10% 499565: ~0.10% 499737: ~0.10% 500546: ~0.10% 501510: ~0.10% 501837: ~0.10% 502910: ~0.10% 510559: ~0.10% 511473: ~0.10% 512440: ~0.10% 513832: ~0.10% 514106: ~0.10% 514902: ~0.10% 515053: ~0.10% 515507: ~0.10% 516205: ~0.10% 519323: ~0.10% 520796: ~0.10% 521492: ~0.10% 522112: ~0.10% 523814: ~0.10% 525505: ~0.10% 525583: ~0.10% 525764: ~0.10% 528105: ~0.10% 529692: ~0.10% 532014: ~0.10% 534952: ~0.10% 538836: ~0.10% 539326: ~0.10% 539504: ~0.10% 541861: ~0.10% 542925: ~0.10% 544853: ~0.10% 545091: ~0.10% 545394: ~0.10% 546521: ~0.10% 546527: ~0.10% 548007: ~0.10% 548100: ~0.10% 550372: ~0.10% 554548: ~0.10% 560711: ~0.10% 560886: ~0.10% 561084: ~0.10% 561114: ~0.10% 561329: ~0.10% 561838: ~0.10% 561946: ~0.10% 562174: ~0.10% 562185: ~0.10% 564894: ~0.10% 566373: ~0.10% 566884: ~0.10% 568110: ~0.10% 569541: ~0.10% 570042: ~0.10% 571286: ~0.10% 571515: ~0.10% 571577: ~0.10% 572354: ~0.10% 573015: ~0.10% 573283: ~0.10% 578249: ~0.10% 578786: ~0.10% 580872: ~0.10% 581072: ~0.10% 581684: ~0.10% 582341: ~0.10% 583169: ~0.10% 583225: ~0.10% 583322: ~0.10% 583889: ~0.10% 584173: ~0.10% 585523: ~0.10% 585660: ~0.10% 587005: ~0.10% 588010: ~0.10% 588337: ~0.10% 590946: ~0.10% 593319: ~0.10% 594039: ~0.10% 594452: ~0.10% 595246: ~0.10% 597157: ~0.10% 597215: ~0.10% 597368: ~0.10% 597453: ~0.10% 598538: ~0.10% 598540: ~0.10% 601120: ~0.10% 604762: ~0.10% 606244: ~0.10% 606935: ~0.10% 609731: ~0.10% 611702: ~0.10% 613040: ~0.10% 614720: ~0.10% 615525: ~0.10% 616416: ~0.10% 618280: ~0.10% 618322: ~0.10% 619151: ~0.10% 619170: ~0.10% 622593: ~0.10% 622755: ~0.10% 623529: ~0.10% 625333: ~0.10% 625780: ~0.10% 626317: ~0.10% 626670: ~0.10% 628510: ~0.10% 630995: ~0.10% 631618: ~0.10% 632001: ~0.10% 634750: ~0.10% 635195: ~0.10% 636256: ~0.10% 636542: ~0.10% 637420: ~0.10% 641046: ~0.10% 643232: ~0.10% 643901: ~0.10% 644517: ~0.10% 645962: ~0.10% 647293: ~0.10% 647443: ~0.10% 648173: ~0.10% 649204: ~0.10% 649243: ~0.10% 650521: ~0.10% 650979: ~0.10% 652493: ~0.10% 654771: ~0.10% 656535: ~0.10% 658715: ~0.10% 659035: ~0.10% 659593: ~0.10% 660535: ~0.10% 662154: ~0.10% 662784: ~0.10% 663142: ~0.10% 664964: ~0.10% 665230: ~0.10% 666319: ~0.10% 666386: ~0.10% 666561: ~0.10% 668060: ~0.10% 668151: ~0.10% 669382: ~0.10% 670341: ~0.10% 673081: ~0.10% 673634: ~0.10% 673875: ~0.10% 673881: ~0.10% 674082: ~0.10% 674852: ~0.10% 675319: ~0.10% 675492: ~0.10% 676147: ~0.10% 676238: ~0.10% 676318: ~0.10% 677459: ~0.10% 677515: ~0.10% 679216: ~0.10% 679307: ~0.10% 680354: ~0.10% 681098: ~0.10% 681873: ~0.10% 683933: ~0.10% 684800: ~0.10% 685690: ~0.10% 685871: ~0.10% 686886: ~0.10% 689687: ~0.10% 689748: ~0.10% 693501: ~0.10% 694425: ~0.10% 694466: ~0.10% 695798: ~0.10% 698130: ~0.10% 700276: ~0.10% 702137: ~0.10% 703138: ~0.10% 704067: ~0.10% 704460: ~0.10% 706199: ~0.10% 706878: ~0.10% 707280: ~0.10% 710580: ~0.10% 710968: ~0.10% 713539: ~0.10% 713584: ~0.10% 714733: ~0.10% 718172: ~0.10% 719545: ~0.10% 720471: ~0.10% 720690: ~0.10% 723568: ~0.10% 724334: ~0.10% 724700: ~0.10% 727908: ~0.10% 730499: ~0.10% 730711: ~0.10% 733963: ~0.10% 734912: ~0.10% 736431: ~0.10% 738012: ~0.10% 742066: ~0.10% 745799: ~0.10% 746161: ~0.10% 748392: ~0.10% 748838: ~0.10% 749148: ~0.10% 751762: ~0.10% 752092: ~0.10% 752527: ~0.10% 753004: ~0.10% 755386: ~0.10% 756558: ~0.10% 756736: ~0.10% 758706: ~0.10% 759523: ~0.10% 760550: ~0.10% 762688: ~0.10% 762918: ~0.10% 765769: ~0.10% 766354: ~0.10% 768119: ~0.10% 769970: ~0.10% 773106: ~0.10% 775964: ~0.10% 776055: ~0.10% 777088: ~0.10% 777529: ~0.10% 779273: ~0.10% 781066: ~0.10% 781136: ~0.10% 782328: ~0.10% 783231: ~0.10% 784413: ~0.10% 785781: ~0.10% 786250: ~0.10% 786845: ~0.10% 788012: ~0.10% 792788: ~0.10% 793182: ~0.10% 794187: ~0.10% 796097: ~0.10% 796117: ~0.10% 797182: ~0.10% 798112: ~0.10% 798215: ~0.10% 802050: ~0.10% 802669: ~0.10% 804168: ~0.10% 804461: ~0.10% 805743: ~0.10% 806286: ~0.10% 808416: ~0.10% 808455: ~0.10% 810577: ~0.10% 811702: ~0.10% 811843: ~0.10% 812439: ~0.10% 815923: ~0.10% 816475: ~0.10% 818312: ~0.10% 818367: ~0.10% 818521: ~0.10% 819278: ~0.10% 819972: ~0.10% 820890: ~0.10% 821188: ~0.10% 821615: ~0.10% 822376: ~0.10% 823136: ~0.10% 823735: ~0.10% 829476: ~0.10% 830591: ~0.10% 832433: ~0.10% 832597: ~0.10% 835043: ~0.10% 835759: ~0.10% 837731: ~0.10% 837942: ~0.10% 839448: ~0.10% 840417: ~0.10% 843622: ~0.10% 844870: ~0.10% 846084: ~0.10% 846807: ~0.10% 847076: ~0.10% 847535: ~0.10% 847766: ~0.10% 847977: ~0.10% 848075: ~0.10% 848326: ~0.10% 852725: ~0.10% 853465: ~0.10% 856427: ~0.10% 857186: ~0.10% 858543: ~0.10% 858830: ~0.10% 861596: ~0.10% 863804: ~0.10% 865139: ~0.10% 866039: ~0.10% 866406: ~0.10% 867180: ~0.10% 868280: ~0.10% 872156: ~0.10% 872791: ~0.10% 872953: ~0.10% 872959: ~0.10% 875015: ~0.10% 876522: ~0.10% 878407: ~0.10% 878710: ~0.10% 878855: ~0.10% 880495: ~0.10% 884335: ~0.10% 884941: ~0.10% 885893: ~0.10% 886713: ~0.10% 887068: ~0.10% 887115: ~0.10% 887220: ~0.10% 887751: ~0.10% 888027: ~0.10% 888069: ~0.10% 888758: ~0.10% 890152: ~0.10% 891137: ~0.10% 891766: ~0.10% 891890: ~0.10% 892662: ~0.10% 892973: ~0.10% 893360: ~0.10% 893915: ~0.10% 893976: ~0.10% 894324: ~0.10% 895709: ~0.10% 897065: ~0.10% 898387: ~0.10% 898554: ~0.10% 899291: ~0.10% 899604: ~0.10% 900513: ~0.10% 900619: ~0.10% 901170: ~0.10% 903238: ~0.10% 904294: ~0.10% 904520: ~0.10% 904666: ~0.10% 904992: ~0.10% 907212: ~0.10% 908062: ~0.10% 908561: ~0.10% 909741: ~0.10% 911034: ~0.10% 911982: ~0.10% 913716: ~0.10% 914819: ~0.10% 915750: ~0.10% 915766: ~0.10% 916125: ~0.10% 916648: ~0.10% 917285: ~0.10% 918194: ~0.10% 920583: ~0.10% 924616: ~0.10% 926035: ~0.10% 927726: ~0.10% 929821: ~0.10% 930300: ~0.10% 930796: ~0.10% 931617: ~0.10% 932719: ~0.10% 933784: ~0.10% 934378: ~0.10% 935900: ~0.10% 936118: ~0.10% 936336: ~0.10% 937231: ~0.10% 938420: ~0.10% 939184: ~0.10% 939567: ~0.10% 941588: ~0.10% 944093: ~0.10% 944912: ~0.10% 945069: ~0.10% 945659: ~0.10% 946110: ~0.10% 950044: ~0.10% 954101: ~0.10% 954147: ~0.10% 958697: ~0.10% 959530: ~0.10% 961721: ~0.10% 963582: ~0.10% 964471: ~0.10% 965026: ~0.10% 966573: ~0.10% 967330: ~0.10% 968176: ~0.10% 968346: ~0.10% 970649: ~0.10% 970873: ~0.10% 971636: ~0.10% 971664: ~0.10% 972336: ~0.10% 973555: ~0.10% 974207: ~0.10% 976896: ~0.10% 981402: ~0.10% 983723: ~0.10% 984653: ~0.10% 985161: ~0.10% 987048: ~0.10% 987107: ~0.10% 987815: ~0.10% 993209: ~0.10% 994360: ~0.10% 994613: ~0.10% 1004305: ~0.10% 1004650: ~0.10% 1004849: ~0.10% 1005118: ~0.10% 1005614: ~0.10% 1005626: ~0.10% 1006835: ~0.10% 1009556: ~0.10% 1011008: ~0.10% 1014010: ~0.10% 1014030: ~0.10% 1016549: ~0.10% 1017016: ~0.10% 1017335: ~0.10% 1018386: ~0.10% 1020640: ~0.10% 1021041: ~0.10% 1021411: ~0.10% 1022998: ~0.10% 1025423: ~0.10% 1025767: ~0.10% 1026066: ~0.10% 1026434: ~0.10% 1027516: ~0.10% 1027703: ~0.10% 1028119: ~0.10% 1028642: ~0.10% 1031554: ~0.10% 1033639: ~0.10% 1033660: ~0.10% 1034832: ~0.10% 1035274: ~0.10% 1037432: ~0.10% 1037536: ~0.10% 1037759: ~0.10% 1039860: ~0.10% 1041131: ~0.10% 1041892: ~0.10% 1043066: ~0.10% 1044326: ~0.10% 1044905: ~0.10% 1047848: ~0.10% 1048062: ~0.10% 1048534: ~0.10% 1049477: ~0.10% 1049533: ~0.10% 1050531: ~0.10% 1050937: ~0.10% 1051747: ~0.10% 1052073: ~0.10% 1052617: ~0.10% 1054049: ~0.10% 1055142: ~0.10% 1056933: ~0.10% 1057358: ~0.10% 1057911: ~0.10% 1061411: ~0.10% 1062328: ~0.10% 1062485: ~0.10% 1062534: ~0.10% 1062794: ~0.10% 1063269: ~0.10% 1063467: ~0.10% 1064568: ~0.10% 1064868: ~0.10% 1065180: ~0.10% 1065481: ~0.10% 1067970: ~0.10% 1068479: ~0.10% 1070203: ~0.10% 1070708: ~0.10% 1071075: ~0.10% 1072038: ~0.10% 1072214: ~0.10% 1072525: ~0.10% 1073953: ~0.10% 1074885: ~0.10% 1075308: ~0.10% 1078872: ~0.10% 1079266: ~0.10% 1079736: ~0.10% 1080075: ~0.10% 1081716: ~0.10% 1137391: ~0.10% 1138530: ~0.10% 1139697: ~0.10% 1139738: ~0.10% 1140119: ~0.10% 1140869: ~0.10% 1141527: ~0.10% 1144693: ~0.10% 1145425: ~0.10% 1149162: ~0.10% 1149207: ~0.10% 1149985: ~0.10% 1150086: ~0.10% 1150731: ~0.10% 1151256: ~0.10% 1151403: ~0.10% 1152236: ~0.10% 1153693: ~0.10% 1154576: ~0.10% 1155859: ~0.10% 1156918: ~0.10% 1158007: ~0.10% 1158559: ~0.10% 1158952: ~0.10% 1159165: ~0.10% 1161242: ~0.10% 1163227: ~0.10% 1164078: ~0.10% 1165251: ~0.10% 1166023: ~0.10% 1166231: ~0.10% 1167002: ~0.10% 1168584: ~0.10% 1169844: ~0.10% 1170663: ~0.10% 1171580: ~0.10% 1172072: ~0.10% 1173371: ~0.10% 1173809: ~0.10% 1174049: ~0.10% 1175044: ~0.10% 1175745: ~0.10% 1176061: ~0.10% 1176414: ~0.10% 1177449: ~0.10% 1178311: ~0.10% 1179069: ~0.10% 1179450: ~0.10% 1181077: ~0.10% 1183293: ~0.10% 1184313: ~0.10% 1185090: ~0.10% 1185669: ~0.10%	size: 32 elements	size: 32 elements

• Samples:

  query_id	document_ids	scores
  237784	[6366584, 4034101, 2325374, 6914618, 6042146, ...]	[0.9999999991784339, 0.42233632827946693, 0.5956354295491569, 0.12644415907455164, 0.6636713730105909, ...]
  904294	[448408, 8743975, 49600, 7339401, 2714261, ...]	[0.9999999991841937, 0.877629062381539, 0.8330146583389045, 0.3116634796692611, 0.4633524534142185, ...]
  412214	[1006306, 4454048, 1949661, 4895656, 675880, ...]	[0.9999999994734676, 0.38790621123137803, 0.3747429039573546, 0.2990538871317199, 0.38420403106055895, ...]

• Loss: pylate.losses.distillation.Distillation

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: steps
• per_device_train_batch_size: 16
• learning_rate: 3e-05
• num_train_epochs: 1
• bf16: True

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: steps
• prediction_loss_only: True
• per_device_train_batch_size: 16
• per_device_eval_batch_size: 8
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 3e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1.0
• num_train_epochs: 1
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.0
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: True
• fp16: False
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: False
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• dispatch_batches: None
• split_batches: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• eval_use_gather_object: False
• average_tokens_across_devices: False
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: proportional

Training Logs

Click to expand

Epoch	Step	Training Loss	NanoClimateFEVER_MaxSim_ndcg@10	NanoDBPedia_MaxSim_ndcg@10	NanoFEVER_MaxSim_ndcg@10	NanoFiQA2018_MaxSim_ndcg@10	NanoHotpotQA_MaxSim_ndcg@10	NanoMSMARCO_MaxSim_ndcg@10	NanoNFCorpus_MaxSim_ndcg@10	NanoNQ_MaxSim_ndcg@10	NanoQuoraRetrieval_MaxSim_ndcg@10	NanoSCIDOCS_MaxSim_ndcg@10	NanoArguAna_MaxSim_ndcg@10	NanoSciFact_MaxSim_ndcg@10	NanoTouche2020_MaxSim_ndcg@10	NanoBEIR_mean_MaxSim_ndcg@10
0.0030	100	0.0383	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.0060	200	0.0328	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.0090	300	0.0319	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.0120	400	0.0313	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.0150	500	0.0294	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.0180	600	0.0265	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.0210	700	0.026	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.0240	800	0.0251	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.0270	900	0.0242	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.0300	1000	0.0245	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.0330	1100	0.0232	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.0360	1200	0.0236	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.0390	1300	0.0231	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.0420	1400	0.0227	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.0450	1500	0.0225	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.0480	1600	0.0222	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.0510	1700	0.0218	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.0540	1800	0.022	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.0570	1900	0.0213	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.0600	2000	0.0214	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.0630	2100	0.0214	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.0660	2200	0.0209	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.0690	2300	0.0204	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.0720	2400	0.0201	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.0750	2500	0.02	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.0780	2600	0.0207	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.0810	2700	0.0199	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.0840	2800	0.0198	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.0870	2900	0.0196	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.0900	3000	0.0197	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.0930	3100	0.0194	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.0960	3200	0.0193	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.0990	3300	0.019	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.1020	3400	0.0187	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.1050	3500	0.0187	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.1080	3600	0.0187	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.1110	3700	0.0183	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.1140	3800	0.0185	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.1170	3900	0.0186	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.1200	4000	0.0188	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.1230	4100	0.019	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.1260	4200	0.018	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.1290	4300	0.0183	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.1320	4400	0.0178	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.1350	4500	0.018	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.1380	4600	0.0175	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.1410	4700	0.0172	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.1440	4800	0.0174	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.1470	4900	0.0176	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.1500	5000	0.017	0.2687	0.5353	0.8792	0.4640	0.8518	0.6460	0.3198	0.6059	0.9229	0.3192	0.3772	0.7166	0.5900	0.5767
0.1530	5100	0.0173	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.1560	5200	0.0175	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.1590	5300	0.0173	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.1620	5400	0.0168	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.1650	5500	0.0164	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.1680	5600	0.0171	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.1710	5700	0.0169	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.1740	5800	0.0166	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.1770	5900	0.0169	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.1801	6000	0.0166	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.1831	6100	0.0164	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.1861	6200	0.0165	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.1891	6300	0.0162	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.1921	6400	0.0161	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.1951	6500	0.0163	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.1981	6600	0.0161	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.2011	6700	0.0161	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.2041	6800	0.0159	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.2071	6900	0.0166	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.2101	7000	0.0158	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.2131	7100	0.0161	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.2161	7200	0.0154	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.2191	7300	0.0151	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.2221	7400	0.0155	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.2251	7500	0.0154	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.2281	7600	0.0157	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.2311	7700	0.0154	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.2341	7800	0.0153	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.2371	7900	0.0155	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.2401	8000	0.015	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.2431	8100	0.0155	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.2461	8200	0.0153	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.2491	8300	0.015	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.2521	8400	0.0149	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.2551	8500	0.0151	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.2581	8600	0.015	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.2611	8700	0.0151	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.2641	8800	0.0152	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.2671	8900	0.0152	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.2701	9000	0.0149	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.2731	9100	0.0145	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.2761	9200	0.0145	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.2791	9300	0.0147	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.2821	9400	0.0151	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.2851	9500	0.0144	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.2881	9600	0.0144	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.2911	9700	0.0147	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.2941	9800	0.0147	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.2971	9900	0.0146	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.3001	10000	0.0144	0.2775	0.5458	0.8872	0.4447	0.8686	0.6438	0.3179	0.6183	0.9151	0.3025	0.4000	0.7288	0.5733	0.5787
0.3031	10100	0.0139	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.3061	10200	0.0141	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.3091	10300	0.0147	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.3121	10400	0.0146	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.3151	10500	0.0143	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.3181	10600	0.0144	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.3211	10700	0.0141	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.3241	10800	0.0147	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.3271	10900	0.0142	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.3301	11000	0.0139	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.3331	11100	0.0142	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.3361	11200	0.0139	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.3391	11300	0.0143	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.3421	11400	0.0142	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.3451	11500	0.0144	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.3481	11600	0.0142	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.3511	11700	0.0137	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.3541	11800	0.0139	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.3571	11900	0.0142	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.3601	12000	0.0136	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.3631	12100	0.0139	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.3661	12200	0.0144	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.3691	12300	0.0136	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.3721	12400	0.0141	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.3751	12500	0.0137	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.3781	12600	0.0136	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.3811	12700	0.0137	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.3841	12800	0.0132	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.3871	12900	0.0136	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.3901	13000	0.0138	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.3931	13100	0.0138	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.3961	13200	0.0138	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.3991	13300	0.0137	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.4021	13400	0.0131	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.4051	13500	0.0136	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.4081	13600	0.0135	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.4111	13700	0.0138	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.4141	13800	0.0135	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.4171	13900	0.0138	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.4201	14000	0.0132	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.4231	14100	0.0136	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.4261	14200	0.0131	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.4291	14300	0.0135	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.4321	14400	0.0138	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.4351	14500	0.013	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.4381	14600	0.0131	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.4411	14700	0.013	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.4441	14800	0.0134	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.4471	14900	0.0133	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.4501	15000	0.0136	0.2846	0.5781	0.8899	0.4652	0.8629	0.6491	0.3255	0.6303	0.9427	0.3168	0.4224	0.7354	0.5816	0.5911
0.4531	15100	0.0133	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.4561	15200	0.0131	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.4591	15300	0.0133	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.4621	15400	0.0129	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.4651	15500	0.013	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.4681	15600	0.0133	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.4711	15700	0.0132	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.4741	15800	0.013	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.4771	15900	0.0132	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.4801	16000	0.0129	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.4831	16100	0.013	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.4861	16200	0.0128	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.4891	16300	0.013	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.4921	16400	0.0135	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.4951	16500	0.0126	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.4981	16600	0.0127	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.5011	16700	0.0128	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.5041	16800	0.0125	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.5071	16900	0.0128	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.5101	17000	0.0129	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.5131	17100	0.0128	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.5161	17200	0.0128	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.5191	17300	0.013	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.5221	17400	0.0126	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.5251	17500	0.0128	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.5281	17600	0.0128	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.5311	17700	0.013	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.5341	17800	0.0127	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.5372	17900	0.0127	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.5402	18000	0.0124	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.5432	18100	0.0125	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.5462	18200	0.0125	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.5492	18300	0.0128	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.5522	18400	0.0125	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.5552	18500	0.0125	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.5582	18600	0.0127	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.5612	18700	0.0124	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.5642	18800	0.0127	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.5672	18900	0.0121	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.5702	19000	0.0123	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.5732	19100	0.012	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.5762	19200	0.0122	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.5792	19300	0.0123	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.5822	19400	0.0126	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.5852	19500	0.0127	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.5882	19600	0.0124	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.5912	19700	0.0127	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.5942	19800	0.0125	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.5972	19900	0.0124	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.6002	20000	0.0124	0.2803	0.5772	0.8907	0.4668	0.8676	0.6476	0.3364	0.6633	0.9129	0.3240	0.4412	0.7460	0.5781	0.5948
0.6032	20100	0.0123	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.6062	20200	0.0122	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.6092	20300	0.0125	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.6122	20400	0.0124	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.6152	20500	0.0122	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.6182	20600	0.0123	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.6212	20700	0.0123	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.6242	20800	0.0122	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.6272	20900	0.012	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.6302	21000	0.0123	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.6332	21100	0.0121	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.6362	21200	0.0121	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.6392	21300	0.0124	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.6422	21400	0.0123	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.6452	21500	0.012	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.6482	21600	0.0123	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.6512	21700	0.0122	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.6542	21800	0.012	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.6572	21900	0.0121	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.6602	22000	0.0124	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.6632	22100	0.0124	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.6662	22200	0.0119	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.6692	22300	0.0117	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.6722	22400	0.0118	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.6752	22500	0.0122	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.6782	22600	0.0116	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.6812	22700	0.0122	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.6842	22800	0.0122	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.6872	22900	0.0119	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.6902	23000	0.0117	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.6932	23100	0.012	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.6962	23200	0.012	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.6992	23300	0.012	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.7022	23400	0.0119	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.7052	23500	0.0117	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.7082	23600	0.0119	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.7112	23700	0.0121	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.7142	23800	0.0118	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.7172	23900	0.0121	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.7202	24000	0.0123	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.7232	24100	0.0121	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.7262	24200	0.0115	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.7292	24300	0.0117	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.7322	24400	0.0116	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.7352	24500	0.0116	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.7382	24600	0.0117	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.7412	24700	0.0119	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.7442	24800	0.0117	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.7472	24900	0.0118	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.7502	25000	0.012	0.2905	0.5880	0.9048	0.4640	0.8575	0.6645	0.3255	0.6545	0.9043	0.3302	0.4645	0.7485	0.5655	0.5971

Framework Versions

• Python: 3.11.13
• Sentence Transformers: 4.0.2
• PyLate: 1.2.0
• Transformers: 4.48.2
• PyTorch: 2.6.0+cu124
• Accelerate: 1.9.0
• Datasets: 4.0.0
• Tokenizers: 0.21.4

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084"
}

PyLate

@misc{PyLate,
title={PyLate: Flexible Training and Retrieval for Late Interaction Models},
author={Chaffin, Antoine and Sourty, Raphaël},
url={https://github.com/lightonai/pylate},
year={2024}
}