ahycl/ettin-not_cleaned-data-1-epoch-final

PyLate model based on jhu-clsp/ettin-encoder-17m

This is a PyLate model finetuned from jhu-clsp/ettin-encoder-17m on the ms-marco-en-bge-gemma dataset. It maps sentences & paragraphs to sequences of 128-dimensional dense vectors and can be used for semantic textual similarity using the MaxSim operator.

Model Details

Model Description

• Model Type: PyLate model
• Base model: jhu-clsp/ettin-encoder-17m
• Document Length: 300 tokens
• Query Length: 32 tokens
• Output Dimensionality: 128 tokens
• Similarity Function: MaxSim
• Training Dataset:
  • ms-marco-en-bge-gemma
• Language: en

Model Sources

• Documentation: PyLate Documentation
• Repository: PyLate on GitHub
• Hugging Face: PyLate models on Hugging Face

Full Model Architecture

ColBERT(
  (0): Transformer({'max_seq_length': 299, 'do_lower_case': False}) with Transformer model: ModernBertModel 
  (1): Dense({'in_features': 256, 'out_features': 128, 'bias': False, 'activation_function': 'torch.nn.modules.linear.Identity'})
)

Usage

First install the PyLate library:

pip install -U pylate

Retrieval

PyLate provides a streamlined interface to index and retrieve documents using ColBERT models. The index leverages the Voyager HNSW index to efficiently handle document embeddings and enable fast retrieval.

Indexing documents

First, load the ColBERT model and initialize the Voyager index, then encode and index your documents:

from pylate import indexes, models, retrieve

# Step 1: Load the ColBERT model
model = models.ColBERT(
    model_name_or_path=pylate_model_id,
)

# Step 2: Initialize the Voyager index
index = indexes.Voyager(
    index_folder="pylate-index",
    index_name="index",
    override=True,  # This overwrites the existing index if any
)

# Step 3: Encode the documents
documents_ids = ["1", "2", "3"]
documents = ["document 1 text", "document 2 text", "document 3 text"]

documents_embeddings = model.encode(
    documents,
    batch_size=32,
    is_query=False,  # Ensure that it is set to False to indicate that these are documents, not queries
    show_progress_bar=True,
)

# Step 4: Add document embeddings to the index by providing embeddings and corresponding ids
index.add_documents(
    documents_ids=documents_ids,
    documents_embeddings=documents_embeddings,
)

Note that you do not have to recreate the index and encode the documents every time. Once you have created an index and added the documents, you can re-use the index later by loading it:

# To load an index, simply instantiate it with the correct folder/name and without overriding it
index = indexes.Voyager(
    index_folder="pylate-index",
    index_name="index",
)

Retrieving top-k documents for queries

Once the documents are indexed, you can retrieve the top-k most relevant documents for a given set of queries. To do so, initialize the ColBERT retriever with the index you want to search in, encode the queries and then retrieve the top-k documents to get the top matches ids and relevance scores:

# Step 1: Initialize the ColBERT retriever
retriever = retrieve.ColBERT(index=index)

# Step 2: Encode the queries
queries_embeddings = model.encode(
    ["query for document 3", "query for document 1"],
    batch_size=32,
    is_query=True,  #  # Ensure that it is set to False to indicate that these are queries
    show_progress_bar=True,
)

# Step 3: Retrieve top-k documents
scores = retriever.retrieve(
    queries_embeddings=queries_embeddings,
    k=10,  # Retrieve the top 10 matches for each query
)

Reranking

If you only want to use the ColBERT model to perform reranking on top of your first-stage retrieval pipeline without building an index, you can simply use rank function and pass the queries and documents to rerank:

from pylate import rank, models

queries = [
    "query A",
    "query B",
]

documents = [
    ["document A", "document B"],
    ["document 1", "document C", "document B"],
]

documents_ids = [
    [1, 2],
    [1, 3, 2],
]

model = models.ColBERT(
    model_name_or_path=pylate_model_id,
)

queries_embeddings = model.encode(
    queries,
    is_query=True,
)

documents_embeddings = model.encode(
    documents,
    is_query=False,
)

reranked_documents = rank.rerank(
    documents_ids=documents_ids,
    queries_embeddings=queries_embeddings,
    documents_embeddings=documents_embeddings,
)

Evaluation

Metrics

Py Late Information Retrieval

• Dataset: ['NanoClimateFEVER', 'NanoDBPedia', 'NanoFEVER', 'NanoFiQA2018', 'NanoHotpotQA', 'NanoMSMARCO', 'NanoNFCorpus', 'NanoNQ', 'NanoQuoraRetrieval', 'NanoSCIDOCS', 'NanoArguAna', 'NanoSciFact', 'NanoTouche2020']
• Evaluated with pylate.evaluation.pylate_information_retrieval_evaluator.PyLateInformationRetrievalEvaluator

Metric	NanoClimateFEVER	NanoDBPedia	NanoFEVER	NanoFiQA2018	NanoHotpotQA	NanoMSMARCO	NanoNFCorpus	NanoNQ	NanoQuoraRetrieval	NanoSCIDOCS	NanoArguAna	NanoSciFact	NanoTouche2020
MaxSim_accuracy@1	0.26	0.72	0.84	0.42	0.92	0.5	0.4	0.54	0.82	0.42	0.2	0.6	0.6939
MaxSim_accuracy@3	0.44	0.86	0.94	0.6	0.98	0.66	0.52	0.76	0.96	0.6	0.48	0.76	0.9184
MaxSim_accuracy@5	0.48	0.9	0.96	0.72	1.0	0.68	0.58	0.8	1.0	0.66	0.64	0.82	0.9592
MaxSim_accuracy@10	0.72	0.94	0.98	0.76	1.0	0.78	0.68	0.84	1.0	0.74	0.76	0.88	1.0
MaxSim_precision@1	0.26	0.72	0.84	0.42	0.92	0.5	0.4	0.54	0.82	0.42	0.2	0.6	0.6939
MaxSim_precision@3	0.1733	0.6067	0.3333	0.2667	0.54	0.22	0.34	0.2533	0.3867	0.2867	0.16	0.2667	0.6395
MaxSim_precision@5	0.12	0.572	0.208	0.228	0.34	0.136	0.32	0.168	0.244	0.224	0.128	0.18	0.6367
MaxSim_precision@10	0.102	0.49	0.106	0.14	0.18	0.078	0.28	0.09	0.128	0.15	0.076	0.098	0.4939
MaxSim_recall@1	0.12	0.0812	0.7767	0.2159	0.46	0.5	0.0335	0.51	0.734	0.0867	0.2	0.575	0.0494
MaxSim_recall@3	0.23	0.164	0.9033	0.3558	0.81	0.66	0.0771	0.7	0.912	0.1767	0.48	0.74	0.1304
MaxSim_recall@5	0.2567	0.2287	0.93	0.4973	0.85	0.68	0.0957	0.76	0.956	0.2287	0.64	0.815	0.2136
MaxSim_recall@10	0.4	0.3339	0.95	0.584	0.9	0.78	0.144	0.81	0.9727	0.3067	0.76	0.87	0.3156
MaxSim_ndcg@10	0.3059	0.5944	0.8864	0.4729	0.8634	0.6351	0.3302	0.6699	0.9086	0.3142	0.4545	0.7357	0.5613
MaxSim_mrr@10	0.3818	0.7956	0.8903	0.5344	0.9507	0.5894	0.4766	0.6447	0.9	0.5304	0.3582	0.6952	0.8086
MaxSim_map@100	0.2305	0.4761	0.8591	0.3849	0.8069	0.6008	0.1419	0.6201	0.8814	0.2477	0.3653	0.6912	0.4155

Nano BEIR

• Dataset: NanoBEIR_mean
• Evaluated with pylate.evaluation.nano_beir_evaluator.NanoBEIREvaluator

Metric	Value
MaxSim_accuracy@1	0.5641
MaxSim_accuracy@3	0.7291
MaxSim_accuracy@5	0.7846
MaxSim_accuracy@10	0.8523
MaxSim_precision@1	0.5641
MaxSim_precision@3	0.3441
MaxSim_precision@5	0.2696
MaxSim_precision@10	0.1855
MaxSim_recall@1	0.334
MaxSim_recall@3	0.4876
MaxSim_recall@5	0.5501
MaxSim_recall@10	0.6251
MaxSim_ndcg@10	0.5948
MaxSim_mrr@10	0.6581
MaxSim_map@100	0.517

Training Details

Training Dataset

ms-marco-en-bge-gemma

• Dataset: ms-marco-en-bge-gemma at d8bad49
• Size: 533,177 training samples
• Columns: query_id, document_ids, and scores
• Approximate statistics based on the first 1000 samples:

  	query_id	document_ids	scores
  type	int	list	list
  details	836: ~0.10% 3582: ~0.10% 4599: ~0.10% 4645: ~0.10% 4853: ~0.10% 5154: ~0.10% 7504: ~0.10% 12283: ~0.10% 12335: ~0.10% 12916: ~0.10% 14049: ~0.10% 14828: ~0.10% 15674: ~0.10% 15813: ~0.10% 16728: ~0.10% 22006: ~0.10% 23675: ~0.10% 24199: ~0.10% 25323: ~0.10% 28517: ~0.10% 29213: ~0.10% 32344: ~0.10% 34071: ~0.10% 34604: ~0.10% 35424: ~0.10% 35445: ~0.10% 36148: ~0.10% 37078: ~0.10% 37826: ~0.10% 38185: ~0.10% 40855: ~0.10% 42077: ~0.10% 43614: ~0.10% 45073: ~0.10% 46289: ~0.10% 47507: ~0.10% 48005: ~0.10% 48629: ~0.10% 48785: ~0.10% 49216: ~0.10% 49636: ~0.10% 49970: ~0.10% 52075: ~0.10% 52725: ~0.10% 54142: ~0.10% 54210: ~0.10% 55032: ~0.10% 59546: ~0.10% 60087: ~0.10% 60862: ~0.10% 60941: ~0.10% 61037: ~0.10% 61762: ~0.10% 62649: ~0.10% 63333: ~0.10% 64197: ~0.10% 64879: ~0.10% 67608: ~0.10% 67627: ~0.10% 69463: ~0.10% 70002: ~0.10% 70429: ~0.10% 72166: ~0.10% 72518: ~0.10% 72607: ~0.10% 72791: ~0.10% 73325: ~0.10% 74078: ~0.10% 74857: ~0.10% 75323: ~0.10% 75816: ~0.10% 76929: ~0.10% 77845: ~0.10% 77889: ~0.10% 78077: ~0.10% 78256: ~0.10% 78401: ~0.10% 78798: ~0.10% 80871: ~0.10% 81089: ~0.10% 82179: ~0.10% 82883: ~0.10% 84168: ~0.10% 86891: ~0.10% 88282: ~0.10% 89346: ~0.10% 89386: ~0.10% 90699: ~0.10% 90795: ~0.10% 91367: ~0.10% 91795: ~0.10% 92070: ~0.10% 92523: ~0.10% 92597: ~0.10% 92753: ~0.10% 92787: ~0.10% 96382: ~0.10% 96455: ~0.10% 97274: ~0.10% 97603: ~0.10% 100904: ~0.10% 101205: ~0.10% 101305: ~0.10% 102707: ~0.10% 103074: ~0.10% 105437: ~0.10% 108207: ~0.10% 109776: ~0.10% 112056: ~0.10% 112955: ~0.10% 112977: ~0.10% 113635: ~0.10% 115280: ~0.10% 115551: ~0.10% 116098: ~0.10% 117658: ~0.10% 120255: ~0.10% 120298: ~0.10% 121437: ~0.10% 123429: ~0.10% 125043: ~0.10% 125979: ~0.10% 126851: ~0.10% 128218: ~0.10% 128804: ~0.10% 129598: ~0.10% 131299: ~0.10% 132114: ~0.10% 133696: ~0.10% 134460: ~0.10% 137602: ~0.10% 137679: ~0.10% 138121: ~0.10% 138260: ~0.10% 138823: ~0.10% 139039: ~0.10% 140392: ~0.10% 140651: ~0.10% 142305: ~0.10% 145653: ~0.10% 145683: ~0.10% 145763: ~0.10% 150202: ~0.10% 151135: ~0.10% 152307: ~0.10% 152675: ~0.10% 153693: ~0.10% 154470: ~0.10% 155587: ~0.10% 157602: ~0.10% 157779: ~0.10% 158565: ~0.10% 159177: ~0.10% 159224: ~0.10% 159341: ~0.10% 159892: ~0.10% 161881: ~0.10% 162414: ~0.10% 163765: ~0.10% 165888: ~0.10% 168048: ~0.10% 168425: ~0.10% 168894: ~0.10% 169991: ~0.10% 170731: ~0.10% 171705: ~0.10% 176165: ~0.10% 176798: ~0.10% 180259: ~0.10% 181243: ~0.10% 182102: ~0.10% 182660: ~0.10% 183426: ~0.10% 183930: ~0.10% 184045: ~0.10% 184676: ~0.10% 185294: ~0.10% 186475: ~0.10% 187155: ~0.10% 188198: ~0.10% 191383: ~0.10% 192165: ~0.10% 193507: ~0.10% 194207: ~0.10% 195056: ~0.10% 197377: ~0.10% 198224: ~0.10% 198546: ~0.10% 202122: ~0.10% 203519: ~0.10% 206220: ~0.10% 209739: ~0.10% 210554: ~0.10% 212638: ~0.10% 213096: ~0.10% 213410: ~0.10% 214255: ~0.10% 217541: ~0.10% 219718: ~0.10% 220993: ~0.10% 223241: ~0.10% 224657: ~0.10% 227101: ~0.10% 227497: ~0.10% 227726: ~0.10% 228099: ~0.10% 228451: ~0.10% 230413: ~0.10% 231416: ~0.10% 233312: ~0.10% 234348: ~0.10% 235869: ~0.10% 237784: ~0.10% 240739: ~0.10% 246495: ~0.10% 246821: ~0.10% 248675: ~0.10% 249798: ~0.10% 249962: ~0.10% 249977: ~0.10% 250019: ~0.10% 250548: ~0.10% 251089: ~0.10% 254878: ~0.10% 255183: ~0.10% 255727: ~0.10% 256321: ~0.10% 258276: ~0.10% 260993: ~0.10% 261247: ~0.10% 262123: ~0.10% 262508: ~0.10% 266047: ~0.10% 267089: ~0.10% 267192: ~0.10% 268642: ~0.10% 269025: ~0.10% 273171: ~0.10% 273864: ~0.10% 274521: ~0.10% 274586: ~0.10% 275037: ~0.10% 275643: ~0.10% 276744: ~0.10% 277212: ~0.10% 277990: ~0.10% 279931: ~0.10% 280012: ~0.10% 281699: ~0.10% 282128: ~0.10% 283298: ~0.10% 284268: ~0.10% 285697: ~0.10% 285905: ~0.10% 287456: ~0.10% 287506: ~0.10% 288154: ~0.10% 289046: ~0.10% 292211: ~0.10% 292588: ~0.10% 293357: ~0.10% 293661: ~0.10% 294123: ~0.10% 299287: ~0.10% 300622: ~0.10% 302135: ~0.10% 303224: ~0.10% 304353: ~0.10% 304820: ~0.10% 310215: ~0.10% 310236: ~0.10% 310409: ~0.10% 311231: ~0.10% 312821: ~0.10% 314244: ~0.10% 314415: ~0.10% 314745: ~0.10% 316385: ~0.10% 316883: ~0.10% 317442: ~0.10% 318639: ~0.10% 318652: ~0.10% 320855: ~0.10% 321867: ~0.10% 322114: ~0.10% 323196: ~0.10% 324868: ~0.10% 327581: ~0.10% 329337: ~0.10% 331572: ~0.10% 331650: ~0.10% 331993: ~0.10% 332500: ~0.10% 334757: ~0.10% 336561: ~0.10% 336791: ~0.10% 337002: ~0.10% 338332: ~0.10% 338456: ~0.10% 339065: ~0.10% 339870: ~0.10% 340599: ~0.10% 341156: ~0.10% 342121: ~0.10% 342381: ~0.10% 343411: ~0.10% 344860: ~0.10% 345924: ~0.10% 346421: ~0.10% 346425: ~0.10% 348157: ~0.10% 351281: ~0.10% 351858: ~0.10% 352641: ~0.10% 353748: ~0.10% 357399: ~0.10% 359787: ~0.10% 359893: ~0.10% 360094: ~0.10% 360168: ~0.10% 361127: ~0.10% 362220: ~0.10% 362560: ~0.10% 366835: ~0.10% 367185: ~0.10% 369045: ~0.10% 371113: ~0.10% 376044: ~0.10% 376524: ~0.10% 377231: ~0.10% 377735: ~0.10% 378574: ~0.10% 379749: ~0.10% 379953: ~0.10% 381834: ~0.10% 384039: ~0.10% 384364: ~0.10% 384398: ~0.10% 384751: ~0.10% 385758: ~0.10% 385893: ~0.10% 386098: ~0.10% 387205: ~0.10% 387374: ~0.10% 388450: ~0.10% 388589: ~0.10% 388593: ~0.10% 389571: ~0.10% 389572: ~0.10% 391531: ~0.10% 391857: ~0.10% 393174: ~0.10% 393426: ~0.10% 396601: ~0.10% 396905: ~0.10% 397801: ~0.10% 398011: ~0.10% 398132: ~0.10% 398721: ~0.10% 399016: ~0.10% 401601: ~0.10% 403876: ~0.10% 403897: ~0.10% 404830: ~0.10% 406102: ~0.10% 406397: ~0.10% 407151: ~0.10% 409373: ~0.10% 410084: ~0.10% 410859: ~0.10% 411693: ~0.10% 411984: ~0.10% 412214: ~0.10% 412560: ~0.10% 413117: ~0.10% 416391: ~0.10% 417066: ~0.10% 417198: ~0.10% 417751: ~0.10% 417778: ~0.10% 420257: ~0.10% 420787: ~0.10% 421001: ~0.10% 421045: ~0.10% 421354: ~0.10% 428114: ~0.10% 429057: ~0.10% 429459: ~0.10% 430319: ~0.10% 431215: ~0.10% 431332: ~0.10% 431488: ~0.10% 432097: ~0.10% 432283: ~0.10% 434131: ~0.10% 434934: ~0.10% 435353: ~0.10% 437793: ~0.10% 438297: ~0.10% 438806: ~0.10% 439016: ~0.10% 439129: ~0.10% 439217: ~0.10% 439755: ~0.10% 440343: ~0.10% 440506: ~0.10% 441030: ~0.10% 441509: ~0.10% 443408: ~0.10% 443686: ~0.10% 445516: ~0.10% 445999: ~0.10% 447039: ~0.10% 447219: ~0.10% 447298: ~0.10% 453040: ~0.10% 453745: ~0.10% 454869: ~0.10% 456224: ~0.10% 456251: ~0.10% 457065: ~0.10% 459890: ~0.10% 460010: ~0.10% 463716: ~0.10% 465235: ~0.10% 470470: ~0.10% 471875: ~0.10% 472462: ~0.10% 474016: ~0.10% 479266: ~0.10% 479360: ~0.10% 480621: ~0.10% 483014: ~0.10% 484553: ~0.10% 485031: ~0.10% 485828: ~0.10% 486664: ~0.10% 488266: ~0.10% 489488: ~0.10% 490992: ~0.10% 491894: ~0.10% 491983: ~0.10% 492620: ~0.10% 493035: ~0.10% 493461: ~0.10% 494255: ~0.10% 496473: ~0.10% 496474: ~0.10% 496516: ~0.10% 496813: ~0.10% 496853: ~0.10% 499553: ~0.10% 499565: ~0.10% 499737: ~0.10% 500057: ~0.10% 500546: ~0.10% 501510: ~0.10% 501978: ~0.10% 503905: ~0.10% 510559: ~0.10% 511473: ~0.10% 512440: ~0.10% 513832: ~0.10% 514106: ~0.10% 514902: ~0.10% 515053: ~0.10% 515507: ~0.10% 516205: ~0.10% 517903: ~0.10% 518096: ~0.10% 520796: ~0.10% 521570: ~0.10% 522112: ~0.10% 523814: ~0.10% 525505: ~0.10% 525583: ~0.10% 525764: ~0.10% 528105: ~0.10% 530985: ~0.10% 532014: ~0.10% 534952: ~0.10% 538836: ~0.10% 539326: ~0.10% 539504: ~0.10% 541861: ~0.10% 542925: ~0.10% 543525: ~0.10% 544853: ~0.10% 545091: ~0.10% 546527: ~0.10% 546753: ~0.10% 548007: ~0.10% 548100: ~0.10% 554548: ~0.10% 555064: ~0.10% 560255: ~0.10% 560711: ~0.10% 561084: ~0.10% 561114: ~0.10% 561329: ~0.10% 561838: ~0.10% 561946: ~0.10% 564894: ~0.10% 566884: ~0.10% 568110: ~0.10% 569541: ~0.10% 570881: ~0.10% 571286: ~0.10% 571515: ~0.10% 571577: ~0.10% 572354: ~0.10% 573015: ~0.10% 573283: ~0.10% 577767: ~0.10% 578249: ~0.10% 578805: ~0.10% 580872: ~0.10% 581072: ~0.10% 581684: ~0.10% 582341: ~0.10% 583169: ~0.10% 583322: ~0.10% 583889: ~0.10% 584173: ~0.10% 585406: ~0.10% 585523: ~0.10% 585660: ~0.10% 587005: ~0.10% 587399: ~0.10% 588010: ~0.10% 588337: ~0.10% 590946: ~0.10% 593319: ~0.10% 595246: ~0.10% 597157: ~0.10% 597215: ~0.10% 597368: ~0.10% 597453: ~0.10% 598538: ~0.10% 601120: ~0.10% 604762: ~0.10% 605111: ~0.10% 605547: ~0.10% 606244: ~0.10% 606935: ~0.10% 607099: ~0.10% 609731: ~0.10% 609910: ~0.10% 610485: ~0.10% 613040: ~0.10% 614720: ~0.10% 615525: ~0.10% 616416: ~0.10% 618280: ~0.10% 619151: ~0.10% 619170: ~0.10% 622593: ~0.10% 622755: ~0.10% 623529: ~0.10% 625333: ~0.10% 625780: ~0.10% 626317: ~0.10% 626670: ~0.10% 628299: ~0.10% 628510: ~0.10% 629166: ~0.10% 630995: ~0.10% 632641: ~0.10% 634324: ~0.10% 634750: ~0.10% 636542: ~0.10% 637420: ~0.10% 641046: ~0.10% 643232: ~0.10% 643901: ~0.10% 644517: ~0.10% 645962: ~0.10% 647293: ~0.10% 647443: ~0.10% 648173: ~0.10% 649204: ~0.10% 650521: ~0.10% 651961: ~0.10% 652493: ~0.10% 655888: ~0.10% 656535: ~0.10% 658715: ~0.10% 659035: ~0.10% 659593: ~0.10% 660535: ~0.10% 662154: ~0.10% 662784: ~0.10% 663142: ~0.10% 666319: ~0.10% 666386: ~0.10% 666561: ~0.10% 668151: ~0.10% 668862: ~0.10% 670341: ~0.10% 671801: ~0.10% 673081: ~0.10% 673634: ~0.10% 673875: ~0.10% 673881: ~0.10% 674082: ~0.10% 675319: ~0.10% 675492: ~0.10% 676147: ~0.10% 676238: ~0.10% 676318: ~0.10% 676431: ~0.10% 677459: ~0.10% 678468: ~0.10% 679216: ~0.10% 679307: ~0.10% 680354: ~0.10% 681098: ~0.10% 681873: ~0.10% 684800: ~0.10% 685613: ~0.10% 685690: ~0.10% 686886: ~0.10% 689687: ~0.10% 689748: ~0.10% 694425: ~0.10% 694466: ~0.10% 698130: ~0.10% 702137: ~0.10% 703138: ~0.10% 704067: ~0.10% 704460: ~0.10% 705420: ~0.10% 706199: ~0.10% 706878: ~0.10% 708333: ~0.10% 710580: ~0.10% 710897: ~0.10% 713539: ~0.10% 713584: ~0.10% 714733: ~0.10% 718172: ~0.10% 719545: ~0.10% 719580: ~0.10% 720471: ~0.10% 720690: ~0.10% 722394: ~0.10% 723568: ~0.10% 724334: ~0.10% 724700: ~0.10% 727908: ~0.10% 728088: ~0.10% 729096: ~0.10% 730499: ~0.10% 730711: ~0.10% 733963: ~0.10% 734912: ~0.10% 736431: ~0.10% 738012: ~0.10% 738173: ~0.10% 739026: ~0.10% 739605: ~0.10% 740181: ~0.10% 742066: ~0.10% 742298: ~0.10% 745799: ~0.10% 748392: ~0.10% 748838: ~0.10% 749148: ~0.10% 751762: ~0.10% 752092: ~0.10% 752527: ~0.10% 753568: ~0.10% 755386: ~0.10% 756558: ~0.10% 756736: ~0.10% 758706: ~0.10% 759523: ~0.10% 760550: ~0.10% 762688: ~0.10% 762918: ~0.10% 763569: ~0.10% 763766: ~0.10% 765769: ~0.10% 766789: ~0.10% 768119: ~0.10% 768537: ~0.10% 773106: ~0.10% 775589: ~0.10% 775964: ~0.10% 776055: ~0.10% 777088: ~0.10% 777529: ~0.10% 778375: ~0.10% 781066: ~0.10% 782328: ~0.10% 783231: ~0.10% 784413: ~0.10% 785781: ~0.10% 786250: ~0.10% 786845: ~0.10% 788012: ~0.10% 791857: ~0.10% 792788: ~0.10% 793182: ~0.10% 794187: ~0.10% 794308: ~0.10% 794318: ~0.10% 796097: ~0.10% 796117: ~0.10% 797182: ~0.10% 798215: ~0.10% 802050: ~0.10% 802669: ~0.10% 804168: ~0.10% 804253: ~0.10% 804461: ~0.10% 805743: ~0.10% 808416: ~0.10% 808455: ~0.10% 810577: ~0.10% 811702: ~0.10% 811843: ~0.10% 815923: ~0.10% 816475: ~0.10% 818312: ~0.10% 818521: ~0.10% 819278: ~0.10% 820890: ~0.10% 821615: ~0.10% 823136: ~0.10% 823735: ~0.10% 829476: ~0.10% 830591: ~0.10% 832433: ~0.10% 832597: ~0.10% 833053: ~0.10% 835043: ~0.10% 835759: ~0.10% 837731: ~0.10% 837942: ~0.10% 839448: ~0.10% 840228: ~0.10% 840417: ~0.10% 841851: ~0.10% 843327: ~0.10% 843622: ~0.10% 844870: ~0.10% 846084: ~0.10% 846807: ~0.10% 847076: ~0.10% 847535: ~0.10% 847977: ~0.10% 848075: ~0.10% 848326: ~0.10% 852725: ~0.10% 853465: ~0.10% 856427: ~0.10% 857186: ~0.10% 858377: ~0.10% 858543: ~0.10% 860426: ~0.10% 863804: ~0.10% 866039: ~0.10% 866406: ~0.10% 867180: ~0.10% 868280: ~0.10% 872156: ~0.10% 872791: ~0.10% 872953: ~0.10% 872959: ~0.10% 875015: ~0.10% 876522: ~0.10% 878407: ~0.10% 878710: ~0.10% 878855: ~0.10% 880495: ~0.10% 882732: ~0.10% 884335: ~0.10% 884941: ~0.10% 885893: ~0.10% 886713: ~0.10% 887068: ~0.10% 887751: ~0.10% 888027: ~0.10% 890152: ~0.10% 891137: ~0.10% 891890: ~0.10% 892662: ~0.10% 892973: ~0.10% 893360: ~0.10% 893915: ~0.10% 893976: ~0.10% 894324: ~0.10% 895709: ~0.10% 897065: ~0.10% 898387: ~0.10% 899291: ~0.10% 899604: ~0.10% 900513: ~0.10% 900619: ~0.10% 901170: ~0.10% 902794: ~0.10% 903238: ~0.10% 904294: ~0.10% 904520: ~0.10% 904992: ~0.10% 907212: ~0.10% 908062: ~0.10% 908561: ~0.10% 911034: ~0.10% 911982: ~0.10% 913716: ~0.10% 914819: ~0.10% 915750: ~0.10% 915766: ~0.10% 916125: ~0.10% 916648: ~0.10% 917285: ~0.10% 918194: ~0.10% 926035: ~0.10% 927726: ~0.10% 929821: ~0.10% 930300: ~0.10% 930796: ~0.10% 931617: ~0.10% 932719: ~0.10% 933784: ~0.10% 934378: ~0.10% 935900: ~0.10% 936118: ~0.10% 936336: ~0.10% 937231: ~0.10% 938420: ~0.10% 939184: ~0.10% 939567: ~0.10% 941588: ~0.10% 944093: ~0.10% 944912: ~0.10% 945069: ~0.10% 945659: ~0.10% 946110: ~0.10% 950044: ~0.10% 954101: ~0.10% 954147: ~0.10% 958697: ~0.10% 959530: ~0.10% 961721: ~0.10% 963582: ~0.10% 964471: ~0.10% 965026: ~0.10% 966573: ~0.10% 967330: ~0.10% 968346: ~0.10% 970649: ~0.10% 970873: ~0.10% 971636: ~0.10% 971664: ~0.10% 973555: ~0.10% 973851: ~0.10% 974207: ~0.10% 976896: ~0.10% 981402: ~0.10% 983723: ~0.10% 984358: ~0.10% 984653: ~0.10% 987107: ~0.10% 987167: ~0.10% 994360: ~0.10% 995049: ~0.10% 1002688: ~0.10% 1004305: ~0.10% 1004650: ~0.10% 1004849: ~0.10% 1005118: ~0.10% 1005614: ~0.10% 1005626: ~0.10% 1005669: ~0.10% 1006835: ~0.10% 1011008: ~0.10% 1012299: ~0.10% 1014010: ~0.10% 1014030: ~0.10% 1016549: ~0.10% 1017016: ~0.10% 1017335: ~0.10% 1018386: ~0.10% 1020640: ~0.10% 1021041: ~0.10% 1021411: ~0.10% 1025341: ~0.10% 1025423: ~0.10% 1025767: ~0.10% 1026066: ~0.10% 1026434: ~0.10% 1027516: ~0.10% 1027703: ~0.10% 1028119: ~0.10% 1028642: ~0.10% 1031554: ~0.10% 1032300: ~0.10% 1033639: ~0.10% 1033660: ~0.10% 1034832: ~0.10% 1035274: ~0.10% 1037432: ~0.10% 1037536: ~0.10% 1037759: ~0.10% 1039860: ~0.10% 1041131: ~0.10% 1041892: ~0.10% 1043066: ~0.10% 1044326: ~0.10% 1044905: ~0.10% 1047848: ~0.10% 1048534: ~0.10% 1049477: ~0.10% 1050531: ~0.10% 1052073: ~0.10% 1052617: ~0.10% 1054049: ~0.10% 1055142: ~0.10% 1056933: ~0.10% 1057358: ~0.10% 1057911: ~0.10% 1061411: ~0.10% 1062328: ~0.10% 1062485: ~0.10% 1062534: ~0.10% 1062794: ~0.10% 1063269: ~0.10% 1063467: ~0.10% 1064568: ~0.10% 1064868: ~0.10% 1065481: ~0.10% 1065565: ~0.10% 1067970: ~0.10% 1068014: ~0.10% 1070203: ~0.10% 1070708: ~0.10% 1072038: ~0.10% 1072214: ~0.10% 1074885: ~0.10% 1075308: ~0.10% 1078872: ~0.10% 1078979: ~0.10% 1079266: ~0.10% 1079736: ~0.10% 1080075: ~0.10% 1081716: ~0.10% 1137391: ~0.10% 1138530: ~0.10% 1139697: ~0.10% 1140119: ~0.10% 1140869: ~0.10% 1141527: ~0.10% 1144693: ~0.10% 1145425: ~0.10% 1149162: ~0.10% 1149207: ~0.10% 1150086: ~0.10% 1150398: ~0.10% 1150731: ~0.10% 1151256: ~0.10% 1151403: ~0.10% 1152236: ~0.10% 1153693: ~0.10% 1155859: ~0.10% 1156918: ~0.10% 1158007: ~0.10% 1158559: ~0.10% 1158952: ~0.10% 1159165: ~0.10% 1161242: ~0.10% 1163227: ~0.10% 1166023: ~0.10% 1166231: ~0.10% 1167002: ~0.10% 1169844: ~0.10% 1170663: ~0.10% 1171580: ~0.10% 1172072: ~0.10% 1172083: ~0.10% 1173371: ~0.10% 1173809: ~0.10% 1174049: ~0.10% 1175044: ~0.10% 1175745: ~0.10% 1176061: ~0.10% 1176414: ~0.10% 1176993: ~0.10% 1177449: ~0.10% 1178311: ~0.10% 1179029: ~0.10% 1179069: ~0.10% 1180579: ~0.10% 1181077: ~0.10% 1183293: ~0.10% 1184313: ~0.10% 1185090: ~0.10% 1185669: ~0.10%	size: 32 elements	size: 32 elements

• Samples:

  query_id	document_ids	scores
  685613	[7546874, 1176459, 197677, 2306318, 8541504, ...]	[0.9999999992804947, 0.24845418756716053, 0.7594154013647826, 0.26644182105618575, 0.390668914839766, ...]
  237784	[6366584, 4034101, 2325374, 6914618, 6042146, ...]	[0.9999999991784339, 0.42233632827946693, 0.5956354295491569, 0.12644415907455164, 0.6636713730105909, ...]
  904294	[448408, 8743975, 49600, 7339401, 2714261, ...]	[0.9999999991841937, 0.877629062381539, 0.8330146583389045, 0.3116634796692611, 0.4633524534142185, ...]

• Loss: pylate.losses.distillation.Distillation

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: steps
• per_device_train_batch_size: 16
• learning_rate: 3e-05
• num_train_epochs: 1
• bf16: True

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: steps
• prediction_loss_only: True
• per_device_train_batch_size: 16
• per_device_eval_batch_size: 8
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 3e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1.0
• num_train_epochs: 1
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.0
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: True
• fp16: False
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: False
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• dispatch_batches: None
• split_batches: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• eval_use_gather_object: False
• average_tokens_across_devices: False
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: proportional

Training Logs

Click to expand

Epoch	Step	Training Loss	NanoClimateFEVER_MaxSim_ndcg@10	NanoDBPedia_MaxSim_ndcg@10	NanoFEVER_MaxSim_ndcg@10	NanoFiQA2018_MaxSim_ndcg@10	NanoHotpotQA_MaxSim_ndcg@10	NanoMSMARCO_MaxSim_ndcg@10	NanoNFCorpus_MaxSim_ndcg@10	NanoNQ_MaxSim_ndcg@10	NanoQuoraRetrieval_MaxSim_ndcg@10	NanoSCIDOCS_MaxSim_ndcg@10	NanoArguAna_MaxSim_ndcg@10	NanoSciFact_MaxSim_ndcg@10	NanoTouche2020_MaxSim_ndcg@10	NanoBEIR_mean_MaxSim_ndcg@10
0.0030	100	0.0366	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.0060	200	0.0325	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.0090	300	0.0308	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.0120	400	0.0277	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.0150	500	0.0268	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.0180	600	0.0264	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.0210	700	0.0254	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.0240	800	0.0247	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.0270	900	0.0246	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.0300	1000	0.0244	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.0330	1100	0.0242	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.0360	1200	0.023	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.0390	1300	0.0233	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.0420	1400	0.0224	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.0450	1500	0.0233	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.0480	1600	0.0221	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.0510	1700	0.0222	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.0540	1800	0.0216	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.0570	1900	0.0215	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.0600	2000	0.0211	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.0630	2100	0.021	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.0660	2200	0.0208	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.0690	2300	0.0205	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.0720	2400	0.0207	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.0750	2500	0.0204	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.0780	2600	0.0201	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.0810	2700	0.0199	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.0840	2800	0.0197	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.0870	2900	0.0201	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.0900	3000	0.0187	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.0930	3100	0.0198	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.0960	3200	0.0192	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.0990	3300	0.0194	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.1020	3400	0.0188	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.1050	3500	0.0193	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.1080	3600	0.0193	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.1110	3700	0.0186	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.1140	3800	0.0187	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.1170	3900	0.0186	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.1200	4000	0.0186	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.1230	4100	0.0181	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.1260	4200	0.0181	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.1290	4300	0.0182	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.1320	4400	0.0184	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.1350	4500	0.0178	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.1380	4600	0.017	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.1410	4700	0.0175	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.1440	4800	0.0174	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.1470	4900	0.0176	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.1500	5000	0.0177	0.2830	0.5426	0.8641	0.4244	0.8531	0.6244	0.3106	0.5798	0.9230	0.3109	0.3815	0.7461	0.5805	0.5711
0.1530	5100	0.0172	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.1560	5200	0.017	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.1590	5300	0.0173	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.1620	5400	0.0169	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.1650	5500	0.017	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.1680	5600	0.0168	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.1710	5700	0.0171	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.1740	5800	0.0165	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.1770	5900	0.0165	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.1801	6000	0.0169	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.1831	6100	0.0167	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.1861	6200	0.0169	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.1891	6300	0.0163	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.1921	6400	0.0165	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.1951	6500	0.0163	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.1981	6600	0.0166	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.2011	6700	0.0163	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.2041	6800	0.0162	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.2071	6900	0.0163	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.2101	7000	0.016	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.2131	7100	0.0164	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.2161	7200	0.0162	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.2191	7300	0.0158	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.2221	7400	0.0156	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.2251	7500	0.0154	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.2281	7600	0.016	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.2311	7700	0.0157	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.2341	7800	0.0161	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.2371	7900	0.0157	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.2401	8000	0.0156	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.2431	8100	0.0156	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.2461	8200	0.0156	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.2491	8300	0.0154	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.2521	8400	0.0153	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.2551	8500	0.0157	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.2581	8600	0.0155	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.2611	8700	0.0156	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.2641	8800	0.0155	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.2671	8900	0.0154	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.2701	9000	0.0152	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.2731	9100	0.0147	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.2761	9200	0.0154	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.2791	9300	0.0149	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.2821	9400	0.0149	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.2851	9500	0.0153	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.2881	9600	0.0146	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.2911	9700	0.0147	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.2941	9800	0.015	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.2971	9900	0.0149	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.3001	10000	0.0146	0.2674	0.5466	0.8739	0.4547	0.8499	0.5933	0.3170	0.6256	0.9321	0.3137	0.3855	0.7387	0.5768	0.5750
0.3031	10100	0.0147	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.3061	10200	0.0147	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.3091	10300	0.0147	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.3121	10400	0.0148	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.3151	10500	0.0147	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.3181	10600	0.0143	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.3211	10700	0.0144	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.3241	10800	0.0145	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.3271	10900	0.0148	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.3301	11000	0.015	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.3331	11100	0.0147	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.3361	11200	0.0148	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.3391	11300	0.0145	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.3421	11400	0.014	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.3451	11500	0.0146	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.3481	11600	0.0143	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.3511	11700	0.0142	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.3541	11800	0.0141	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.3571	11900	0.0139	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.3601	12000	0.0141	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.3631	12100	0.0142	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.3661	12200	0.0143	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.3691	12300	0.0138	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.3721	12400	0.0137	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.3751	12500	0.0139	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.3781	12600	0.0142	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.3811	12700	0.0136	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.3841	12800	0.0138	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.3871	12900	0.014	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.3901	13000	0.0137	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.3931	13100	0.014	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.3961	13200	0.0141	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.3991	13300	0.014	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.4021	13400	0.0139	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.4051	13500	0.0137	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.4081	13600	0.0138	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.4111	13700	0.0137	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.4141	13800	0.0136	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.4171	13900	0.0138	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.4201	14000	0.0139	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.4231	14100	0.0137	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.4261	14200	0.0137	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.4291	14300	0.0138	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.4321	14400	0.0135	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.4351	14500	0.0133	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.4381	14600	0.0132	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.4411	14700	0.0134	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.4441	14800	0.0132	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.4471	14900	0.0131	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.4501	15000	0.0131	0.2885	0.5666	0.8718	0.4695	0.8453	0.6405	0.3128	0.6500	0.9257	0.3081	0.3923	0.7361	0.5852	0.5840
0.4531	15100	0.0129	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.4561	15200	0.0137	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.4591	15300	0.0134	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.4621	15400	0.0132	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.4651	15500	0.0131	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.4681	15600	0.0131	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.4711	15700	0.0134	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.4741	15800	0.0127	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.4771	15900	0.0136	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.4801	16000	0.0129	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.4831	16100	0.0129	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.4861	16200	0.0131	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.4891	16300	0.0131	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.4921	16400	0.013	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.4951	16500	0.0128	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.4981	16600	0.0129	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.5011	16700	0.0131	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.5041	16800	0.0131	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.5071	16900	0.0129	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.5101	17000	0.0127	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.5131	17100	0.013	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.5161	17200	0.0128	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.5191	17300	0.013	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.5221	17400	0.0129	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.5251	17500	0.0129	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.5281	17600	0.0128	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.5311	17700	0.0132	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.5341	17800	0.0127	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.5372	17900	0.0128	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.5402	18000	0.0129	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.5432	18100	0.0123	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.5462	18200	0.0126	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.5492	18300	0.0127	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.5522	18400	0.0129	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.5552	18500	0.0127	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.5582	18600	0.0125	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.5612	18700	0.0125	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.5642	18800	0.0129	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.5672	18900	0.0127	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.5702	19000	0.0122	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.5732	19100	0.0124	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.5762	19200	0.0127	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.5792	19300	0.0127	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.5822	19400	0.0125	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.5852	19500	0.0124	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.5882	19600	0.0122	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.5912	19700	0.0123	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.5942	19800	0.0127	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.5972	19900	0.0123	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.6002	20000	0.0124	0.2807	0.5738	0.8748	0.4586	0.8533	0.6174	0.3227	0.6215	0.9219	0.3104	0.4132	0.7348	0.5696	0.5810
0.6032	20100	0.0126	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.6062	20200	0.0122	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.6092	20300	0.0126	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.6122	20400	0.0124	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.6152	20500	0.0123	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.6182	20600	0.0125	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.6212	20700	0.0126	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.6242	20800	0.0125	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.6272	20900	0.0125	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.6302	21000	0.0125	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.6332	21100	0.012	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.6362	21200	0.012	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.6392	21300	0.0123	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.6422	21400	0.0122	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.6452	21500	0.0123	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.6482	21600	0.0121	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.6512	21700	0.0122	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.6542	21800	0.0121	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.6572	21900	0.012	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.6602	22000	0.0123	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.6632	22100	0.0122	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.6662	22200	0.0121	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.6692	22300	0.0126	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.6722	22400	0.0123	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.6752	22500	0.0119	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.6782	22600	0.0125	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.6812	22700	0.0127	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.6842	22800	0.0122	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.6872	22900	0.0122	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.6902	23000	0.0118	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.6932	23100	0.0119	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.6962	23200	0.0126	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.6992	23300	0.0122	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.7022	23400	0.0118	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.7052	23500	0.0124	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.7082	23600	0.0119	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.7112	23700	0.0122	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.7142	23800	0.0123	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.7172	23900	0.0118	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.7202	24000	0.0118	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.7232	24100	0.0119	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.7262	24200	0.0119	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.7292	24300	0.0121	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.7322	24400	0.0117	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.7352	24500	0.0119	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.7382	24600	0.0119	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.7412	24700	0.0121	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.7442	24800	0.0122	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.7472	24900	0.0118	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.7502	25000	0.0118	0.2940	0.5881	0.8842	0.4741	0.8653	0.6288	0.3414	0.6734	0.9000	0.3151	0.4361	0.7387	0.5610	0.5923
0.7532	25100	0.012	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.7562	25200	0.0115	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.7592	25300	0.0118	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.7622	25400	0.0116	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.7652	25500	0.012	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.7682	25600	0.0119	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.7712	25700	0.0119	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.7742	25800	0.0113	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.7772	25900	0.0118	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.7802	26000	0.0119	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.7832	26100	0.0119	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.7862	26200	0.0115	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.7892	26300	0.0118	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.7922	26400	0.0123	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.7952	26500	0.0116	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.7982	26600	0.0113	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.8012	26700	0.0117	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.8042	26800	0.0119	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.8072	26900	0.012	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.8102	27000	0.0116	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.8132	27100	0.0119	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.8162	27200	0.0119	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.8192	27300	0.012	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.8222	27400	0.012	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.8252	27500	0.0117	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.8282	27600	0.0116	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.8312	27700	0.0119	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.8342	27800	0.0112	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.8372	27900	0.0117	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.8402	28000	0.0115	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.8432	28100	0.0115	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.8462	28200	0.0115	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.8492	28300	0.0116	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.8522	28400	0.0115	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.8552	28500	0.0119	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.8582	28600	0.0117	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.8612	28700	0.0117	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.8642	28800	0.0115	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.8672	28900	0.0119	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.8702	29000	0.012	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.8732	29100	0.0117	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.8762	29200	0.0119	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.8792	29300	0.0113	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.8822	29400	0.0114	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.8852	29500	0.0119	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.8882	29600	0.0118	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.8912	29700	0.0115	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.8943	29800	0.0115	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.8973	29900	0.0117	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.9003	30000	0.0113	0.3059	0.5944	0.8864	0.4729	0.8634	0.6351	0.3302	0.6699	0.9086	0.3142	0.4545	0.7357	0.5613	0.5948
0.9033	30100	0.0115	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.9063	30200	0.0118	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.9093	30300	0.0115	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.9123	30400	0.0115	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.9153	30500	0.0118	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.9183	30600	0.0117	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.9213	30700	0.0119	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.9243	30800	0.0114	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.9273	30900	0.0113	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.9303	31000	0.0113	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.9333	31100	0.0116	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.9363	31200	0.0113	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.9393	31300	0.0114	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.9423	31400	0.0114	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.9453	31500	0.0118	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.9483	31600	0.0114	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.9513	31700	0.0115	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.9543	31800	0.0112	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.9573	31900	0.0115	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.9603	32000	0.0116	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.9633	32100	0.0114	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.9663	32200	0.0114	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.9693	32300	0.0115	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.9723	32400	0.0117	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.9753	32500	0.0114	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.9783	32600	0.0115	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.9813	32700	0.0115	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.9843	32800	0.0118	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.9873	32900	0.0114	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.9903	33000	0.0113	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.9933	33100	0.0116	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.9963	33200	0.0117	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.9993	33300	0.0115	-	-	-	-	-	-	-	-	-	-	-	-	-	-

Framework Versions

• Python: 3.11.13
• Sentence Transformers: 4.0.2
• PyLate: 1.2.0
• Transformers: 4.48.2
• PyTorch: 2.6.0+cu124
• Accelerate: 1.10.0
• Datasets: 4.0.0
• Tokenizers: 0.21.4

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084"
}

PyLate

@misc{PyLate,
title={PyLate: Flexible Training and Retrieval for Late Interaction Models},
author={Chaffin, Antoine and Sourty, Raphaël},
url={https://github.com/lightonai/pylate},
year={2024}
}