Upload 9 files

.gitattributes

@@ -34,3 +34,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
 0_StaticEmbedding/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

README.md

@@ -1,142 +1,172 @@
 ---
 base_model: jinaai/jina-embeddings-v3
-library_name: sentence-transformers
-pipeline_tag: sentence-similarity
+language:
+- multilingual
+- af
+- am
+- ar
+- as
+- az
+- be
+- bg
+- bn
+- br
+- bs
+- ca
+- cs
+- cy
+- da
+- de
+- el
+- en
+- eo
+- es
+- et
+- eu
+- fa
+- fi
+- fr
+- fy
+- ga
+- gd
+- gl
+- gu
+- ha
+- he
+- hi
+- hr
+- hu
+- hy
+- id
+- is
+- it
+- ja
+- jv
+- ka
+- kk
+- km
+- kn
+- ko
+- ku
+- ky
+- la
+- lo
+- lt
+- lv
+- mg
+- mk
+- ml
+- mn
+- mr
+- ms
+- my
+- ne
+- nl
+- 'no'
+- om
+- or
+- pa
+- pl
+- ps
+- pt
+- ro
+- ru
+- sa
+- sd
+- si
+- sk
+- sl
+- so
+- sq
+- sr
+- su
+- sv
+- sw
+- ta
+- te
+- th
+- tl
+- tr
+- ug
+- uk
+- ur
+- uz
+- vi
+- xh
+- yi
+- zh
+library_name: model2vec
+license: mit
+model_name: onnx
 tags:
+- embeddings
+- static-embeddings
 - sentence-transformers
-- sentence-similarity
-- feature-extraction
 ---
 
-# SentenceTransformer based on jinaai/jina-embeddings-v3
+# onnx Model Card
 
-This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [jinaai/jina-embeddings-v3](https://huggingface.co/jinaai/jina-embeddings-v3). It maps sentences & paragraphs to a 1024-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.
+This [Model2Vec](https://github.com/MinishLab/model2vec) model is a distilled version of the [jinaai/jina-embeddings-v3](https://huggingface.co/jinaai/jina-embeddings-v3) Sentence Transformer. It uses static embeddings, allowing text embeddings to be computed orders of magnitude faster on both GPU and CPU. It is designed for applications where computational resources are limited or where real-time performance is critical.
 
-## Model Details
 
-### Model Description
-- **Model Type:** Sentence Transformer
-- **Base model:** [jinaai/jina-embeddings-v3](https://huggingface.co/jinaai/jina-embeddings-v3) <!-- at revision 4be32c2f5d65b95e4bcce473545b7883ec8d2edd -->
-- **Maximum Sequence Length:** inf tokens
-- **Output Dimensionality:** 1024 tokens
-- **Similarity Function:** Cosine Similarity
-<!-- - **Training Dataset:** Unknown -->
-<!-- - **Language:** Unknown -->
-<!-- - **License:** Unknown -->
-
-### Model Sources
-
-- **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
-- **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
-- **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers)
-
-### Full Model Architecture
+## Installation
 
+Install model2vec using pip:
 ```
-SentenceTransformer(
-  (0): StaticEmbedding(
-    (embedding): EmbeddingBag(250002, 1024, mode='mean')
-  )
-)
+pip install model2vec
 ```
 
 ## Usage
+Load this model using the `from_pretrained` method:
+```python
+from model2vec import StaticModel
 
-### Direct Usage (Sentence Transformers)
-
-First install the Sentence Transformers library:
+# Load a pretrained Model2Vec model
+model = StaticModel.from_pretrained("onnx")
 
-```bash
-pip install -U sentence-transformers
+# Compute text embeddings
+embeddings = model.encode(["Example sentence"])
 ```
 
-Then you can load this model and run inference.
+Alternatively, you can distill your own model using the `distill` method:
 ```python
-from sentence_transformers import SentenceTransformer
-
-# Download from the 🤗 Hub
-model = SentenceTransformer("Thaweewat/jina-embedding-v3-m2v-1024")
-# Run inference
-sentences = [
-    'The weather is lovely today.',
-    "It's so sunny outside!",
-    'He drove to the stadium.',
-]
-embeddings = model.encode(sentences)
-print(embeddings.shape)
-# [3, 1024]
-
-# Get the similarity scores for the embeddings
-similarities = model.similarity(embeddings, embeddings)
-print(similarities.shape)
-# [3, 3]
-```
-
-<!--
-### Direct Usage (Transformers)
-
-<details><summary>Click to see the direct usage in Transformers</summary>
-
-</details>
--->
-
-<!--
-### Downstream Usage (Sentence Transformers)
+from model2vec.distill import distill
 
-You can finetune this model on your own dataset.
+# Choose a Sentence Transformer model
+model_name = "BAAI/bge-base-en-v1.5"
 
-<details><summary>Click to expand</summary>
+# Distill the model
+m2v_model = distill(model_name=model_name, pca_dims=256)
 
-</details>
--->
-
-<!--
-### Out-of-Scope Use
+# Save the model
+m2v_model.save_pretrained("m2v_model")
+```
 
-*List how the model may foreseeably be misused and address what users ought not to do with the model.*
--->
+## How it works
 
-<!--
-## Bias, Risks and Limitations
+Model2vec creates a small, fast, and powerful model that outperforms other static embedding models by a large margin on all tasks we could find, while being much faster to create than traditional static embedding models such as GloVe. Best of all, you don't need any data to distill a model using Model2Vec.
 
-*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
--->
+It works by passing a vocabulary through a sentence transformer model, then reducing the dimensionality of the resulting embeddings using PCA, and finally weighting the embeddings using zipf weighting. During inference, we simply take the mean of all token embeddings occurring in a sentence.
 
-<!--
-### Recommendations
+## Additional Resources
 
-*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
--->
+- [All Model2Vec models on the hub](https://huggingface.co/models?library=model2vec)
+- [Model2Vec Repo](https://github.com/MinishLab/model2vec)
+- [Model2Vec Results](https://github.com/MinishLab/model2vec?tab=readme-ov-file#results)
+- [Model2Vec Tutorials](https://github.com/MinishLab/model2vec/tree/main/tutorials)
 
-## Training Details
+## Library Authors
 
-### Framework Versions
-- Python: 3.10.12
-- Sentence Transformers: 3.2.0
-- Transformers: 4.44.2
-- PyTorch: 2.4.1+cu121
-- Accelerate: 0.34.2
-- Datasets: 
-- Tokenizers: 0.19.1
+Model2Vec was developed by the [Minish Lab](https://github.com/MinishLab) team consisting of [Stephan Tulkens](https://github.com/stephantul) and [Thomas van Dongen](https://github.com/Pringled).
 
 ## Citation
 
-### BibTeX
-
-<!--
-## Glossary
-
-*Clearly define terms in order to be accessible across audiences.*
--->
-
-<!--
-## Model Card Authors
-
-*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
--->
-
-<!--
-## Model Card Contact
-
-*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
--->
+Please cite the [Model2Vec repository](https://github.com/MinishLab/model2vec) if you use this model in your work.
+```
+@software{minishlab2024model2vec,
+  authors = {Stephan Tulkens, Thomas van Dongen},
+  title = {Model2Vec: Turn any Sentence Transformer into a Small Fast Model},
+  year = {2024},
+  url = {https://github.com/MinishLab/model2vec},
+}
+```

config.json

@@ -1 +1,13 @@
-{"tokenizer_name": "jinaai/jina-embeddings-v3", "apply_pca": 256, "apply_zipf": true, "hidden_dim": 256, "seq_length": 1000000, "normalize": false}
+{
+    "model_type": "model2vec",
+    "architectures": [
+        "StaticModel"
+    ],
+    "tokenizer_name": "jinaai/jina-embeddings-v3",
+    "apply_pca": 1024,
+    "apply_zipf": null,
+    "sif_coefficient": 0.0001,
+    "hidden_dim": 1024,
+    "seq_length": 1000000,
+    "normalize": true
+}

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:4ddaa9abbb557e9a66eac96b61f9d92455d737d4deac7ee663cf0ded24978e6d
+oid sha256:96a4ad4fbb7fc15c91e9707f966d2563a10fdd12ca1f537acade2d75a707c519
 size 1024008288

modules.json

@@ -1,8 +1,14 @@
 [
-  {
-    "idx": 0,
-    "name": "0",
-    "path": "0_StaticEmbedding",
-    "type": "sentence_transformers.models.StaticEmbedding"
-  }
+    {
+        "idx": 0,
+        "name": "0",
+        "path": ".",
+        "type": "sentence_transformers.models.StaticEmbedding"
+    },
+    {
+        "idx": 1,
+        "name": "1",
+        "path": "1_Normalize",
+        "type": "sentence_transformers.models.Normalize"
+    }
 ]

onnx/model.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:32ca8c3d06d5d5487c075c795be8cfda729be26cc0ba62578ad4ec05ff0e811b
+size 1024012377

special_tokens_map.json

@@ -0,0 +1,9 @@
+{
+  "bos_token": "<s>",
+  "cls_token": "<s>",
+  "eos_token": "</s>",
+  "mask_token": "<mask>",
+  "pad_token": "<pad>",
+  "sep_token": "</s>",
+  "unk_token": "<unk>"
+}

tokenizer_config.json

@@ -1,54 +1,55 @@
 {
-  "added_tokens_decoder": {
-    "0": {
-      "content": "<s>",
-      "lstrip": false,
-      "normalized": false,
-      "rstrip": false,
-      "single_word": false,
-      "special": true
+    "added_tokens_decoder": {
+        "0": {
+            "content": "<s>",
+            "lstrip": false,
+            "normalized": false,
+            "rstrip": false,
+            "single_word": false,
+            "special": true
+        },
+        "1": {
+            "content": "<pad>",
+            "lstrip": false,
+            "normalized": false,
+            "rstrip": false,
+            "single_word": false,
+            "special": true
+        },
+        "2": {
+            "content": "</s>",
+            "lstrip": false,
+            "normalized": false,
+            "rstrip": false,
+            "single_word": false,
+            "special": true
+        },
+        "250001": {
+            "content": "<mask>",
+            "lstrip": true,
+            "normalized": false,
+            "rstrip": false,
+            "single_word": false,
+            "special": true
+        },
+        "3": {
+            "content": "<unk>",
+            "lstrip": false,
+            "normalized": false,
+            "rstrip": false,
+            "single_word": false,
+            "special": true
+        }
     },
-    "1": {
-      "content": "<pad>",
-      "lstrip": false,
-      "normalized": false,
-      "rstrip": false,
-      "single_word": false,
-      "special": true
-    },
-    "2": {
-      "content": "</s>",
-      "lstrip": false,
-      "normalized": false,
-      "rstrip": false,
-      "single_word": false,
-      "special": true
-    },
-    "3": {
-      "content": "<unk>",
-      "lstrip": false,
-      "normalized": false,
-      "rstrip": false,
-      "single_word": false,
-      "special": true
-    },
-    "250001": {
-      "content": "<mask>",
-      "lstrip": true,
-      "normalized": false,
-      "rstrip": false,
-      "single_word": false,
-      "special": true
-    }
-  },
-  "bos_token": "<s>",
-  "clean_up_tokenization_spaces": true,
-  "cls_token": "<s>",
-  "eos_token": "</s>",
-  "mask_token": "<mask>",
-  "model_max_length": 8194,
-  "pad_token": "<pad>",
-  "sep_token": "</s>",
-  "tokenizer_class": "XLMRobertaTokenizer",
-  "unk_token": "<unk>"
-}
+    "bos_token": "<s>",
+    "clean_up_tokenization_spaces": false,
+    "cls_token": "<s>",
+    "eos_token": "</s>",
+    "extra_special_tokens": {},
+    "mask_token": "<mask>",
+    "model_max_length": 1000000000000000019884624838656,
+    "pad_token": "<pad>",
+    "sep_token": "</s>",
+    "tokenizer_class": "XLMRobertaTokenizerFast",
+    "unk_token": "<unk>"
+}