Update README.md

README.md

@@ -4,66 +4,61 @@ tags:
 - sentence-similarity
 - feature-extraction
 - transformers
+- Qwen2
 pipeline_tag: sentence-similarity
 library_name: sentence-transformers
+license: other
+license_name: qodoai-open-rail-m
+license_link: LICENSE
+base_model:
+- Alibaba-NLP/gte-Qwen2-7B-instruct
 ---
 
-# SentenceTransformer
-
-This is a [sentence-transformers](https://www.SBERT.net) model trained. It maps sentences & paragraphs to a 3584-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.
-
-## Model Details
-
-### Model Description
-- **Model Type:** Sentence Transformer
-<!-- - **Base model:** [Unknown](https://huggingface.co/unknown) -->
-- **Maximum Sequence Length:** 32768 tokens
-- **Output Dimensionality:** 3584 dimensions
-- **Similarity Function:** Cosine Similarity
-<!-- - **Training Dataset:** Unknown -->
-<!-- - **Language:** Unknown -->
-<!-- - **License:** Unknown -->
-
-### Model Sources
-
-- **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
-- **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
-- **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers)
-
-### Full Model Architecture
-
+## Qodo-Embed-1 
+**Qodo-Embed-1 is a state-of-the-art** code embedding model designed for retrieval tasks in the software development domain.
+It is offered in two sizes: lite (1.5B) and medium (7B). The model is optimized for natural language-to-code and code-to-code retrieval, making it highly effective for applications such as code search, retrieval-augmented generation (RAG), and contextual understanding of programming languages.
+This model outperforms all previous open-source models in the COIR and MTab leaderboards, achieving best-in-class performance with a significantly smaller size compared to competing models.
+
+### Languages Supported: 
+* Python
+* C++
+* C#
+* Go
+* Java
+* Javascript
+* PHP
+* Ruby
+* Typescript
+ 
+## Model Information 
+- Model Size: 7B 
+- Embedding Dimension: 3584
+- Max Input Tokens: 32k
+
+## Requirements
 ```
-SentenceTransformer(
-  (0): Transformer({'max_seq_length': 32768, 'do_lower_case': False}) with Transformer model: Qwen2Model 
-  (1): Pooling({'word_embedding_dimension': 3584, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': True, 'include_prompt': True})
-)
+transformers>=4.39.2
+flash_attn>=2.5.6
 ```
 
 ## Usage
 
-### Direct Usage (Sentence Transformers)
+### Sentence Transformers
 
-First install the Sentence Transformers library:
-
-```bash
-pip install -U sentence-transformers
-```
-
-Then you can load this model and run inference.
 ```python
 from sentence_transformers import SentenceTransformer
 
 # Download from the 🤗 Hub
-model = SentenceTransformer("sentence_transformers_model_id")
+model = SentenceTransformer("Qodo/Qodo-Embed-1-7B")
 # Run inference
 sentences = [
-    'The weather is lovely today.',
-    "It's so sunny outside!",
-    'He drove to the stadium.',
+    'accumulator = sum(item.value for item in collection)',  
+    'result = reduce(lambda acc, curr: acc + curr.amount, data, 0)',  
+    'matrix = [[i*j for j in range(n)] for i in range(n)]'  
 ]
 embeddings = model.encode(sentences)
 print(embeddings.shape)
-# [3, 3584]
+# [3, 1536]
 
 # Get the similarity scores for the embeddings
 similarities = model.similarity(embeddings, embeddings)
@@ -71,71 +66,85 @@ print(similarities.shape)
 # [3, 3]
 ```
 
-<!--
-### Direct Usage (Transformers)
-
-<details><summary>Click to see the direct usage in Transformers</summary>
-
-</details>
--->
-
-<!--
-### Downstream Usage (Sentence Transformers)
-
-You can finetune this model on your own dataset.
+### Transformers
 
-<details><summary>Click to expand</summary>
-
-</details>
--->
-
-<!--
-### Out-of-Scope Use
-
-*List how the model may foreseeably be misused and address what users ought not to do with the model.*
--->
-
-<!--
-## Bias, Risks and Limitations
-
-*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
--->
-
-<!--
-### Recommendations
-
-*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
--->
-
-## Training Details
-
-### Framework Versions
-- Python: 3.10.12
-- Sentence Transformers: 3.4.1
-- Transformers: 4.49.0
-- PyTorch: 2.5.1+cu124
-- Accelerate: 1.1.1
-- Datasets: 3.1.0
-- Tokenizers: 0.21.0
-
-## Citation
-
-### BibTeX
-
-<!--
-## Glossary
+```python
+import torch
+import torch.nn.functional as F
+
+from torch import Tensor
+from transformers import AutoTokenizer, AutoModel
+
+
+def last_token_pool(last_hidden_states: Tensor,
+                 attention_mask: Tensor) -> Tensor:
+    left_padding = (attention_mask[:, -1].sum() == attention_mask.shape[0])
+    if left_padding:
+        return last_hidden_states[:, -1]
+    else:
+        sequence_lengths = attention_mask.sum(dim=1) - 1
+        batch_size = last_hidden_states.shape[0]
+        return last_hidden_states[torch.arange(batch_size, device=last_hidden_states.device), sequence_lengths]
+
+
+# Each query must come with a one-sentence instruction that describes the task
+queries = [
+      'how to handle memory efficient data streaming',
+      'implement binary tree traversal'
+  ]
+
+documents = [
+        """def process_in_chunks():
+            buffer = deque(maxlen=1000)
+            for record in source_iterator:
+                buffer.append(transform(record))
+                if len(buffer) >= 1000:
+                    yield from buffer
+                    buffer.clear()""",
+
+        """class LazyLoader:
+            def __init__(self, source):
+                self.generator = iter(source)
+                self._cache = []
+
+            def next_batch(self, size=100):
+                while len(self._cache) < size:
+                    try:
+                        self._cache.append(next(self.generator))
+                    except StopIteration:
+                        break
+                return self._cache.pop(0) if self._cache else None""",
+
+        """def dfs_recursive(root):
+            if not root:
+                return []
+            stack = []
+            stack.extend(dfs_recursive(root.right))
+            stack.append(root.val)
+            stack.extend(dfs_recursive(root.left))
+            return stack"""
+    ]
+input_texts = queries + documents
+
+tokenizer = AutoTokenizer.from_pretrained('Qodo/Qodo-Embed-1-7B', trust_remote_code=True)
+model = AutoModel.from_pretrained('Qodo/Qodo-Embed-1-1.5B', trust_remote_code=True)
+
+max_length = 8192
+
+# Tokenize the input texts
+batch_dict = tokenizer(input_texts, max_length=max_length, padding=True, truncation=True, return_tensors='pt')
+outputs = model(**batch_dict)
+embeddings = last_token_pool(outputs.last_hidden_state, batch_dict['attention_mask'])
+
+# normalize embeddings
+embeddings = F.normalize(embeddings, p=2, dim=1)
+scores = (embeddings[:2] @ embeddings[2:].T) * 100
+print(scores.tolist())
+```
 
-*Clearly define terms in order to be accessible across audiences.*
--->
 
-<!--
-## Model Card Authors
 
-*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
--->
 
-<!--
-## Model Card Contact
+## License
+[Qodo-Model-Terms-of-Service](https://www.qodo.ai/qodo-model-terms-of-service/)
 
-*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
--->