Update Splade Index

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+corpus.jsonl filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,60 @@
+---
+language: en
+library_name: splade-index
+tags:
+- splade
+- splade-index
+- retrieval
+- search
+- sparse
+---
+
+# Splade-Index
+
+This is an index created with the [splade-index](https://github.com/rasyosef/splade-index) library (version `0.1.2`)
+
+## Installation
+
+You can install the `splade-index` library with `pip`:
+
+```bash
+pip install "splade-index==0.1.2"
+
+# Include extra dependencies like stemmer
+pip install "splade-index[full]==0.1.2"
+
+# For huggingface hub usage
+pip install huggingface_hub
+```
+
+## Load this Index
+
+You can use the following code to load this SPLADE index from Hugging Face hub:
+
+```python
+import os
+from sentence_transformers import SparseEncoder
+from splade_index import SPLADE
+
+# Download the SPLADE model that was used to create the index from the HuggingFace Hub
+model_id = "the-splade-model-id" # Enter the splade model id
+model = SparseEncoder(model_id)
+
+# Set your huggingface token if repo is private
+token = os.environ["HF_TOKEN"]
+repo_id = "rasyosef/natural_questions_108k_splade_index"
+
+# Load a SPLADE index from the Hugging Face model hub
+retriever = SPLADE.load_from_hub(repo_id, model=model, token=token)
+```
+
+## Stats
+
+This dataset was created using the following data:
+
+| Statistic | Value |
+| --- | --- |
+| Number of documents | 108593 |
+| Number of tokens | 20265694 |
+| Average tokens per document | 186.62 |
+

corpus.jsonl

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8d1cf8ec2ec3f922161cd24ffac66d8c70dc8947006d3bdd80c1d52aeb1b21ed
+size 49244396

csc.index.npz

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4a9f750d4328cce9750a3ed54e403b82ab33593249c02a78c66ce94b6da6c3bb
+size 112269260

params.index.json

@@ -0,0 +1,7 @@
+{
+    "dtype": "float32",
+    "int_dtype": "int32",
+    "num_docs": 108593,
+    "version": "0.1.2",
+    "backend": "numpy"
+}