Duplicate from ai-forever/ruclip-vit-large-patch14-336

Co-authored-by: ai-forever <[email protected]>

.gitattributes

@@ -0,0 +1,27 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bin.* filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text 
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zstandard filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,63 @@
+# ruclip-vit-large-patch14-336
+
+**RuCLIP** (**Ru**ssian **C**ontrastive **L**anguage–**I**mage **P**retraining) is a multimodal model 
+for obtaining images and text similarities and rearranging captions and pictures. 
+RuCLIP builds on a large body of work on zero-shot transfer, computer vision, natural language processing and 
+multimodal learning. 
+
+Model was trained by [Sber AI](https://github.com/sberbank-ai) and [SberDevices](https://sberdevices.ru/) teams.  
+* Task: `text ranking`; `image ranking`; `zero-shot image classification`;
+* Type: `encoder`
+* Num Parameters: `430M`
+* Training Data Volume: `240 million text-image pairs`
+* Language: `Russian`
+* Context Length: `77`
+* Transformer Layers: `12`
+* Transformer Width: `768`
+* Transformer Heads: `12`
+* Image Size: `336`
+* Vision Layers: `24`
+* Vision Width: `1024`
+* Vision Patch Size: `14`
+
+## Usage [Github](https://github.com/sberbank-ai/ru-clip)
+
+```
+pip install ruclip
+```
+
+```python
+clip, processor = ruclip.load("ruclip-vit-large-patch14-336", device="cuda")
+```
+
+## Performance
+We have evaluated the performance on the following datasets:
+
+| Dataset       | Metric Name    | Metric Result       |
+|:--------------|:---------------|:--------------------|
+| Food101       | acc            | 0.712		      	   |
+| CIFAR10       | acc            | 0.906	             |
+| CIFAR100      | acc            | 0.591               |
+| Birdsnap      | acc            | 0.213               |
+| SUN397        | acc            | 0.523               |
+| Stanford Cars | acc            | 0.659               |
+| DTD           | acc            | 0.408	             |
+| MNIST         | acc            | 0.242	             |
+| STL10         | acc            | 0.956	             |
+| PCam          | acc            | 0.554               |
+| CLEVR         | acc            | 0.142               |
+| Rendered SST2 | acc            | 0.539               |
+| ImageNet      | acc            | 0.488               |
+| FGVC Aircraft | mean-per-class | 0.075               |
+| Oxford Pets   | mean-per-class | 0.546               |
+| Caltech101    | mean-per-class | 0.835               |
+| Flowers102    | mean-per-class | 0.517               |
+| HatefulMemes  | roc-auc        | 0.519               |
+
+
+# Authors
+
++ Alex Shonenkov: [Github](https://github.com/shonenkov), [Kaggle GM](https://www.kaggle.com/shonenkov)
++ Daniil Chesakov: [Github](https://github.com/Danyache)
++ Denis Dimitrov: [Github](https://github.com/denndimitrov)
++ Igor Pavlov: [Github](https://github.com/boomb0om)

bpe.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:26db7928d1a022215fc5a1948c46d17c8e39e471b4d0f8b3d1edfd91c7c62571
+size 747907

config.json

@@ -0,0 +1,14 @@
+{
+  "embed_dim": 768,
+  "image_resolution": 336,
+  "vision_layers": 24,
+  "vision_width": 1024,
+  "vision_patch_size":  14,
+  "context_length": 77,
+  "vocab_size": 49408,
+  "transformer_width": 768,
+  "transformer_heads": 12,
+  "transformer_layers": 12,
+  "mean": [0.48145466, 0.4578275, 0.40821073],
+  "std": [0.26862954, 0.26130258, 0.27577711]
+}

pytorch_model.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:39edca30dbb7421989cc78e8787bd3b8ad6829ac6f1279f9c29a1535fe86bb9f
+size 1711937797