jordan0811 commited on
Commit
0315ad2
·
verified ·
1 Parent(s): 7de6efb

Create README.md

Browse files
Files changed (1) hide show
  1. README.md +58 -0
README.md ADDED
@@ -0,0 +1,58 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ language:
4
+ - en
5
+ base_model:
6
+ - qihoo360/fg-clip2-base
7
+ tags:
8
+ - CLIP
9
+ - FG-CLIP
10
+ - FG-CLIP2
11
+ - Image-Text Encoder
12
+ ---
13
+
14
+ # FG-CLIP2
15
+
16
+ The version of FG-CLIP2 has been converted to run on the Axera NPU using w8a16 quantization. Compatible with Pulsar2 version: 4.2
17
+
18
+ If you want to know how to convert the FG-CLIP2 model into an axmodel that can run on the axera npu board, please read [this link](https://github.com/Jordan-5i/FG-CLIP/tree/main/ax_tools) in detail.
19
+
20
+
21
+ ## Support Platform
22
+ - AX650
23
+
24
+ ## End-of-board inference time
25
+ | Stage | Time |
26
+ |------|------|
27
+ | image_encoder | 125.197 ms |
28
+ | text_encoder | 10.817 ms |
29
+
30
+ ## How to use
31
+
32
+ Download all files from this repository to the device
33
+
34
+ Run the following command:
35
+ ```bash
36
+ python3 run_axmodel.py
37
+ ```
38
+ Model input and output examples are as follows:
39
+ 1. the image you want to input:
40
+
41
+ ![](bedroom.jpg)
42
+
43
+ 2. The description of the image content:
44
+
45
+ ```bash
46
+ [
47
+ "一个简约风格的卧室角落,黑色金属衣架上挂着多件米色和白色的衣物,下方架子放着两双浅色鞋子,旁边是一盆绿植,左侧可见一张铺有白色床单和灰色枕头的床。",
48
+ "一个简约风格的卧室角落,黑色金属衣架上挂着多件红色和蓝色的衣物,下方架子放着两双黑色高跟鞋,旁边是一盆绿植,左侧可见一张铺有白色床单和灰色枕头的床。",
49
+ "一个简约风格的卧室角落,黑色金属衣架上挂着多件米色和白色的衣物,下方架子放着两双运动鞋,旁边是一盆仙人掌,左侧可见一张铺有白色床单和灰色枕头的床。",
50
+ "一个繁忙的街头市场,摊位上摆满水果,背景是高楼大厦,人们在喧闹中购物。"
51
+ ]
52
+ ```
53
+
54
+ 3. The similarity between the output of the image encoder and the text encoder is
55
+
56
+ ```bash
57
+ Logits per image: tensor([[9.8757e-01, 4.7755e-03, 7.6510e-03, 1.3484e-14]], dtype=torch.float64)
58
+ ```