deepghs/siglip_beta

WARNING: Do not consider anything in this repo production ready.

Checkpoints

• siglip_swinv2_base_2025_02_22_18h56m54s
  Text encoder trained on top of frozen SmilingWolf/wd-swinv2-tagger-v3, so pretty much SigLIT style. Compatible with existing DeepGHS indexes/embeddings.
• siglip_swinv2_base_2025_05_02_22h02m36s
  Based on siglip_swinv2_base_2025_02_22_18h56m54s, with unfrozen image encoder. So SigLIP with warm start, I guess.
• siglip_eva02_base_2025_05_02_21h53m54s
  A test with a different architecture, trained from scratch using SigLIP.

Usage Example

See deepghs/search_image_by_image_or_text for example usage.

Compatibility

The checkpoints in this repo have been structured for compatibility with the dghs-imgutils package.

You can run it locally with the following 2 approaches

Infer with Code

pip install dghs-imgutils>=0.17.0

from imgutils.generic import siglip_predict

pred = siglip_predict(
    images=[
        'https://huggingface.co/datasets/narugo1992/nzb_files/resolve/main/eshuushuu_51.webp',
    ],
    texts=[
        # short tags
        '1girl',
        '1boy',
        'orange_hair',
        'blue_hair',

        # long text
        '1girl, solo, thighhighs, orange_shirt, twintails, brown_hair, hair_bun, long_hair, double_bun, '
        'zettai_ryouiki, jar, sitting, bow, school_uniform, long_sleeves, smile, pink_bow, skirt, orange_skirt, '
        'very_long_hair, black_thighhighs, orange_dress, miniskirt',
        'food, halo, red_eyes, side_ponytail, skirt, macaron, pink_hair, sailor_collar, holding, '
        'black_sailor_collar, cake, long_hair, drumsticks, black_skirt, pleated_skirt, pink_halo, 1girl, '
        'ahoge, red_neckerchief, chibi, neckerchief, long_sleeves, holding_food, sash, blush, holding_drumsticks, '
        'multiple_views, white_cardigan, looking_at_viewer,'
    ],
    repo_id='deepghs/siglip_beta',
    model_name='smilingwolf/siglip_swinv2_base_2025_02_22_18h56m54s'
)
print(pred)
# [[2.5059912e-02 1.7571157e-04 2.1646977e-03 1.8494057e-04 1.0000000e+00
#   3.8877626e-15]]

Launch a Gradio Demo

pip install dghs-imgutils[demo]>=0.17.0

from imgutils.generic import SigLIPModel

SigLIPModel(
    repo_id='deepghs/siglip_beta',
).launch_demo(
    default_model_name='smilingwolf/siglip_swinv2_base_2025_02_22_18h56m54s'
)