how do i interpret the results

#2
by cuiyi0326 - opened

I'm using the default policies and got the following results. How should i interpret the results? if i understand correctly, the logits tensor shape (3,2) corresponds to 3 default policies and every police has two values yes and no. so my question is why do i always get two [nan, nan]?

ShieldGemma2ImageClassifierOutputWithNoAttention(loss=None, logits=tensor([[ nan, nan],
[ nan, nan],
[35.1562, 53.1972]], device='cuda:0'), hidden_states=None, probabilities=tensor([[ nan, nan],
[ nan, nan],
[1.4618e-08, 1.0000e+00]], device='cuda:0'))

Use Latest Version Transformers,I get results like this: tensor([[5.3998e-02, 9.4600e-01],
[3.6518e-01, 6.3482e-01],
[1.8238e-08, 1.0000e+00]]),The probability of looking pornographic, dangerous, bloody and violent

Google org

Hello, the policy outputs scores for Yes and No respectively for each policy (policy is provided by text as you put) @BITDDD
@cuiyi0326 can you put your inference code here for me to reproduce?

Google org

Hello again, I built a notebook on how to properly use ShieldGemma 2 with custom policies: https://github.com/merveenoyan/smol-vision/blob/main/ShieldGemma_2_for_Vision_LM_Safety.ipynb

Your need to confirm your account before you can post a new comment.

Sign up or log in to comment