inoki-giskard/scan-report-temp · Report for mrm8488/distilroberta-finetuned-financial-news-sentiment-analysis

Hi Team,

This is a report from Giskard Bot Scan 🐢.

We have identified 1 potential vulnerabilities in your model based on an automated scan.

This automated analysis evaluated the model on the dataset financial_phrasebank (subset sentences_allagree, split train).

👉Robustness issues (1)

Vulnerability	Level	Data slice	Metric	Transformation	Deviation
Robustness	major 🔴	—	Fail rate = 0.301	Transform to uppercase	301/1000 tested samples (30.1%) changed prediction after perturbation

🔍✨Examples

When feature “text” is perturbed with the transformation “Transform to uppercase”, the model changes its prediction in 30.1% of the cases. We expected the predictions not to be affected by this transformation.

	text	Transform to uppercase(text)	Original prediction	Prediction after perturbation
118	Finnish lifting equipment maker Konecranes Oyj said on July 30 , 2008 that its net profit rose to 71.2 mln euro ( $ 111.1 mln ) for the first half of 2008 from 57.1 mln euro ( $ 89.1 mln ) for the same period of 2007 .	FINNISH LIFTING EQUIPMENT MAKER KONECRANES OYJ SAID ON JULY 30 , 2008 THAT ITS NET PROFIT ROSE TO 71.2 MLN EURO ( $ 111.1 MLN ) FOR THE FIRST HALF OF 2008 FROM 57.1 MLN EURO ( $ 89.1 MLN ) FOR THE SAME PERIOD OF 2007 .	positive (p = 1.00)	neutral (p = 1.00)
875	Net sales will , however , increase from 2005 .	NET SALES WILL , HOWEVER , INCREASE FROM 2005 .	positive (p = 1.00)	neutral (p = 1.00)
823	In the end of 2006 , the number of outlets will rise to 60-70 .	IN THE END OF 2006 , THE NUMBER OF OUTLETS WILL RISE TO 60-70 .	positive (p = 1.00)	neutral (p = 1.00)

Checkout out the Giskard Space and test your model.

Disclaimer: it's important to note that automated scans may produce false positives or miss certain vulnerabilities. We encourage you to review the findings and assess the impact accordingly.