In-House Evaluation Is Not Enough: Towards Robust Third-Party Flaw Disclosure for General-Purpose AI
Paper
•
2503.16861
•
Published
Journalists on Hugging Face
community
AI & ML interests
Democratizing access to useful AI tools and resources for journalists
Recent Activity
View all activity
JournalistsonHF's activity
evijit
authored
a
paper
6 days ago
evijit
authored
2
papers
20 days ago
Post
3181
Today in Privacy & AI Tooling - introducing a nifty new tool to examine where data goes in open-source apps on 🤗
HF Spaces have tons (100Ks!) of cool demos leveraging or examining AI systems - and because most of them are OSS we can see exactly how they handle user data 📚🔍
That requires actually reading the code though, which isn't always easy or quick! Good news: code LMs have gotten pretty good at automatic review, so we can offload some of the work - here I'm using Qwen/Qwen2.5-Coder-32B-Instruct to generate reports and it works pretty OK 🙌
The app works in three stages:
1. Download all code files
2. Use the Code LM to generate a detailed report pointing to code where data is transferred/(AI-)processed (screen 1)
3. Summarize the app's main functionality and data journeys (screen 2)
4. Build a Privacy TLDR with those inputs
It comes with a bunch of pre-reviewed apps/Spaces, great to see how many process data locally or through (private) HF endpoints 🤗
Note that this is a POC, lots of exciting work to do to make it more robust, so:
- try it: yjernite/space-privacy
- reach out to collab: yjernite/space-privacy
HF Spaces have tons (100Ks!) of cool demos leveraging or examining AI systems - and because most of them are OSS we can see exactly how they handle user data 📚🔍
That requires actually reading the code though, which isn't always easy or quick! Good news: code LMs have gotten pretty good at automatic review, so we can offload some of the work - here I'm using Qwen/Qwen2.5-Coder-32B-Instruct to generate reports and it works pretty OK 🙌
The app works in three stages:
1. Download all code files
2. Use the Code LM to generate a detailed report pointing to code where data is transferred/(AI-)processed (screen 1)
3. Summarize the app's main functionality and data journeys (screen 2)
4. Build a Privacy TLDR with those inputs
It comes with a bunch of pre-reviewed apps/Spaces, great to see how many process data locally or through (private) HF endpoints 🤗
Note that this is a POC, lots of exciting work to do to make it more robust, so:
- try it: yjernite/space-privacy
- reach out to collab: yjernite/space-privacy
BrigitteTousi
posted
an
update
about 1 month ago
Post
3055
AI agents are transforming how we interact with technology, but how sustainable are they? 🌍
Design choices — like model size and structure — can massively impact energy use and cost. ⚡💰 The key takeaway: smaller, task-specific models can be far more efficient than large, general-purpose ones.
🔑 Open-source models offer greater transparency, allowing us to track energy consumption and make more informed decisions on deployment. 🌱 Open-source = more efficient, eco-friendly, and accountable AI.
Read our latest, led by @sasha with assists from myself + @yjernite 🤗
https://huggingface.co/blog/sasha/ai-agent-sustainability
Design choices — like model size and structure — can massively impact energy use and cost. ⚡💰 The key takeaway: smaller, task-specific models can be far more efficient than large, general-purpose ones.
🔑 Open-source models offer greater transparency, allowing us to track energy consumption and make more informed decisions on deployment. 🌱 Open-source = more efficient, eco-friendly, and accountable AI.
Read our latest, led by @sasha with assists from myself + @yjernite 🤗
https://huggingface.co/blog/sasha/ai-agent-sustainability
- 1 reply
fdaudens
updated
a
Space
about 1 month ago
fdaudens
published
a
Space
about 1 month ago
evijit
authored
a
paper
about 1 month ago
fdaudens
updated
a
Space
about 1 month ago
louisbrulenaudet
posted
an
update
about 2 months ago
Post
1018
I’ve just released logfire-callback on PyPI, designed to facilitate monitoring of Hugging Face Transformer training loops using Pydantic Logfire 🤗
The callback will automatically log training start with configuration parameters, periodic metrics and training completion ⏱️
Install the package using pip:
First, ensure you have a Logfire API token and set it as an environment variable:
Then use the callback in your training code:
If you have any feedback, please reach out at @louisbrulenaudet
The callback will automatically log training start with configuration parameters, periodic metrics and training completion ⏱️
Install the package using pip:
pip install logfire-callbackFirst, ensure you have a Logfire API token and set it as an environment variable:
export LOGFIRE_TOKEN=your_logfire_tokenThen use the callback in your training code:
from transformers import Trainer, TrainingArguments
from logfire_callback import LogfireCallback
# Initialize your model, dataset, etc.
training_args = TrainingArguments(
output_dir="./results",
num_train_epochs=3,
# ... other training arguments
)
trainer = Trainer(
model=model,
args=training_args,
train_dataset=train_dataset,
callbacks=[LogfireCallback()] # Add the Logfire callback here
)
trainer.train()If you have any feedback, please reach out at @louisbrulenaudet
fdaudens
updated
2
collections
about 2 months ago
BrigitteTousi
posted
an
update
about 2 months ago
Post
3418
LeRobot goes to driving school! 🚗🚗🚗
Hugging Face just announced a new collab with Yaak to bring the largest open-source self-driving dataset to LeRobot!
Major kudos to HF's @cadene , as well as @sandhawalia , @Shnissen and the Yaak team!
Check out the blog post here: https://huggingface.co/blog/lerobot-goes-to-driving-school
Hugging Face just announced a new collab with Yaak to bring the largest open-source self-driving dataset to LeRobot!
Major kudos to HF's @cadene , as well as @sandhawalia , @Shnissen and the Yaak team!
Check out the blog post here: https://huggingface.co/blog/lerobot-goes-to-driving-school
-
- 1 reply
BrigitteTousi
posted
an
update
about 2 months ago
Post
3738
Regardless of X being down or not, so glad I can rely on HF Posts for AI news ❤️🤗
- 1 reply
fdaudens
updated
a
Space
2 months ago
yjernite
authored
a
paper
2 months ago
fdaudens
published
a
Space
2 months ago
DmitryRyumin
posted
an
update
3 months ago
Post
3838
🚀🎭🌟 New Research Alert - WACV 2025 (Avatars Collection)! 🌟🎭🚀
📄 Title: EmoVOCA: Speech-Driven Emotional 3D Talking Heads 🔝
📝 Description: EmoVOCA is a data-driven method for generating emotional 3D talking heads by combining speech-driven lip movements with expressive facial dynamics. This method has been developed to overcome the limitations of corpora and to achieve state-of-the-art animation quality.
👥 Authors: @FedeNoce , Claudio Ferrari, and Stefano Berretti
📅 Conference: WACV, 28 Feb – 4 Mar, 2025 | Arizona, USA 🇺🇸
📄 Paper: https://arxiv.org/abs/2403.12886
🌐 Github Page: https://fedenoce.github.io/emovoca/
📁 Repository: https://github.com/miccunifi/EmoVOCA
🚀 CVPR-2023-24-Papers: https://github.com/DmitryRyumin/CVPR-2023-24-Papers
🚀 WACV-2024-Papers: https://github.com/DmitryRyumin/WACV-2024-Papers
🚀 ICCV-2023-Papers: https://github.com/DmitryRyumin/ICCV-2023-Papers
📚 More Papers: more cutting-edge research presented at other conferences in the DmitryRyumin/NewEraAI-Papers curated by @DmitryRyumin
🚀 Added to the Avatars Collection: DmitryRyumin/avatars-65df37cdf81fec13d4dbac36
🔍 Keywords: #EmoVOCA #3DAnimation #TalkingHeads #SpeechDriven #FacialExpressions #MachineLearning #ComputerVision #ComputerGraphics #DeepLearning #AI #WACV2024
📄 Title: EmoVOCA: Speech-Driven Emotional 3D Talking Heads 🔝
📝 Description: EmoVOCA is a data-driven method for generating emotional 3D talking heads by combining speech-driven lip movements with expressive facial dynamics. This method has been developed to overcome the limitations of corpora and to achieve state-of-the-art animation quality.
👥 Authors: @FedeNoce , Claudio Ferrari, and Stefano Berretti
📅 Conference: WACV, 28 Feb – 4 Mar, 2025 | Arizona, USA 🇺🇸
📄 Paper: https://arxiv.org/abs/2403.12886
🌐 Github Page: https://fedenoce.github.io/emovoca/
📁 Repository: https://github.com/miccunifi/EmoVOCA
🚀 CVPR-2023-24-Papers: https://github.com/DmitryRyumin/CVPR-2023-24-Papers
🚀 WACV-2024-Papers: https://github.com/DmitryRyumin/WACV-2024-Papers
🚀 ICCV-2023-Papers: https://github.com/DmitryRyumin/ICCV-2023-Papers
📚 More Papers: more cutting-edge research presented at other conferences in the DmitryRyumin/NewEraAI-Papers curated by @DmitryRyumin
🚀 Added to the Avatars Collection: DmitryRyumin/avatars-65df37cdf81fec13d4dbac36
🔍 Keywords: #EmoVOCA #3DAnimation #TalkingHeads #SpeechDriven #FacialExpressions #MachineLearning #ComputerVision #ComputerGraphics #DeepLearning #AI #WACV2024
- 1 reply
louisbrulenaudet
posted
an
update
3 months ago
Post
3361
I am pleased to introduce my first project built upon Hugging Face’s smolagents framework, integrated with Alpaca for financial market analysis automation 🦙🤗
The project implements technical indicators such as the Relative Strength Index (RSI) and Bollinger Bands to provide momentum and volatility analysis. Market data is retrieved through the Alpaca API, enabling access to historical price information across various timeframes.
AI-powered insights are generated using Hugging Face’s inference API, facilitating the analysis of market trends through natural language processing with DuckDuckGo search integration for real-time sentiment analysis based on financial news 🦆
Link to the GitHub project: https://github.com/louisbrulenaudet/agentic-market-tool
The project implements technical indicators such as the Relative Strength Index (RSI) and Bollinger Bands to provide momentum and volatility analysis. Market data is retrieved through the Alpaca API, enabling access to historical price information across various timeframes.
AI-powered insights are generated using Hugging Face’s inference API, facilitating the analysis of market trends through natural language processing with DuckDuckGo search integration for real-time sentiment analysis based on financial news 🦆
Link to the GitHub project: https://github.com/louisbrulenaudet/agentic-market-tool
fdaudens
updated
a
collection
3 months ago