README_CSV_DEPLOYMENT.md

CSV-First Deployment for Hugging Face Spaces

This document explains how to deploy your Optimus Agent Performance dashboard to Hugging Face Spaces using a CSV-first approach to avoid API rate limiting issues.

Overview

The CSV-first approach works by:

1. Local Data Generation: Run data fetching and preprocessing locally where you have full API access
2. CSV Export: Save preprocessed data to CSV files
3. Space Deployment: Upload CSV files to your Hugging Face Space
4. CSV-First Loading: The Space app prioritizes loading from CSV files over making API calls

Files Added for CSV-First Deployment

1. load_from_csv.py

Contains functions to load data from CSV files:

• load_apr_data_from_csv() - Loads APR data from CSV
• load_roi_data_from_csv() - Loads ROI data from CSV
• load_statistics_from_csv() - Loads statistics from CSV
• check_csv_data_availability() - Checks which CSV files are available
• get_data_freshness_info() - Analyzes how fresh the CSV data is

2. generate_csv_for_space.py

Interactive script to generate CSV files locally:

• Fetches fresh data from the API
• Applies all preprocessing (including initial_value_fixer.py)
• Saves to CSV files ready for Space deployment
• Provides deployment guidance

3. Modified app.py

Updated visualization functions with CSV-first logic:

• generate_apr_visualizations() - Now tries CSV first, falls back to API
• generate_roi_visualizations() - Now tries CSV first, falls back to API
• Imports CSV loading functions
• Maintains backward compatibility

Deployment Workflow

Step 1: Generate CSV Files Locally

Run the CSV generation script on your local machine:

python generate_csv_for_space.py

This will:

• Check existing CSV files and their freshness
• Fetch fresh data from the API (no rate limits locally)
• Apply preprocessing using initial_value_fixer.py
• Save CSV files: optimus_apr_values.csv, optimus_apr_statistics.csv, optimus_roi_values.csv

Step 2: Upload Files to Hugging Face Space

Upload these files to your Hugging Face Space repository:

• app.py (modified with CSV-first logic)
• load_from_csv.py (new CSV loading functions)
• initial_value_fixer.py (existing preprocessing)
• fetch_and_preprocess_data.py (existing data functions)
• optimus_apr_values.csv (generated data)
• optimus_apr_statistics.csv (generated statistics)
• optimus_roi_values.csv (generated data)

Step 3: Deploy Space

Your Space will now:

1. Try CSV first: Load data from uploaded CSV files
2. Fast loading: No API calls needed, instant visualization
3. Fallback: If CSV files are missing, fall back to API (with rate limits)
4. Error handling: Show appropriate messages if both CSV and API fail

CSV File Structure

APR Data (optimus_apr_values.csv)

apr,adjusted_apr,timestamp,agent_id,agent_name,is_dummy,metric_type,roi,address
15.5,12.3,2025-06-14 10:30:00,123,Agent_1,False,APR,0.85,0x123...

ROI Data (optimus_roi_values.csv)

roi,timestamp,agent_id,agent_name,is_dummy,metric_type
0.85,2025-06-14 10:30:00,123,Agent_1,False,ROI

Statistics (optimus_apr_statistics.csv)

agent_id,agent_name,total_points,apr_points,avg_apr,avg_adjusted_apr,latest_timestamp
123,Agent_1,100,50,15.5,12.3,2025-06-14 10:30:00

Benefits

✅ Advantages

• No Rate Limits: Space doesn't make API calls
• Fast Loading: Instant visualization from CSV
• Reliable: No dependency on external API availability
• Preprocessed: Data includes all transformations from initial_value_fixer.py
• Fresh Data: Update CSV files as needed locally

⚠️ Considerations

• Manual Updates: Need to regenerate CSV files for fresh data
• File Size: CSV files add to Space storage
• Data Staleness: CSV data becomes stale over time

Updating Data

To update your Space with fresh data:

1. Run locally: python generate_csv_for_space.py
2. Upload new CSV files to your Space repository
3. Space auto-updates with new data

Monitoring

Data Freshness

The app logs data freshness information:

from load_from_csv import get_data_freshness_info
freshness = get_data_freshness_info()
# Shows how old the CSV data is

CSV Availability

Check which CSV files are available:

from load_from_csv import check_csv_data_availability
availability = check_csv_data_availability()
# Shows file status, size, modification date

Troubleshooting

CSV Files Not Loading

1. Check file names match exactly: optimus_apr_values.csv, optimus_roi_values.csv
2. Verify CSV files are in the Space root directory
3. Check Space logs for CSV loading errors

Data Issues

1. Regenerate CSV files locally with fresh API data
2. Verify preprocessing ran correctly in generate_csv_for_space.py
3. Check CSV file structure matches expected format

Fallback to API

If CSV loading fails, the app falls back to API calls:

1. This may trigger rate limits in Hugging Face Spaces
2. Check Space logs for API errors
3. Ensure CSV files are properly uploaded

Best Practices

1. Regular Updates: Run generate_csv_for_space.py daily or weekly
2. Monitor Logs: Check Space logs for CSV loading status
3. Backup CSV: Keep local copies of generated CSV files
4. Test Locally: Test CSV loading locally before deploying
5. Version Control: Track CSV generation timestamps

Example Space Configuration

Your Space should include these files:

├── app.py                          # Modified with CSV-first logic
├── load_from_csv.py               # CSV loading functions
├── initial_value_fixer.py         # Preprocessing logic
├── fetch_and_preprocess_data.py   # Data functions
├── optimus_apr_values.csv         # Generated APR data
├── optimus_apr_statistics.csv     # Generated statistics
├── optimus_roi_values.csv         # Generated ROI data
├── requirements.txt               # Dependencies
└── README.md                      # Space documentation

This approach ensures your Hugging Face Space runs reliably without hitting API rate limits while maintaining all the preprocessing and visualization features of your application.