README.md

metadata

title: Odia OCR Annotation + Synthetic Generator
emoji: 🧩
colorFrom: indigo
colorTo: yellow
sdk: docker
sdk_version: 1.0.0
pinned: false

Odia OCR Annotation + Synthetic Text Generator

A unified repository that provides:

• An OCR annotation tool (React frontend + FastAPI backend) to upload images, run OCR via Gemini, edit validated text, and export CSVs.
• A synthetic text generator (exposed via backend API) to render Odia/Sanskrit-like text with realistic paper/effects, including HuggingFace dataset processing.

Repository Structure

• backend/
  • app/main.py: FastAPI app with two routers: /api/ocr and /api/synthetic
  • app/api/routers/ocr.py: OCR endpoints (upload, OCR, annotations import/export)
  • app/api/routers/synthetic.py: Synthetic generation endpoints
  • app/services/: Shared services
    • ocr_processor.py: Gemini OCR
    • annotations.py: CSV/JSON I/O
    • synthetic/: generator modules (config, core, effects, backgrounds, text_renderer, transformations, huggingface_processor)
  • data/: runtime storage
    • uploaded_images/: uploaded images (served at /images)
    • annotations/: annotations.csv and JSON
    • synth_outputs/: generated images and CSVs (served at /static/synthetic)
  • requirements.txt: backend dependencies
• frontend/
  • Vite + React + Tailwind app
  • Routes: /ocr (annotation UI) and /synthetic (generator UI)
• content/static/: NotoSans Oriya fonts used by generator

Run Locally

1. Backend

• pip install -r backend/requirements.txt
• From backend/: uvicorn app.main:app --reload
• Static mounts:
  • /images → backend/data/uploaded_images
  • /static/synthetic → backend/data/synth_outputs

2. Frontend

• cd frontend && npm install && npm run dev
• Open http://localhost:5173
• Use navigation to switch between OCR and Synthetic pages

OCR API (FastAPI)

• POST /api/ocr/upload:
  • Multipart files field: files
  • Stores images in backend/data/uploaded_images
• POST /api/ocr/process:
  • JSON: { "api_key": "<GEMINI_KEY>", "image_filenames": ["img1.png", ...] }
  • Returns: { "img1.png": "extracted text", ... }
• GET /api/ocr/annotations:
  • Returns current annotations, valid/missing images
• POST /api/ocr/save:
  • JSON: { "<filename>": { "extracted_text": "...", "validated_text": "..." } }
  • Saves to CSV and JSON in backend/data/annotations
• POST /api/ocr/import:
  • Multipart: file (CSV), image_folder (e.g., uploaded_images)
  • Validates and returns annotations + image presence
• POST /api/ocr/export:
  • JSON: { annotations: {...}, validated_texts: {...} }
  • Returns a downloadable CSV

Note: Legacy endpoints (/upload/, /process-ocr/, etc.) are temporarily supported for the older UI. Prefer /api/ocr/... going forward.

Synthetic API (FastAPI)

• POST /api/synthetic/generate
  • Modes: single, comprehensive, ultra-realistic, huggingface
  • Request body examples:
    • Non-HF: { "mode": "single", "text": "some Odia text", "output_subdir": "demo_run_01" }
    • HF CSV: { "mode": "huggingface", "dataset_url": "https://.../data.csv", "text_column": "text", "max_samples": 100, "output_subdir": "hf_demo" }
  • Response:
    • Non-HF: { "status": "ok", "output_dir": "/static/synthetic/<job_id>" }
    • HF: { "status": "ok", "output_dir": "/static/synthetic/<job_id>", "csv": "/static/synthetic/<job_id>/dataset.csv", "images_dir": "/static/synthetic/<job_id>/images" }
  • Outputs are stored under backend/data/synth_outputs/<job_id>/ and publicly served at /static/synthetic/<job_id>/....

Fonts

• Generator uses fonts from content/static/.
• Default: NotoSansOriya_Condensed-Regular.ttf (configurable). Ensure the directory exists.

Effects & Styles

• Paper styles: lined paper, old paper, birch, parchment
• Effects: rotation, brightness/contrast/noise/blur, fold/crease, ink bleed, perspective, shadows, morphological ops, scanner artifacts, lens distortion, washboard/cylinder warps

Notes

• The backend expects the Gemini API key to be provided per-request to /api/ocr/process. Do not hardcode keys server-side.
• For HuggingFace datasets, the backend uses datasets when possible, or downloads raw CSV URLs.
• You can browse generated outputs via the links returned by /api/synthetic/generate.

Deploy to Hugging Face Spaces (Docker)

This repo includes a multi-stage Dockerfile to deploy both backend and the built frontend as a single Space.

Steps:

• Create a new Space → Type: Docker
• Push this repository to the Space
• In Space Settings:
  • Enable Persistent Storage
  • (Optional) Add Secrets/Env Vars as needed, e.g., DATA_DIR=/data (default already) and FRONTEND_DIST=/app/frontend_dist
• The container exposes port 7860 by default.

What the image does:

• Builds the frontend (frontend/) and copies the dist/ to /app/frontend_dist
• Installs backend dependencies and runs uvicorn app.main:app from backend/
• Serves:
  • API at /api/...
  • Uploaded images at /images
  • Synthetic outputs at /static/synthetic
  • Frontend SPA at / (served from /app/frontend_dist)

1. Paper Textures: Realistic paper fiber patterns using Perlin noise
2. Aging Effects: Edge darkening and aging patterns
3. Physical Damage: Fold lines, creases, and ink bleeding
4. Scanner Artifacts: Dust, compression artifacts, scanning lines
5. Geometric Distortions: Perspective changes, cylindrical warping
6. Lighting Effects: Shadows and lens distortions

Font Requirements

The generator requires appropriate fonts for text rendering. Default configuration expects:

• Font directory: /content/static/
• Font file: NotoSansOriya_ExtraCondensed-Regular.ttf

You can specify custom fonts using --font-dir and --font parameters.

Performance Tips

• Use --max-samples to limit processing for large datasets
• Disable advanced effects with --no-advanced-effects for faster generation
• Use multiprocessing with --use-multiprocessing for batch jobs
• Adjust image dimensions to balance quality and speed

Error Handling

The package includes comprehensive error handling:

• Graceful fallbacks for missing dependencies
• Detailed logging for debugging
• Validation of input parameters
• Safe handling of malformed datasets

Contributing

The modular structure makes it easy to extend:

• Add new effects in effects.py
• Implement new background styles in backgrounds.py
• Create custom transformations in transformations.py
• Extend dataset processing in huggingface_processor.py

License

[Add your license information here]

---

Note: This is a complete rewrite of the original monolithic code into a modular, extensible package with added HuggingFace dataset processing capabilities.