---
title: VQA
emoji: 🚀
colorFrom: gray
colorTo: yellow
sdk: docker
pinned: false
license: mit
short_description: VQA API Endpoint
---

Check out the configuration reference at
https://huggingface.co/docs/hub/spaces-config-reference

# VizWiz Visual Question Answering API

This repository contains a FastAPI backend for a Visual Question Answering (VQA)
system trained on the VizWiz dataset.

## Features

- Upload images and ask questions about them
- Get answers with confidence scores
- Session management for asking multiple questions about the same image
- Health check endpoint for monitoring
- API documentation with Swagger UI

## Project Structure

```
project_root/
├── app/
│   ├── main.py               # Main FastAPI application
│   ├── models/               # Model definitions
│   │   ├── __init__.py
│   │   └── vqa_model.py      # VQA model implementation
│   ├── routers/              # API route definitions
│   │   ├── __init__.py
│   │   └── vqa.py            # VQA-related endpoints
│   ├── services/             # Business logic
│   │   ├── __init__.py
│   │   ├── model_service.py  # Model loading and inference
│   │   └── session_service.py # Session management
│   ├── utils/                # Utility functions
│   │   ├── __init__.py
│   │   └── image_utils.py    # Image processing utilities
│   └── config.py             # Application configuration
├── models/                   # Directory for model files
├── uploads/                  # Directory for uploaded images
├── .env                      # Environment variables
└── requirements.txt          # Project dependencies
```

## Installation

1. Clone the repository:

```bash
git clone https://github.com/dixisouls/vizwiz-vqa-api.git
cd vizwiz-vqa-api
```

2. Create a virtual environment:

```bash
python -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate
```

3. Install dependencies:

```bash
pip install -r requirements.txt
```

4. Create necessary directories:

```bash
mkdir -p models uploads
```

5. Place your trained model in the `models` directory.

6. Update the `.env` file with your configuration.

## Running the Application

```bash
uvicorn app.main:app --host 0.0.0.0 --port 8000 --reload
```

The API will be available at http://localhost:8000.

API documentation is available at:

- Swagger UI: http://localhost:8000/docs
- ReDoc: http://localhost:8000/redoc

## API Endpoints

### Health Check

```
GET /health
```

Returns the health status of the API.

### Upload Image

```
POST /api/vqa/upload
```

Upload an image and create a new session.

### Ask Question

```
POST /api/vqa/ask
```

Ask a question about an uploaded image.

### Get Session

```
GET /api/vqa/session/{session_id}
```

Get session information including question history.

### Reset Session

```
DELETE /api/vqa/session/{session_id}
```

Reset a session to start fresh.

## Environment Variables

- `DEBUG`: Enable debug mode (default: False)
- `MODEL_PATH`: Path to the trained model (default: ./models/vqa_model_best.pt)
- `TEXT_MODEL`: Name of the text model (default: bert-base-uncased)
- `VISION_MODEL`: Name of the vision model (default:
  google/vit-base-patch16-384)
- `HUGGINGFACE_TOKEN`: Hugging Face API token
- `UPLOAD_DIR`: Directory for uploaded images (default: ./uploads)

## License

[MIT License](LICENSE)