Safetensors
qwen2

QWEN2.5-32B-2600s-FP8: Advanced Multilingual Translation Model

Overview

FINGU-AI/QWEN2.5-32B-2600s-FP8 is a fine-tuned version of Qwen 2.5 32B, specifically optimized for multilingual translation across 16 different languages. This model has been extensively fine-tuned to enhance its translation capabilities, making it competitive with high-tier models like 72B in terms of translation accuracy and fluency.

Fine-Tuning Process

Data Collection

To improve the model's understanding and translation capabilities, we curated and synthesized a large dataset consisting of:

  • High-quality multilingual conversational datasets.
  • Real-world dialogues spanning general, business, and technical domains.
  • Translated datasets covering diverse linguistic structures and idiomatic expressions.

Multilingual Enhancement

To advance its translation capabilities, we leveraged:

  • Translation Expansion: The collected dataset was translated into 16 different languages to ensure robust multilingual performance.
  • Benchmarking Against High-Tier Models: We utilized state-of-the-art translation models, including Gemini and other top-ranking translation models with high BLEU and COMET scores, to refine our translation quality.
  • Reinforcement Learning with Human Feedback (RLHF): Translation outputs were evaluated and iteratively improved based on feedback from native speakers and linguistic experts.

Training and Optimization

  • Base Model: Qwen 2.5 32B FP8
  • Fine-Tuning Framework: LoRA + QLoRA for efficient training
  • Batch Size: Optimized for multi-GPU environments
  • Precision: FP8 for efficient computation without sacrificing performance
  • Training Iterations: Over 2600 steps on multi-H100 GPUs

Key Improvements

  • Enhanced Multilingual Translation: The model now achieves translation fluency comparable to 72B models across multiple language pairs.
  • Diverse Conversational Understanding: Improved ability to process and generate accurate translations for various contexts, including business, casual, and formal speech.
  • Optimized for Low-Latency Inference: Fine-tuned with efficiency in mind, making it suitable for real-time translation applications.

Performance Evaluation

The model was evaluated using:

  • BLEU, COMET, and chrF scores: To measure translation quality across multiple languages.
  • Human Evaluation: Involving bilingual speakers and linguistic professionals to validate accuracy and fluency.
  • Comparisons with SOTA Models: Benchmarked against high-performance models like GPT-4, Gemini, and LLaMA-3 to ensure top-tier translation quality.

Usage

This model is suitable for:

  • High-quality machine translation across multiple languages
  • Conversational AI with multilingual capabilities
  • Cross-lingual content generation and customer support
  • NLP applications requiring robust and accurate translation

Limitations

  • While translation quality is highly competitive, niche dialects or highly technical documents may require additional fine-tuning.
  • Performance may vary slightly depending on the deployment environment and inference settings.

Citation

If you use this model, please cite:

@misc{FINGU-AI-QWEN2.5-32B-2600s-FP8,
  author = {FINGU-AI},
  title = {FINGU-AI/QWEN2.5-32B-2600s-FP8: Advanced Multilingual Translation Model},
  year = {2025},
  publisher = {Hugging Face},
  url = {https://huggingface.co/FINGU-AI/QWEN2.5-32B-2600s-FP8}
}

License

This model follows the licensing terms of the original Qwen 2.5 32B model. Ensure compliance with regional translation regulations before deploying in production environments.

Downloads last month
6
Safetensors
Model size
32.8B params
Tensor type
BF16
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Dataset used to train FINGU-AI/QWEN2.5-32B-2600s