---
license: llama3.2
language:
- en
base_model:
- meta-llama/Llama-3.2-3B-Instruct
pipeline_tag: text-generation
tags:
- Speech Recognition
- ATC
- Unsloth
- LoRA-Merged
---

# ATC Communication Expert Model (Merged)

A fine-tuned model specialized in improving and analyzing Air Traffic Control (ATC) communications, with LoRA adapters merged into the base model.

## Model Details

### Model Description

This model is a fine-tuned version of meta-llama/Llama-3.2-3B-Instruct with merged LoRA adapters, optimized for processing Air Traffic Control communications. It can:

- Improve raw ATC transcripts with proper punctuation and formatting
- Identify communication intentions (pilot requests, ATC instructions, etc.)
- Extract key information such as flight numbers, altitudes, headings, and other numerical data
- Analyze speaker roles and communication patterns

The model was created by merging LoRA adapters (fine-tuned on ATC communications) into the Llama 3B base model, creating a unified model optimized for this specialized domain.

- **Developed by:** [Sang-Buster](https://github.com/Sang-Buster)
- **Model type:** Llama 3B with merged LoRA adapters
- **Language(s):** English, specialized for ATC terminology
- **License:** Same as the base model
- **Finetuned from model:** meta-llama/Llama-3.2-3B-Instruct

## Uses

### Direct Use

This model is intended for:
- Transcribing and formatting raw ATC communications
- Training ATC communication skills
- Analyzing ATC communication patterns
- Extracting structured data from ATC communications
- Educational purposes for those learning ATC communication protocols

### Downstream Use

The model can be integrated into:
- Air traffic management training systems
- Communication analysis tools
- ATC transcript post-processing pipelines
- Aviation safety monitoring systems
- Radio communication enhancement systems

### Out-of-Scope Use

This model is not suitable for:
- Real-time ATC operations or safety-critical decision-making
- Full language translation (it's specialized for ATC terminology only)
- General language processing outside the ATC domain
- Any application where model errors could impact flight safety

## Bias, Risks, and Limitations

- The model is specialized for ATC communications and may not perform well on general text
- It may have limitations with accents or non-standard ATC phraseology
- Performance depends on audio transcription quality for real-world applications
- Not intended for safety-critical applications without human verification
- May have biases based on the training data distribution

### Recommendations

- Always have human verification for safety-critical applications
- Use in conjunction with standard ATC protocols, not as a replacement
- Provide clear domain context for optimal performance
- Test thoroughly with diverse ATC communications before deployment
- Consider fine-tuning further on your specific ATC subdomain if needed

## How to Get Started with the Model

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

# Load the model and tokenizer
model = AutoModelForCausalLM.from_pretrained(
    "atc_llama_merged",
    torch_dtype="auto",
    device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained("atc_llama_merged")

# Process an ATC message
instruction = "As an ATC communication expert, improve this transcript and analyze its intentions and data."
message = "southwest five niner two turn left heading three four zero descend and maintain flight level two five zero"

prompt = f"<|begin_of_text|><|header_start|>user<|header_end|>\n\n{instruction}\n\nOriginal: {message}<|eot|><|header_start|>assistant<|header_end|>\n\n"
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

# Generate improved transcript and analysis
outputs = model.generate(**inputs, max_new_tokens=512, do_sample=False)
response = tokenizer.decode(outputs[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True)
print(response)
```

## Model Creation Process

### Base Model and Adapters

- **Base model:** meta-llama/Llama-3.2-3B-Instruct
- **Adapter source:** LoRA adapters fine-tuned on ATC communications data
- **Merge method:** PEFT adapter merging into base model weights

### Merging Procedure

The model creation involved:
1. Loading the base Llama 3B model
2. Loading LoRA adapters fine-tuned on ATC communications data
3. Merging the adapters into the base model's weights
4. Saving the resulting unified model

## Evaluation
 
### Testing

The model should be tested on diverse ATC communications, including:
- Clearances and instructions
- Pilot requests and reports
- Emergency communications
- Different accents and speaking patterns

## Technical Specifications

### Model Architecture and Objective

- **Base architecture:** meta-llama/Llama-3.2-3B-Instruct
- **Adaptation method:** LoRA adapters merged into base weights
- **Training objective:** Improving and analyzing ATC communications

### Model Card Contact

For issues or questions about this model, please open a discussion in the repository.