SOV-Transcribe-1-DFlash

This is a DFlash draft model for speculative decoding with SOV-Transcribe-1. It proposes several tokens at a time for the main model to verify, which speeds up transcription without changing its accuracy beyond run-to-run variation. It cannot transcribe audio on its own.

Model summary

Property	Details
Parameters	430M
Architecture	5-layer DFlash drafter conditioned on main-model layers 1, 6, 11, 17, 22 and 27
Precision	BF16
Draft length	15 tokens
License	Apache 2.0

Usage

Add the draft to a vLLM deployment of SOV-Transcribe-1:

--speculative-config '{"method": "dflash", "model": "salesforce-sovereign-intelligence/SOV-Transcribe-1-DFlash", "num_speculative_tokens": 15}' --max-num-seqs 64

vLLM runs SOV-Transcribe-1 with the vLLM plugin described in the main model card until native support is released; --max-num-seqs 64 keeps vLLM's start-up memory profiling within a 96 GB GPU.

With Transformers, load it as the assistant model of SOV-Transcribe-1 (see the main model card for model and inputs):

from transformers import AutoModel

draft = AutoModel.from_pretrained(
    "salesforce-sovereign-intelligence/SOV-Transcribe-1-DFlash", trust_remote_code=True, dtype=model.dtype, device_map=model.device
)
outputs = model.generate(**inputs, assistant_model=draft, max_new_tokens=8192)

Limitations

• Works only with SOV-Transcribe-1.
• The speedup depends on the workload: about 2–3× for one request at a time, less for clips under 10 seconds and for long audio.