Model card for qwen3_vit_306m_merge.qwen_drive_1_0_4b

A Qwen ViT image feature model extracted from Qwen-Drive-1.0-4B. This is the classifier-ready wrapper retaining the native spatial merger and LLM-width projection, followed by average pooling and affine-free LayerNorm.

NOTE: This checkpoint is a native timm remap of the original vision weights, with no additional training. It contains no language-model weights or trained image-classification head.

Model Notes

• Image inputs repeat one frame across the original temporal patch kernel. The temporal Conv3d weights are summed into a Conv2d for this image-only implementation.
• The backbone uses GELU-tanh MLPs, learned absolute positions and axial 2D RoPE. Absolute positions are interpolated for the input grid; RoPE is regenerated at each size.
• The timm transforms normalize RGB pixels using mean=(0.5, 0.5, 0.5) and std=(0.5, 0.5, 0.5). Rectangular inputs are supported. Each image dimension must be divisible by 16; any variant using the 2×2 merger requires divisibility by 32.
• forward_features() returns projected NLC tokens from the native merger. encoder.forward_features() returns raw NHWC backbone features. forward() returns pooled image embeddings until a classification head is added.

Model Details

• Model Type: Image Feature Encoder
• Model Stats:
  • Params (M): 332.7
  • GMACs: 974.8
  • Activations (M): 2610.9
  • Image size: 768 x 768
• Source revision: 28484089a7cc8c335cf5089fb0745cf7c49b6eaa
• License source: https://raw.githubusercontent.com/QwenLM/Qwen-Drive-1.0/28091c1532e869bc7aee91fc0aef6b3e6fd0b2e0/LICENSE
• Original: https://huggingface.co/Qwen/Qwen-Drive-1.0-4B
• License: Apache 2.0
• Backbone width: 1024
• Projection width: 2560
• Papers:
  • Qwen-Drive-1.0: An Initial Step towards a Vision-Language Foundation Model for Autonomous Driving: https://arxiv.org/abs/2609.00111
  • PyTorch Image Models: https://github.com/huggingface/pytorch-image-models

Model Usage

Image Features

import torch
import timm
from PIL import Image

model = timm.create_model('hf-hub:timm/qwen3_vit_306m_merge.qwen_drive_1_0_4b', pretrained=True).eval()
data_config = timm.data.resolve_model_data_config(model)
transform = timm.data.create_transform(**data_config, is_training=False)
image = Image.open('image.jpg').convert('RGB')
x = transform(image).unsqueeze(0)

with torch.inference_mode():
    output = model(x)  # (1, 2560): image embeddings
    features = model.forward_features(x)  # (1, 576, 2560): projected spatial tokens (NLC)

Intermediate Feature Maps

with torch.inference_mode():
    maps = model.forward_intermediates(
        x, indices=3, output_fmt='NCHW', intermediates_only=True,
    )
for feature_map in maps:
    print(feature_map.shape)  # (1, 1024, 48, 48)

Classification Fine-tuning

model = timm.create_model(
    'hf-hub:timm/qwen3_vit_306m_merge.qwen_drive_1_0_4b', pretrained=True, num_classes=45,
)
logits = model(x)  # (1, 45)

The new linear head is randomly initialized and must be trained on your target dataset.

Citation

@misc{zhou2026qwendrive10initialstepvisionlanguage,
  title={Qwen-Drive-1.0: An Initial Step towards a Vision-Language Foundation Model for Autonomous Driving},
  author={Xin Zhou and Zongchuang Zhao and Zhibo Yang and Mingsheng Li and Humen Zhong and Shuai Bai and Du Chu and Ruizhe Chen and Zhaohai Li and Jun Tang and Qiuyue Wang and Mingkun Yang and Jiazhao Zhang and Dayiheng Liu and Dingkang Liang and Xiang Bai},
  year={2026},
  eprint={2609.00111},
  archivePrefix={arXiv},
  primaryClass={cs.CV},
  url={https://arxiv.org/abs/2609.00111}
}

@misc{rw2019timm,
  author = {Ross Wightman},
  title = {PyTorch Image Models},
  year = {2019},
  publisher = {GitHub},
  journal = {GitHub repository},
  doi = {10.5281/zenodo.4414861},
  howpublished = {\url{https://github.com/huggingface/pytorch-image-models}}
}