Let me know appropriate version of Triton, getting triton.compiler.errors
When running the example, I get the triton compiler error during the model.generate as below. (7B-Instruct-AWQ model generate success...)
My env : rtx3090, triton==3.2.0 (also checked 3.3.0), transformers==4.51.3, torch==2.6.0, autoawq==0.2.9, torchvision==0.21.0 with cu12
triton.compiler.errors.CompilationError: at 108:22:
masks_s = masks_sk[:, None] & masks_sn[None, :]
scales_ptrs = scales_ptr + offsets_s
scales = tl.load(scales_ptrs, mask=masks_s)
scales = tl.broadcast_to(scales, (BLOCK_SIZE_K, BLOCK_SIZE_N))
b = (b >> shifts) & 0xF
zeros = (zeros >> shifts) & 0xF
b = (b - zeros) * scales
b = b.to(c_ptr.type.element_ty)
# Accumulate results.
accumulator = tl.dot(a, b, accumulator, out_dtype=accumulator_dtype)