Sampling parameters to tau2-bench?
#8
by
lewtun
- opened
Hello Qwen team,
I was wondering if you could share the sampling parameters you used for tau2-bench
? By default, the benchmark uses temperature=0
and I wonder if you changed this to the values adopted in the Qwen3 tech report (namely, temperature=0.7
, top_p=0.8
, top_k=20
, and presence_penalty=1.5
)?
Also, if you used vllm
to generate the trajectories, can you share the command that was used to create the server? Currently I create the server as follows:
vllm serve Qwen/Qwen3-4B-Instruct-2507 \
--tensor-parallel-size 1 \
--trust-remote-code \
--enable-auto-tool-choice \
--tool-call-parser hermes \
--host 0.0.0.0 --port 8000
I am unsure if you use a different --tool-call-parser
Thank you!
lewtun
changed discussion title from
Sampling parameters to `tau2-bench`
to Sampling parameters to tau2-bench?