Response format
#2
by
MatVet
- opened
When executing the tools the model marked as righ to use given the user query, how am I supposed to return the tools' response to the model? as part of the "user" role? in a specific format? what about when there is a single tool or multiple tools? Do I feed each response in a separate message?
They should give an example. But you can look at the jinja chat template to understand it.
Here is the part for putting the tool calls in
{%- if message["tool_calls"] -%}
{{- "[" -}}
{%- for tool_call_function in message.tool_calls -%}
{%- set tool_call = tool_call_function.function -%}
{{- "{\"name\": \"" + tool_call.name + "\", " -}}
{{- "\"arguments\": " -}}
{{- tool_call.arguments | tojson -}}
{{- "}" -}}
{%- if not loop.last -%}
{{- ", " -}}
{%- endif -%}
{%- endfor -%}
{{- "]" -}}
{{- "<|eot_id|>" -}}
{%- elif message["content"] -%}
{{- message["content"] | trim + "<|eot_id|>" -}}
{%- else -%}
{{- "[]\n" + "<|eot_id|>" -}}
{%- endif -%}
So I use this
messages.append({
"role": "assistant",
"tool_calls": tool_calls
})
where tools_calls comes from the llm output
And then look at the section in the chat template on tool responses:
{%- elif message.role == "tool" or message.role == "ipython" -%}
{{- "<|start_header_id|>" + "ipython" + "<|end_header_id|>\n\n" -}}
{%- set content = message["content"] -%}
{%- if content is mapping or content is iterable and content is not string -%}
{{- content | tojson -}}
{%- else -%}
{{- content -}}
{%- endif -%}
{{- "<|eot_id|>" -}}
{%- endif -%}
So I give tool responses like this:
tool_response_messages.append({
"role": "tool",
"content": content
})
I initially used the openai format including the tool_call_id and name, but according to that chat template those fields would get ignored.