Skip to content

Tool calling

Let a model call functions you define, so it can fetch data or take actions and then use the result in its response. Mango Inference uses the OpenAI-compatible tools interface.

When to use it

Use tool calling when the model needs to reach outside the conversation (look up a record, call an API, run a calculation) before answering.

Define tools and make a request

from openai import OpenAI

client = OpenAI(base_url="https://api.mangoboost.io/v1", api_key="<your-api-key>")

tools = [{
    "type": "function",
    "function": {
        "name": "get_weather",
        "description": "Get the current weather for a city.",
        "parameters": {
            "type": "object",
            "properties": {"city": {"type": "string"}},
            "required": ["city"],
        },
    },
}]

resp = client.chat.completions.create(
    model="zai-org/GLM-5.3",
    messages=[{"role": "user", "content": "What's the weather in Seoul?"}],
    tools=tools,
)

Handle the tool call

Tool calling is a round trip: the model asks, you answer, the model concludes.

1. The model returns tool_calls instead of a final answer. Its finish_reason is tool_calls, and content is usually empty:

{
  "index": 0,
  "message": {
    "role": "assistant",
    "content": "",
    "tool_calls": [
      {
        "id": "call_abc123",
        "type": "function",
        "function": {"name": "get_weather", "arguments": "{\"city\": \"Seoul\"}"}
      }
    ]
  },
  "finish_reason": "tool_calls"
}

arguments is a JSON-encoded string, not an object. Parse it before use, and validate it: it is model output, so it can be malformed or name arguments you did not define.

2. Run the function, then send the result back. Append the assistant's message verbatim, then one role: "tool" message per call, matching tool_call_id to the id you received:

import json

msg = resp.choices[0].message
messages = [{"role": "user", "content": "What's the weather in Seoul?"}, msg]

for call in msg.tool_calls:
    args = json.loads(call.function.arguments)
    result = get_weather(**args)  # your function
    messages.append({
        "role": "tool",
        "tool_call_id": call.id,
        "content": json.dumps(result),
    })

final = client.chat.completions.create(
    model="zai-org/GLM-5.3",
    messages=messages,
    tools=tools,
)
print(final.choices[0].message.content)

3. The model answers using the result. Keep passing tools on the follow-up call. The model may want another round trip, so loop until finish_reason is no longer tool_calls, with a cap on iterations.

Dropping the assistant message breaks the round trip

The role: "tool" messages only make sense after the assistant message that requested them. If you append results without it, the model sees tool output for a call it never made.

Supported models

Tool calling is a property of the model, not of the platform: Mango Inference passes tools through, and whether the model uses them depends on how it was trained. Check the model card for the model you plan to use, and verify with a real call before you depend on it. A model that does not support tools will typically ignore them and answer in prose rather than fail.

List the models your key can call with GET /v1/models; see Models and pricing.