Programmatic Tool Calling

Programmatic Tool Calling lets a model write and run JavaScript that coordinates the tools in a Responses API request. A program can call tools in parallel, use loops and conditions, and keep intermediate results in the hosted runtime. This is useful when a task needs a sequence of related tool calls or needs to process large tool outputs before returning a result.

Your application decides whether Programmatic Tool Calling is available and which eligible tools the model can call directly, from a program, or either way. It continues to run any client-owned tool calls.

Check the model page before enabling Programmatic Tool Calling.

Understand the runtime environment

OpenAI runs each generated program in a fresh, isolated V8 runtime. The runtime supports JavaScript with top-level await, but it does not provide Node.js, package installation, direct network access, a general-purpose filesystem, subprocess execution, a console, or persistent JavaScript state between program executions. Programs can interact with external systems only through tools enabled in the request and can emit output with text(...) or image(...).

Programmatic Tool Calling supports Zero Data Retention (ZDR) workflows without requiring a persistent code-execution container. ZDR must be enabled for the organization or project; setting store: false enables stateless continuation but does not enable ZDR by itself. Eligibility and retention depend on the complete request, including its model, tools, and third-party services; see data controls.

Choose when to use Programmatic Tool Calling

Use Programmatic Tool Calling when a stage has predictable control flow and code can return a smaller structured result. Use direct tool calling when one call is sufficient, each result requires fresh model judgment, or the work requires approval or preservation of citations or native artifacts.

Configure Programmatic Tool Calling

Add the programmatic_tool_calling hosted tool to the request. Then set allowed_callers on each eligible tool that the program can invoke.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
[
  {
    "type": "function",
    "name": "get_inventory",
    "description": "Return an object with sku (string) and available_units (number).",
    "parameters": {
      "type": "object",
      "properties": {
        "sku": { "type": "string" }
      },
      "required": ["sku"],
      "additionalProperties": false
    },
    "output_schema": {
      "type": "object",
      "properties": {
        "sku": { "type": "string" },
        "available_units": { "type": "number" }
      },
      "required": ["sku", "available_units"],
      "additionalProperties": false
    },
    "allowed_callers": ["programmatic"]
  },
  {
    "type": "programmatic_tool_calling"
  }
]

allowed_callers controls how the model can invoke a tool:

parameters describes the function arguments. When a function returns predictable structured data, output_schema describes the JSON object encoded in its function_call_output.output string. Define both so generated JavaScript can use the returned fields reliably.

Supported tools

The following tool types support allowed_callers: ["programmatic"]:

• function and custom
• mcp
• apply_patch
• Local and hosted shell
• code_interpreter

For MCP tools, the tool’s require_approval policy can pause the program until you approve the call.

For OpenAI-hosted tools, review the tool’s data-retention and security guidance before enabling it in a program.

Combine with tool search

Tool search runs as a top-level Responses API tool, not from inside generated JavaScript. Function, custom, and MCP tools with defer_loading: true are not initially available to a program. After the model loads a matching tool, a later program can invoke it through tools.* when its allowed_callers includes "programmatic". An already-running program cannot invoke tool search, so the model must load deferred tools before starting a program that needs them.

Guide routing when both modes are available

When your application lets the model call a function directly or from a program, assign each route to a specific workflow stage. Generic instructions such as “use Programmatic Tool Calling efficiently” don’t identify the intended boundary. For example:

<tool_orchestration>
Use Programmatic Tool Calling for [bounded stage] using only [eligible tools].
Run independent calls concurrently when safe. Use only documented tool input
and output fields.

Process and reduce the intermediate results, then emit exactly [program result shape],
including the evidence needed for the final answer.

Stop when [condition] is met. Retry transient failures at most [R] times.
Do not repeat completed calls or perform side-effecting actions. If a required
result is still missing, return a clear structured failure.

Use direct tool calls for [semantic judgment, approval, or final validation].
</tool_orchestration>

Here is an example of how to use this template:

<tool_orchestration>
Use Programmatic Tool Calling to compare inventory with demand for sku_123
using only get_inventory and get_demand. Run both calls concurrently. Use
only documented tool input and output fields.

Process and reduce the intermediate results, then emit exactly one JSON object
with sku, available_units, requested_units, and shortage_units, where
shortage_units is max(requested_units - available_units, 0). Include
available_units and requested_units as evidence for the calculation.

Stop when both tool results contain the required fields. Retry transient
failures at most 1 time. Do not repeat completed calls or perform
side-effecting actions. If a required result is still missing, return a clear
structured failure.

Use direct tool calls only for approval before any inventory-changing action.
</tool_orchestration>

For workflows that need both modes, define one handoff and avoid switching routes or repeating work. If a safe fallback exists, define it once and limit its retries.

Understand program response items

Each API call still returns the standard Responses API object. Programmatic Tool Calling doesn’t introduce a separate response envelope. When the model uses Programmatic Tool Calling, the response’s output array can contain:

• A program item containing the generated JavaScript, a call_id, and an opaque fingerprint used to resume or replay the program.
• A function_call item made by the program. It has its own call_id, which your application uses to return the function result. Its caller.caller_id matches the program’s call_id.
• A program_output item containing the program’s final result and status. Its call_id matches the program’s call_id, and its status is completed or incomplete.

These are separate top-level items in response.output; the caller field records their execution relationship.

For example, a program can pause while your application runs get_inventory and get_demand:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
[
  {
    "type": "program",
    "id": "prog_123",
    "call_id": "call_prog_123",
    "code": "const [stock, demand] = await Promise.all([tools.get_inventory({ sku: 'sku_123' }), tools.get_demand({ sku: 'sku_123' })]); text(JSON.stringify({ sku: stock.sku, available_units: stock.available_units, requested_units: demand.requested_units, shortage_units: Math.max(demand.requested_units - stock.available_units, 0) }));",
    "fingerprint": "opaque_replay_state"
  },
  {
    "type": "function_call",
    "id": "fc_123",
    "call_id": "call_inventory_123",
    "name": "get_inventory",
    "arguments": "{\"sku\":\"sku_123\"}",
    "caller": {
      "type": "program",
      "caller_id": "call_prog_123"
    }
  },
  {
    "type": "function_call",
    "id": "fc_456",
    "call_id": "call_demand_123",
    "name": "get_demand",
    "arguments": "{\"sku\":\"sku_123\"}",
    "caller": {
      "type": "program",
      "caller_id": "call_prog_123"
    }
  }
]

These examples show only the relevant items from response.output; they omit the surrounding standard Responses object. After your application returns the nested function results, a later response can contain the complete program_output item:

1
2
3
4
5
6
7
{
  "type": "program_output",
  "id": "prog_out_123",
  "call_id": "call_prog_123",
  "result": "{\"sku\":\"sku_123\",\"available_units\":42,\"requested_units\":31,\"shortage_units\":0}",
  "status": "completed"
}

The JSON string in program_output.result follows the program result shape from your instructions. The surrounding program_output item follows the API contract shown above. These are separate contracts. A final message can arrive with the program output or in a later response, so continue until you receive that message.

OpenAI runs the model-generated JavaScript in the hosted runtime. Your application executes returned client-owned function calls; it does not execute the generated JavaScript.

Return the function result as a function_call_output. Copy caller from the function call without changing it. The service uses that value to resume the correct program.

Continue after client-owned function calls

A program can pause more than once as it reaches client-owned tools. Continue until the response contains a final assistant message:

1. Send the request with the hosted tool and functions that allow programmatic calls.
2. Run every returned client-owned function call.
3. Return each function result with the original call_id and caller.
4. Handle an incomplete response before continuing.
5. If the response contains no pending function_call items and no final message item, continue from that response. With store: false, replay its output items; for a stored response, use previous_response_id.
6. Stop when the response contains a final message item. Read response.output_text or the message’s refusal content.

The following example uses store: false, preserves every response item, and returns each function result to the program:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
import OpenAI from "openai";

const client = new OpenAI();

const implementations = {
  get_inventory: async ({ sku }) => ({ sku, available_units: 42 }),
  get_demand: async ({ sku }) => ({ sku, requested_units: 31 }),
};

const tools = [
  {
    type: "function",
    name: "get_inventory",
    description:
      "Return an object with sku (string) and available_units (number).",
    parameters: {
      type: "object",
      properties: { sku: { type: "string" } },
      required: ["sku"],
      additionalProperties: false,
    },
    output_schema: {
      type: "object",
      properties: {
        sku: { type: "string" },
        available_units: { type: "number" },
      },
      required: ["sku", "available_units"],
      additionalProperties: false,
    },
    allowed_callers: ["programmatic"],
  },
  {
    type: "function",
    name: "get_demand",
    description:
      "Return an object with sku (string) and requested_units (number).",
    parameters: {
      type: "object",
      properties: { sku: { type: "string" } },
      required: ["sku"],
      additionalProperties: false,
    },
    output_schema: {
      type: "object",
      properties: {
        sku: { type: "string" },
        requested_units: { type: "number" },
      },
      required: ["sku", "requested_units"],
      additionalProperties: false,
    },
    allowed_callers: ["programmatic"],
  },
  { type: "programmatic_tool_calling" },
];

const input = [
  {
    role: "user",
    content: "Compare inventory with demand for sku_123.",
  },
];

while (true) {
  const response = await client.responses.create({
    model: "YOUR_MODEL_ID",
    store: false,
    input,
    tools,
    include: ["reasoning.encrypted_content"],
  });

  if (response.status !== "completed") {
    throw new Error(`Response ended with status ${response.status}`);
  }

  // Preserve every output item, including program and reasoning items.
  input.push(...response.output);

  const calls = response.output.filter(
    (item) => item.type === "function_call",
  );

  if (calls.length === 0) {
    const message = response.output.find((item) => item.type === "message");
    if (message) {
      const refusal = message.content.find((part) => part.type === "refusal");
      console.log(response.output_text || refusal?.refusal || "");
      break;
    }
    continue;
  }

  const outputs = await Promise.all(
    calls.map(async (call) => {
      const run = implementations[call.name];
      if (!run) throw new Error(`Unknown tool: ${call.name}`);

      const result = await run(JSON.parse(call.arguments));
      return {
        type: "function_call_output",
        call_id: call.call_id,
        output: JSON.stringify(result),
        // Preserve caller so the runtime can resume the correct program.
        caller: call.caller,
      };
    }),
  );

  input.push(...outputs);
}

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
import json
from openai import OpenAI

client = OpenAI()


def get_inventory(sku):
    return {"sku": sku, "available_units": 42}


def get_demand(sku):
    return {"sku": sku, "requested_units": 31}


implementations = {
    "get_inventory": get_inventory,
    "get_demand": get_demand,
}

tools = [
    {
        "type": "function",
        "name": "get_inventory",
        "description": "Return an object with sku (string) and available_units (number).",
        "parameters": {
            "type": "object",
            "properties": {"sku": {"type": "string"}},
            "required": ["sku"],
            "additionalProperties": False,
        },
        "output_schema": {
            "type": "object",
            "properties": {
                "sku": {"type": "string"},
                "available_units": {"type": "number"},
            },
            "required": ["sku", "available_units"],
            "additionalProperties": False,
        },
        "allowed_callers": ["programmatic"],
    },
    {
        "type": "function",
        "name": "get_demand",
        "description": "Return an object with sku (string) and requested_units (number).",
        "parameters": {
            "type": "object",
            "properties": {"sku": {"type": "string"}},
            "required": ["sku"],
            "additionalProperties": False,
        },
        "output_schema": {
            "type": "object",
            "properties": {
                "sku": {"type": "string"},
                "requested_units": {"type": "number"},
            },
            "required": ["sku", "requested_units"],
            "additionalProperties": False,
        },
        "allowed_callers": ["programmatic"],
    },
    {"type": "programmatic_tool_calling"},
]

input_items = [
    {
        "role": "user",
        "content": "Compare inventory with demand for sku_123.",
    }
]

while True:
    response = client.responses.create(
        model="YOUR_MODEL_ID",
        store=False,
        input=input_items,
        tools=tools,
        include=["reasoning.encrypted_content"],
    )

    if response.status != "completed":
        raise RuntimeError(f"Response ended with status {response.status}")

    # Preserve every output item, including program and reasoning items.
    input_items.extend(
        item.model_dump(exclude_none=True) for item in response.output
    )

    calls = [item for item in response.output if item.type == "function_call"]
    if not calls:
        message = next((item for item in response.output if item.type == "message"), None)
        if message:
            refusal = next(
                (part.refusal for part in message.content if part.type == "refusal"),
                "",
            )
            print(response.output_text or refusal)
            break
        continue

    for call in calls:
        run = implementations.get(call.name)
        if run is None:
            raise ValueError(f"Unknown tool: {call.name}")

        result = run(**json.loads(call.arguments))
        input_items.append(
            {
                "type": "function_call_output",
                "call_id": call.call_id,
                "output": json.dumps(result),
                # Preserve caller so the runtime can resume the correct program.
                "caller": call.caller.model_dump() if call.caller else None,
            }
        )

When you store responses, you can continue from previous_response_id instead of resending all earlier response items. Send the new function_call_output items as the next input. With store: false, replay the complete sequence in order, including every program, reasoning, function-call, function-call-output, and program_output item.

For stateless reasoning-model requests, include reasoning.encrypted_content and replay the returned reasoning items. See conversation state for the general stateless pattern.

Design tools for programs

• Return structured, compact data that JavaScript can inspect without parsing prose.
• Use output_schema to define each tool’s expected return fields and types, and document its error behavior. If the return shape isn’t known in advance, keep the tool direct so the model can inspect the result.
• Define the exact program result shape and required evidence. Return a clear structured failure when the program can’t produce a valid result.
• Make function calls idempotent when possible. A retry or replay shouldn’t repeat an unsafe side effect.
• Check arguments and permissions for each call in your application, even when it comes from a hosted program.
• Give tools specific names and descriptions so the model can compose them correctly.
• Require application-level approval before high-impact actions, regardless of the caller.

Evaluate Programmatic Tool Calling

Programmatic Tool Calling can reduce the amount of intermediate tool output added to model context, but the effect depends on the task and tool responses. Start with direct tool calling as a baseline, then compare both approaches on representative tasks.

Define the final-answer quality bar and required evidence before measuring efficiency. Evaluate token use and tool calls alongside correctness, completeness, and evidence coverage, and make any accepted quality tradeoff explicit.

Measure:

• Final-answer correctness, completeness, and evidence coverage.
• Input and total tokens, end-to-end latency, and cost.
• Model turns, tool calls, retries, and recovery behavior.
• Safety outcomes, especially for side effects and approval requirements.
• Whether the route that ran matched the intended workflow stage.

Related guides

• Use function calling to define client-owned functions.
• Use tool search to defer large tool definitions until a model needs them.
• Use conversation state to continue stored or stateless Responses API requests.
• Review data controls before choosing a storage mode.