# OpenAI Developers — full documentation > Single-file Markdown export covering OpenAI API, Ads, Plugins, Workspace Agents, Codex, and Agentic Commerce. Curated indexes: - https://developers.openai.com/api/llms.txt - https://developers.openai.com/ads/llms.txt - https://developers.openai.com/plugins/llms.txt - https://developers.openai.com/workspace-agents/llms.txt - https://developers.openai.com/codex/llms.txt - https://developers.openai.com/commerce/llms.txt - https://developers.openai.com/blog/llms.txt - https://developers.openai.com/cookbook/llms.txt - https://developers.openai.com/learn/llms.txt - https://developers.openai.com/tracks.md ## OpenAI API # Actions in ChatKit Actions are a way for the ChatKit SDK frontend to trigger a streaming response without the user submitting a message. They can also be used to trigger side-effects outside ChatKit SDK. ## Triggering actions ### In response to user interaction with widgets Actions can be triggered by attaching an `ActionConfig` to any widget node that supports it. For example, you can respond to click events on Buttons. When a user clicks on this button, the action will be sent to your server where you can update the widget, run inference, stream new thread items, etc. ```python button = Button( label="Example", onClickAction=ActionConfig( type="example", payload={"id": 123}, ), ) ``` Actions can also be sent imperatively by your frontend with `sendAction()`. This is probably most useful when you need ChatKit to respond to interaction happening outside ChatKit, but it can also be used to chain actions when you need to respond on both the client and the server (more on that below). ```javascript await chatKit.sendAction({ type: "example", payload: { id: 123 }, }); ``` ## Handling actions ### On the server By default, actions are sent to your server. You can handle actions on your server by implementing the `action` method on `ChatKitServer`. ```python class MyChatKitServer(ChatKitServer[RequestContext]): async def action( self, thread: ThreadMetadata, action: Action[str, Any], sender: WidgetItem | None, context: RequestContext, ) -> AsyncIterator[Event]: if action.type == "example": await do_thing(action.payload["id"]) # Often you'll want to add a HiddenContextItem so the model # can see that the user did something. await self.store.add_thread_item( thread.id, HiddenContextItem( id="item_123", thread_id=thread.id, created_at=datetime.now(), content="The user did a thing", ), context, ) # Then you might want to run inference to stream a response # back to the user. async for event in self.generate(context, thread): yield event ``` Treat actions and their payloads as untrusted data because the client sends them to your server. ### Client Sometimes you’ll want to handle actions in your client integration. To do that you need to specify that the action should be sent to your client-side action handler by adding `handler="client"` to the `ActionConfig`. ```python button = Button( label="Example", onClickAction=ActionConfig(type="example", payload={"id": 123}, handler="client"), ) ``` Then, when the action is triggered, it will then be passed to a callback that you provide when instantiating ChatKit. ```javascript async function handleWidgetAction(action) { if (action.type === "example") { const res = await doSomething(action); // You can fire off actions to your server from here as well. // For example, stream new thread items or update a widget. await chatKit.sendAction({ type: "example_complete", payload: res, }); } } chatKit.setOptions({ // Other options... widgets: { onAction: handleWidgetAction }, }); ``` ## Strongly typed actions By default `Action` and `ActionConfig` are not strongly typed. However, we do expose a `create` helper on `Action` that generates `ActionConfig`s from a set of strongly-typed actions. ```python class ExamplePayload(BaseModel): id: int ExampleAction = Action[Literal["example"], ExamplePayload] OtherAction = Action[Literal["other"], None] AppAction = Annotated[ ExampleAction | OtherAction, Field(discriminator="type"), ] ActionAdapter: TypeAdapter[AppAction] = TypeAdapter(AppAction) def parse_app_action(action: Action[str, Any]) -> AppAction: return ActionAdapter.validate_python(action) # Usage in a widget # Action provides a create helper which makes it easy to generate # ActionConfigs from strongly typed actions. button = Button( label="Example", onClickAction=ExampleAction.create(ExamplePayload(id=123)), ) # usage in action handler class MyChatKitServer(ChatKitServer[RequestContext]): async def action( self, thread: ThreadMetadata, action: Action[str, Any], sender: WidgetItem | None, context: RequestContext, ) -> AsyncIterator[Event]: # add custom error handling if needed app_action = parse_app_action(action) if app_action.type == "example": await do_thing(app_action.payload.id) yield ThreadItemDoneEvent( item=AssistantMessageItem( id=self.store.generate_item_id("message", thread, context), thread_id=thread.id, created_at=datetime.now(), content=[AssistantMessageContent(text="Action complete.")], ) ) ``` ## Use widgets and actions to create custom forms When widget nodes that take user input are mounted inside a `Form`, the values from those fields will be included in the `payload` of all actions that originate from within the `Form`. Form values are keyed in the `payload` by their `name` e.g. - `Select(name="title")` → `action.payload.title` - `Select(name="todo.title")` → `action.payload.todo.title` ```python form = Form( direction="col", validation="native", onSubmitAction=ActionConfig( type="update_todo", payload={"id": todo.id}, ), children=[ Title(value="Edit Todo"), Text(value="Title", color="secondary", size="sm"), Text( value=todo.title, editable=EditableProps(name="title", required=True), ), Text(value="Description", color="secondary", size="sm"), Text( value=todo.description, editable=EditableProps(name="description"), ), Button(label="Save", submit=True), ], ) class MyChatKitServer(ChatKitServer[RequestContext]): async def action( self, thread: ThreadMetadata, action: Action[str, Any], sender: WidgetItem | None, context: RequestContext, ) -> AsyncIterator[Event]: if action.type == "update_todo": todo_id = action.payload["id"] # Any action that originates from within the Form will # include title and description. title = action.payload["title"] description = action.payload["description"] await update_todo(todo_id, title, description) yield ThreadItemDoneEvent( item=AssistantMessageItem( id=self.store.generate_item_id("message", thread, context), thread_id=thread.id, created_at=datetime.now(), content=[AssistantMessageContent(text="Todo updated.")], ) ) ``` ### Validation `Form` uses basic native form validation; enforcing `required` and `pattern` on fields where they are configured and blocking submission when the form has any invalid field. We may add new validation modes with better UX, more expressive validation, custom error display, etc in the future. Until then, widgets are not a great medium for complex forms with tricky validation. If you have this need, a better pattern would be to use client side action handling to trigger a modal, show a custom form there, then pass the result back into ChatKit with `sendAction`. ### Treating `Card` as a `Form` You can pass `asForm=True` to `Card` and it will behave as a `Form`, running validation and passing collected fields to the Card’s `confirm` action. ### Payload key collisions If there is a naming collision with some other existing pre-defined key on your payload, the form value will be ignored. This is probably a bug, so we’ll emit an `error` event when we see this. ## Control loading state interactions in widgets Use `ActionConfig.loadingBehavior` to control how actions trigger different loading states in a widget. ```python button = Button( label="This may take a while...", onClickAction=ActionConfig( type="long_running_action_that_should_block_other_ui_interactions", loadingBehavior="container", ), ) ``` | Value | Behavior | | ----------- | ------------------------------------------------------------------------------------------------------------------------------- | | `auto` | The action will adapt to how it’s being used. (_default_) | | `self` | The action triggers loading state on the widget node that the action was bound to. | | `container` | The action triggers loading state on the entire widget container. This causes the widget to fade out slightly and become inert. | | `none` | No loading state | ### Using `auto` behavior Generally, we recommend using `auto`, which is the default. `auto` triggers loading states based on where the action is bound, for example: - `Button.onClickAction` → `self` - `Select.onChangeAction` → `none` - `Card.confirm.action` → `container` --- # Admin APIs Admin APIs let you automate organization management workflows such as user invitations, audit log review, project administration, API key management, spend limits and alerts, data retention, and rate limit operations. Use them for back-office automation, security workflows, and operational tooling that should run outside the dashboard. For endpoint details, see the [Administration API reference](https://developers.openai.com/api/reference/administration/overview), including [Admin API keys](https://developers.openai.com/api/reference/resources/admin/subresources/organization/subresources/admin_api_keys), [Invites](https://developers.openai.com/api/reference/resources/admin/subresources/organization/subresources/invites), [Users](https://developers.openai.com/api/reference/resources/admin/subresources/organization/subresources/users), [Projects](https://developers.openai.com/api/reference/resources/admin/subresources/organization/subresources/projects), [Spend limits](https://developers.openai.com/api/reference/resources/admin/subresources/organization/subresources/spend_limit), and [Audit logs](https://developers.openai.com/api/reference/resources/admin/subresources/organization/subresources/audit_logs). ## Use an Admin API key with the SDK To access these endpoints, [create an Admin API key](https://platform.openai.com/settings/organization/admin-keys). Admin API keys cannot be used for non-administration endpoints. Support for Admin APIs was added in these SDK versions, which may require updating your SDK version: - Node: `6.36.0` - Python: `2.34.0` - Go: `3.34.0` - Ruby: `0.61.0` - Java: `4.34.0` Set `OPENAI_ADMIN_KEY`, then initialize the SDK for your language. Set up the SDK with an Admin API key ```javascript import OpenAI from "openai"; const client = new OpenAI({ adminAPIKey: process.env.OPENAI_ADMIN_KEY, }); ``` ```python import os from openai import OpenAI client = OpenAI( admin_api_key=os.environ["OPENAI_ADMIN_KEY"], ) ``` ```go package main import ( "os" "github.com/openai/openai-go/v3" "github.com/openai/openai-go/v3/option" ) func main() { client := openai.NewClient( option.WithAdminAPIKey(os.Getenv("OPENAI_ADMIN_KEY")), ) _ = client } ``` ```java import com.openai.client.OpenAIClient; import com.openai.client.okhttp.OpenAIOkHttpClient; OpenAIClient client = OpenAIOkHttpClient.builder().adminApiKey(System.getenv("OPENAI_ADMIN_KEY")).build(); ``` ```ruby require "openai" openai = OpenAI::Client.new( admin_api_key: ENV.fetch("OPENAI_ADMIN_KEY") ) ``` ## Restrict model access for projects Use project model permissions to set an allowlist or denylist for a project. Set `mode` to `allow_list` to allow only the listed models, or set `mode` to `deny_list` to block the listed models while allowing other available models. Model IDs must be visible to the organization, including visible fine-tuned model snapshots. Set a project model allowlist/denylist ```javascript const modelPermissions = await client.admin.organization.projects.modelPermissions.update("proj_abc", { mode: "allow_list", model_ids: ["gpt-4.1", "o3"], }); console.log(modelPermissions.mode); ``` ```python model_permissions = client.admin.organization.projects.model_permissions.update( "proj_abc", mode="allow_list", model_ids=["gpt-4.1", "o3"], ) print(model_permissions.mode) ``` ```go ctx := context.Background() modelPermissions, err := client.Admin.Organization.Projects.ModelPermissions.Update( ctx, "proj_abc", openai.AdminOrganizationProjectModelPermissionUpdateParams{ Mode: openai.AdminOrganizationProjectModelPermissionUpdateParamsModeAllowList, ModelIDs: []string{"gpt-4.1", "o3"}, }, ) if err != nil { panic(err) } println(modelPermissions.Mode) ``` ```java import com.openai.models.admin.organization.projects.modelpermissions.ModelPermissionUpdateParams; import com.openai.models.admin.organization.projects.modelpermissions.ProjectModelPermissions; import java.util.List; ProjectModelPermissions modelPermissions = client .admin() .organization() .projects() .modelPermissions() .update( "proj_abc", ModelPermissionUpdateParams.builder() .mode(ModelPermissionUpdateParams.Mode.ALLOW_LIST) .modelIds(List.of("gpt-4.1", "o3")) .build()); System.out.println(modelPermissions.mode()); ``` ```ruby model_permissions = openai.admin.organization.projects.model_permissions.update( "proj_abc", mode: :allow_list, model_ids: ["gpt-4.1", "o3"] ) puts(model_permissions.mode) ``` ## Set an organization spend limit Use the [Spend Limits endpoint](https://developers.openai.com/api/reference/resources/admin/subresources/organization/subresources/spend_limit) to create or replace your organization's monthly hard spend limit. Set `threshold_amount` in cents. The following example sets a $100 monthly limit: ```bash curl -X POST https://api.openai.com/v1/organization/spend_limit \ -H "Authorization: Bearer $OPENAI_ADMIN_KEY" \ -H "Content-Type: application/json" \ -d '{ "threshold_amount": 10000, "currency": "USD", "interval": "month" }' ``` When tracked spend reaches a hard limit, affected API requests return a `429` error. For details, see the [spend limits guide](https://developers.openai.com/api/docs/guides/spend-limits). ## Manage spend limit alerts Use project spend alerts to notify your team when project spend reaches a threshold. Threshold amounts are specified in cents. Create a project spend limit alert ```javascript const spendAlert = await client.admin.organization.projects.spendAlerts.create( "proj_abc", { currency: "USD", interval: "month", notification_channel: { recipients: ["billing@example.com"], type: "email", subject_prefix: "[OpenAI spend]", }, threshold_amount: 50000, } ); console.log(spendAlert.id); ``` ```python spend_alert = client.admin.organization.projects.spend_alerts.create( "proj_abc", currency="USD", interval="month", notification_channel={ "recipients": ["billing@example.com"], "type": "email", "subject_prefix": "[OpenAI spend]", }, threshold_amount=50000, ) print(spend_alert.id) ``` ```go ctx := context.Background() spendAlert, err := client.Admin.Organization.Projects.SpendAlerts.New( ctx, "proj_abc", openai.AdminOrganizationProjectSpendAlertNewParams{ Currency: openai.AdminOrganizationProjectSpendAlertNewParamsCurrencyUsd, Interval: openai.AdminOrganizationProjectSpendAlertNewParamsIntervalMonth, NotificationChannel: openai.AdminOrganizationProjectSpendAlertNewParamsNotificationChannel{ Recipients: []string{"billing@example.com"}, Type: "email", SubjectPrefix: openai.String("[OpenAI spend]"), }, ThresholdAmount: 50000, }, ) if err != nil { panic(err) } println(spendAlert.ID) ``` ```java import com.openai.models.admin.organization.projects.spendalerts.ProjectSpendAlert; import com.openai.models.admin.organization.projects.spendalerts.SpendAlertCreateParams; ProjectSpendAlert spendAlert = client .admin() .organization() .projects() .spendAlerts() .create( "proj_abc", SpendAlertCreateParams.builder() .currency(SpendAlertCreateParams.Currency.USD) .interval(SpendAlertCreateParams.Interval.MONTH) .notificationChannel( SpendAlertCreateParams.NotificationChannel.builder() .addRecipient("billing@example.com") .subjectPrefix("[OpenAI spend]") .build()) .thresholdAmount(50000L) .build()); System.out.println(spendAlert.id()); ``` ```ruby spend_alert = openai.admin.organization.projects.spend_alerts.create( "proj_abc", currency: :USD, interval: :month, notification_channel: { recipients: ["billing@example.com"], type: :email, subject_prefix: "[OpenAI spend]" }, threshold_amount: 50_000 ) puts(spend_alert.id) ``` ## Manage data retention Use project data retention controls to override or inherit the organization's retention policy for a project. Set `retention_type` to `organization_default` to inherit the organization setting. Set project data retention ```javascript const dataRetention = await client.admin.organization.projects.dataRetention.update("proj_abc", { retention_type: "organization_default", }); console.log(dataRetention.type); ``` ```python data_retention = client.admin.organization.projects.data_retention.update( "proj_abc", retention_type="organization_default", ) print(data_retention.type) ``` ```go ctx := context.Background() dataRetention, err := client.Admin.Organization.Projects.DataRetention.Update( ctx, "proj_abc", openai.AdminOrganizationProjectDataRetentionUpdateParams{ RetentionType: openai.AdminOrganizationProjectDataRetentionUpdateParamsRetentionTypeOrganizationDefault, }, ) if err != nil { panic(err) } println(dataRetention.Type) ``` ```java import com.openai.models.admin.organization.projects.dataretention.DataRetentionUpdateParams; import com.openai.models.admin.organization.projects.dataretention.ProjectDataRetention; ProjectDataRetention dataRetention = client .admin() .organization() .projects() .dataRetention() .update( "proj_abc", DataRetentionUpdateParams.builder() .retentionType(DataRetentionUpdateParams.RetentionType.ORGANIZATION_DEFAULT) .build()); System.out.println(dataRetention.type()); ``` ```ruby data_retention = openai.admin.organization.projects.data_retention.update( "proj_abc", retention_type: :organization_default ) puts(data_retention.type) ``` ## Invite a user by email Use the Invites endpoint to send an organization invitation to an email address. Invite a user by email ```javascript const invite = await client.admin.organization.invites.create({ email: "user@example.com", role: "reader", }); console.log(invite.id); ``` ```python invite = client.admin.organization.invites.create( email="user@example.com", role="reader", ) print(invite.id) ``` ```go ctx := context.Background() invite, err := client.Admin.Organization.Invites.New(ctx, openai.AdminOrganizationInviteNewParams{ Email: "user@example.com", Role: openai.AdminOrganizationInviteNewParamsRoleReader, }) if err != nil { panic(err) } println(invite.ID) ``` ```java import com.openai.models.admin.organization.invites.Invite; import com.openai.models.admin.organization.invites.InviteCreateParams; Invite invite = client .admin() .organization() .invites() .create( InviteCreateParams.builder() .email("user@example.com") .role(InviteCreateParams.Role.READER) .build()); System.out.println(invite.id()); ``` ```ruby invite = openai.admin.organization.invites.create( email: "user@example.com", role: :reader ) puts(invite.id) ``` ## Retrieve audit logs Use the Audit Logs endpoint to list recent user actions and configuration changes for the organization. Retrieve audit logs ```javascript const auditLogs = await client.admin.organization.auditLogs.list({ limit: 10, }); console.log(auditLogs.data); ``` ```python audit_logs = client.admin.organization.audit_logs.list(limit=10) for audit_log in audit_logs.data: print(audit_log.id) ``` ```go ctx := context.Background() auditLogs, err := client.Admin.Organization.AuditLogs.List(ctx, openai.AdminOrganizationAuditLogListParams{ Limit: openai.Int(10), }) if err != nil { panic(err) } for _, auditLog := range auditLogs.Data { println(auditLog.ID) } ``` ```java import com.openai.models.admin.organization.auditlogs.AuditLogListParams; var page = client .admin() .organization() .auditLogs() .list(AuditLogListParams.builder().limit(10L).build()); page.data().forEach(auditLog -> System.out.println(auditLog.id())); ``` ```ruby audit_logs = openai.admin.organization.audit_logs.list(limit: 10) (audit_logs.data || []).each do |audit_log| puts(audit_log.id) end ``` --- # Advanced integrations with ChatKit When you need full control—custom authentication, data residency, on‑prem deployment, or bespoke agent orchestration—you can run ChatKit on your own infrastructure. Use OpenAI's advanced self‑hosted option to use your own server and customized ChatKit. Agent Builder-hosted ChatKit workflows are in a transition window. For new ChatKit apps, build on your own server-side agent implementation with the ChatKit SDKs and the Agents SDK. See [ChatKit transition guidance →](https://developers.openai.com/api/docs/guides/chatkit) ## Run ChatKit on your own infrastructure At a high level, an advanced ChatKit integration is a process of building your own ChatKit server and adding widgets to build out your chat surface. You'll use OpenAI APIs and your ChatKit server to build a custom chat powered by OpenAI models. ![OpenAI-hosted ChatKit](https://cdn.openai.com/API/docs/images/self-hosted.png) ## Set up your ChatKit server Follow the [server guide on GitHub](https://github.com/openai/chatkit-python/blob/main/docs/server.md) to learn how to handle incoming requests, run tools, and stream results back to the client. The snippets below highlight the main components. ### 1. Install the server package ```bash pip install openai-chatkit ``` ### 2. Implement a server class `ChatKitServer` drives the conversation. Override `respond` to stream events whenever a user message or client tool output arrives. Helpers like `stream_agent_response` connect the server to the Agents SDK. ```python class MyChatKitServer(ChatKitServer[RequestContext]): async def respond( self, thread: ThreadMetadata, input: UserMessageItem | ClientToolCallOutputItem | None, context: RequestContext, ) -> AsyncIterator[Event]: items_page = await self.store.load_thread_items( thread.id, after=None, limit=20, order="desc", context=context, ) input_items = await simple_to_agent_input(list(reversed(items_page.data))) agent_context = AgentContext( thread=thread, store=self.store, request_context=context, ) result = Runner.run_streamed( assistant_agent, input_items, context=agent_context, ) async for event in stream_agent_response(agent_context, result): yield event ``` ### 3. Expose the endpoint Use your framework of choice to forward HTTP requests to the server instance. For example, with FastAPI: ```python from fastapi import FastAPI, Request, Response from fastapi.responses import StreamingResponse app = FastAPI() data_store = MemoryStore() server = MyChatKitServer(data_store) @app.post("/chatkit") async def chatkit_endpoint(request: Request): result = await server.process(await request.body(), {}) if isinstance(result, StreamingResult): return StreamingResponse(result, media_type="text/event-stream") return Response(content=result.json, media_type="application/json") ``` ### 4. Establish data store contract Implement `chatkit.store.Store` to persist threads, messages, and files using your preferred database. For local development, you can use an in-memory `Store` implementation. For production, use durable storage and consider storing the models as JSON blobs so library updates can evolve the schema without migrations. ### 5. Provide file store contract Provide a `FileStore` implementation if you support uploads. ChatKit works with direct uploads (the client POSTs the file to your endpoint) or two-phase uploads (the client requests a signed URL, then uploads to cloud storage). Expose previews to support inline thumbnails and handle deletions when threads are removed. ### 6. Trigger client tools from the server Client tools must be registered both in the client options and on your agent. Use `ctx.context.client_tool_call` to enqueue a call from an Agents SDK tool. ```python @function_tool(description_override="Add an item to the user's todo list.") async def add_to_todo_list(ctx: RunContextWrapper[AgentContext], item: str) -> None: ctx.context.client_tool_call = ClientToolCall( name="add_to_todo_list", arguments={"item": item}, ) assistant_agent = Agent[AgentContext]( model="gpt-6-astra", name="Assistant", instructions="You are a helpful assistant", tools=[add_to_todo_list], tool_use_behavior=StopAtTools(stop_at_tool_names=[add_to_todo_list.name]), ) ``` ### 7. Use thread metadata and state Use `thread.metadata` to store server-side state such as the previous Responses API run ID or custom labels. Metadata is not exposed to the client but is available in every `respond` call. ### 8. Get tool status updates Long-running tools can stream progress to the UI with `ProgressUpdateEvent`. ChatKit replaces the progress event with the next assistant message or widget output. ### 9. Using server context Pass a custom context object to `server.process(body, context)` to enforce permissions or propagate user identity through your store and file store implementations. ## Add inline interactive widgets Widgets let agents surface rich UI inside the chat surface. Use them for cards, forms, text blocks, lists, and other layouts. The helper `stream_widget` can render a widget immediately or stream updates as they arrive. ```python async def respond( self, thread: ThreadMetadata, input: UserMessageItem | ClientToolCallOutputItem | None, context: RequestContext, ) -> AsyncIterator[Event]: widget = Card( children=[ Text( id="description", value="Generated summary", ) ] ) async for event in stream_widget( thread, widget, generate_id=lambda item_type: self.store.generate_item_id( item_type, thread, context ), ): yield event ``` ChatKit ships with a wide set of widget nodes (cards, lists, forms, text, buttons, and more). See [widgets guide on GitHub](https://github.com/openai/chatkit-python/blob/main/docs/widgets.md) for all components, props, and streaming guidance. See the [Widget Builder](https://widgets.chatkit.studio/) to explore and create widgets in an interactive UI. ## Use actions Actions let the ChatKit UI trigger work without sending a user message. Attach an `ActionConfig` to any widget node that supports it—buttons, selects, and other controls can stream new thread items or update widgets in place. When a widget lives inside a `Form`, ChatKit includes the collected form values in the action payload. On the server, implement the `action` method on `ChatKitServer` to process the payload and optionally stream additional events. You can also handle actions on the client by setting `handler="client"` and responding in JavaScript before forwarding follow-up work to the server. See the [actions guide on GitHub](https://github.com/openai/chatkit-python/blob/main/docs/actions.md) for patterns like chaining actions, creating strongly typed payloads, and coordinating client/server handlers. ## Resources Use the following resources and reference to complete your integration. ### Design resources - Download [OpenAI Sans Variable](https://drive.google.com/file/d/10-dMu1Oknxg3cNPHZOda9a1nEkSwSXE1/view?usp=sharing). - Duplicate the file and customize components for your product. ### Events reference ChatKit emits `CustomEvent` instances from the Web Component. Listen for lifecycle events and read payload data from `event.detail`: ```javascript chatkit.addEventListener("chatkit.error", (event) => { console.error(event.detail.error); }); chatkit.addEventListener("chatkit.response.start", () => { console.log("Response started"); }); chatkit.addEventListener("chatkit.response.end", () => { console.log("Response ended"); }); chatkit.addEventListener("chatkit.thread.change", (event) => { console.log("Active thread:", event.detail.threadId); }); chatkit.addEventListener("chatkit.log", (event) => { console.log(event.detail.name, event.detail.data); }); ``` ### Options reference | Option | Type | Description | Default | | --------------- | -------------------------- | ---------------------------------------------------------- | -------------- | | `apiURL` | `string` | Endpoint that implements the ChatKit server protocol. | _required_ | | `fetch` | `typeof fetch` | Override fetch calls (for custom headers or auth). | `window.fetch` | | `theme` | `"light" \| "dark"` | UI theme. | `"light"` | | `initialThread` | `string \| null` | Thread to open on mount; `null` shows the new thread view. | `null` | | `clientTools` | `Record` | Client-executed tools exposed to the model. | | | `header` | `object \| boolean` | Header configuration or `false` to hide the header. | `true` | | `newThreadView` | `object` | Customize greeting text and starter prompts. | | | `messages` | `object` | Configure message features (feedback, annotations, etc.). | | | `composer` | `object` | Control attachments, entity tags, and placeholder text. | | | `entities` | `object` | Callbacks for entity lookup, click handling, and previews. | | ### Plain-text aliases - "light" | "dark" - string | null - object | boolean --- # Advanced usage OpenAI's text generation models (often called generative pre-trained transformers or large language models) have been trained to understand natural language, code, and images. The models provide text outputs in response to their inputs. The text inputs to these models are also referred to as "prompts." Designing a prompt is essentially how you “program” a large language model, usually by providing instructions or some examples of how to successfully complete a task. ## Reproducible outputs Chat Completions are non-deterministic by default (which means model outputs may differ from request to request). That being said, we offer some control towards deterministic outputs by giving you access to the [`seed`](https://developers.openai.com/api/reference/resources/chat#chat-create-seed) parameter and the [`system_fingerprint`](https://developers.openai.com/api/reference/resources/completions#completions/object-system_fingerprint) response field. To receive (mostly) deterministic outputs across API calls, you can: - Set the [seed](https://developers.openai.com/api/reference/resources/chat#chat-create-seed) parameter to any integer of your choice and use the same value across requests you'd like deterministic outputs for. - Ensure all other parameters (like `prompt` or `temperature`) are the exact same across requests. Sometimes, determinism may be impacted due to necessary changes OpenAI makes to model configurations on our end. To help you keep track of these changes, we expose the [`system_fingerprint`](https://developers.openai.com/api/reference/resources/chat#chat/object-system_fingerprint) field. If this value is different, you may see different outputs due to changes we've made on our systems. [Deterministic outputs Explore the new seed parameter in the OpenAI cookbook](https://developers.openai.com/cookbook/examples/reproducible_outputs_with_the_seed_parameter) ## Managing tokens Language models read and write text in chunks called tokens. In English, a token can be as short as one character or as long as one word (for example, `a` or ` apple`), and in some languages tokens can be even shorter than one character or even longer than one word. As a rough rule of thumb, 1 token is approximately 4 characters or 0.75 words for English text. Check out our [Tokenizer tool](https://platform.openai.com/tokenizer) to test specific strings and see how they are translated into tokens. For example, the string `"ChatGPT is great!"` is encoded into six tokens: `["Chat", "G", "PT", " is", " great", "!"]`. The total number of tokens in an API call affects: - How much your API call costs, as you pay per token - How long your API call takes, as writing more tokens takes more time - Whether your API call works at all, as total tokens must be below the model's maximum limit (4097 tokens for `gpt-3.5-turbo`) Both input and output tokens count toward these quantities. For example, if your API call used 10 tokens in the message input and you received 20 tokens in the message output, you would be billed for 30 tokens. Note however that for some models the price per token is different for tokens in the input vs. the output (see the [pricing](https://openai.com/api/pricing) page for more information). To see how many tokens are used by an API call, check the `usage` field in the API response (for example, `response['usage']['total_tokens']`). Chat models like `gpt-3.5-turbo` and `gpt-4-turbo-preview` use tokens in the same way as the models available in the completions API, but because of their message-based formatting, it's more difficult to count how many tokens will be used by a conversation. Below is an example function for counting tokens for messages passed to `gpt-3.5-turbo-0613`. The exact way that messages are converted into tokens may change from model to model. So when future model versions are released, the answers returned by this function may be only approximate. ```python def num_tokens_from_messages(messages, model="gpt-3.5-turbo-0613"): """Returns the number of tokens used by a list of messages.""" try: encoding = tiktoken.encoding_for_model(model) except KeyError: encoding = tiktoken.get_encoding("cl100k_base") if model == "gpt-3.5-turbo-0613": # note: future models may deviate from this num_tokens = 0 for message in messages: num_tokens += ( 4 # every message follows {role/name}\n{content}\n ) for key, value in message.items(): num_tokens += len(encoding.encode(value)) if key == "name": # if there's a name, the role is omitted num_tokens += -1 # role is always required and always 1 token num_tokens += 2 # every reply is primed with assistant return num_tokens raise ValueError( f"num_tokens_from_messages() only supports gpt-3.5-turbo-0613, not {model}." ) ``` Next, create a message and pass it to the function defined above to see the token count, this should match the value returned by the API usage parameter: ```python messages = [ { "role": "system", "content": "You are a helpful, pattern-following assistant that translates corporate jargon into plain English.", }, { "role": "system", "name": "example_user", "content": "New synergies will help drive top-line growth.", }, { "role": "system", "name": "example_assistant", "content": "Things working well together will increase revenue.", }, { "role": "system", "name": "example_user", "content": "Let's circle back when we have more bandwidth to touch base on opportunities for increased leverage.", }, { "role": "system", "name": "example_assistant", "content": "Let's talk later when we're less busy about how to do better.", }, { "role": "user", "content": "This late pivot means we don't have time to boil the ocean for the client deliverable.", }, ] model = "gpt-3.5-turbo-0613" print(f"{num_tokens_from_messages(messages, model)} prompt tokens counted.") # Should show ~126 total_tokens ``` To confirm the number generated by our function above is the same as what the API returns, create a new Chat Completion: ```javascript import OpenAI from "openai"; const client = new OpenAI(); const response = await client.chat.completions.create({ model, messages, temperature: 0, }); console.log(`${response.usage.prompt_tokens} prompt tokens used.`); ``` ```python # example token count from the OpenAI API from openai import OpenAI client = OpenAI() response = client.chat.completions.create( model=model, messages=messages, temperature=0, ) print(f"{response.usage.prompt_tokens} prompt tokens used.") ``` ```java import com.openai.client.OpenAIClient; import com.openai.client.okhttp.OpenAIOkHttpClient; import com.openai.models.chat.completions.ChatCompletionCreateParams; ChatCompletionCreateParams params = ChatCompletionCreateParams.builder() .model("gpt-3.5-turbo-0613") .addUserMessage("Translate this sentence into plain English.") .temperature(0) .build(); var completion = client.chat().completions().create(params); var usage = completion.usage().orElseThrow(() -> new IllegalStateException("No usage returned")); System.out.println(usage.promptTokens() + " prompt tokens used."); ``` To see how many tokens are in a text string without making an API call, use OpenAI’s [tiktoken](https://github.com/openai/tiktoken) Python library. Example code can be found in the OpenAI Cookbook’s guide on [how to count tokens with tiktoken](https://developers.openai.com/cookbook/examples/how_to_count_tokens_with_tiktoken). Each message passed to the API consumes the number of tokens in the content, role, and other fields, plus a few extra for behind-the-scenes formatting. This may change slightly in the future. If a conversation has too many tokens to fit within a model’s maximum limit (for example, more than 4097 tokens for `gpt-3.5-turbo` or more than 128k tokens for `gpt-4o`), you will have to truncate, omit, or otherwise shrink your text until it fits. Beware that if a message is removed from the messages input, the model will lose all knowledge of it. Note that long conversations are more likely to receive incomplete replies. For example, a `gpt-3.5-turbo` conversation that is 4090 tokens long will have its reply cut off after just 6 tokens. ## Parameter details ### Frequency and presence penalties The frequency and presence penalties found in the [Chat Completions API](https://developers.openai.com/api/reference/resources/chat) and [Legacy Completions API](https://developers.openai.com/api/reference/resources/completions) can be used to reduce the likelihood of sampling repetitive sequences of tokens. They work by directly modifying the logits (un-normalized log-probabilities) with an additive contribution. ```python mu[j] = mu[j] - c[j] * alpha_frequency - float(c[j] > 0) * alpha_presence ``` ```ruby mu[j] = mu[j] - c[j] * alpha_frequency - ((c[j] > 0) ? alpha_presence : 0.0) ``` Where: - `mu[j]` is the logits of the j-th token - `c[j]` is how often that token was sampled prior to the current position - The presence penalty subtracts `alpha_presence` if `c[j] > 0` and 0 otherwise - `alpha_frequency` is the frequency penalty coefficient - `alpha_presence` is the presence penalty coefficient As we can see, the presence penalty is a one-off additive contribution that applies to all tokens that have been sampled at least once and the frequency penalty is a contribution that is proportional to how often a particular token has already been sampled. Reasonable values for the penalty coefficients are around 0.1 to 1 if the aim is to just reduce repetitive samples somewhat. If the aim is to strongly suppress repetition, then one can increase the coefficients up to 2, but this can noticeably degrade the quality of samples. Negative values can be used to increase the likelihood of repetition. ### Token log probabilities The [`logprobs`](https://developers.openai.com/api/reference/resources/chat#chat-create-logprobs) parameter found in the [Chat Completions API](https://developers.openai.com/api/reference/resources/chat) and [Legacy Completions API](https://developers.openai.com/api/reference/resources/completions), when requested, provides the log probabilities of each output token, and a limited number of the most likely tokens at each token position alongside their log probabilities. This can be useful in some cases to assess the confidence of the model in its output, or to examine alternative responses the model might have given. ### Other parameters See the full [API reference documentation](https://platform.openai.com/docs/api-reference/chat) to learn more. --- # Agent Builder **Agent Builder** is a visual canvas for building multi-step agent workflows. You can start from templates, drag and drop nodes for each step in your workflow, provide typed inputs and outputs, and preview runs using live data. When you're ready to deploy, embed the workflow into your site with ChatKit, or download the SDK code to run it yourself. OpenAI is deprecating Agent Builder. Existing users can continue using it during the transition window, and the product is scheduled to shut down on November 30, 2026. ChatKit remains available. See the [deprecations page](https://developers.openai.com/api/docs/deprecations#2026-06-03-agent-builder) for the current timeline. Use this guide to learn the process and parts of building agents. ## Agents and workflows To build useful agents, you create workflows for them. A **workflow** is a combination of agents, tools, and control-flow logic. A workflow encapsulates all steps and actions involved in handling your tasks or powering your chats, with working code you can deploy when you're ready. Open Agent Builder There are three main steps in building agents to handle tasks: 1. Design a workflow in [Agent Builder](https://platform.openai.com/agent-builder). This defines your agents and how they'll work. 1. Publish your workflow. It's an object with an ID and versioning. 1. Deploy your workflow. Pass the ID into your [ChatKit](https://developers.openai.com/api/docs/guides/chatkit) integration, or download the Agents SDK code to deploy your workflow yourself. ## Compose with nodes In Agent Builder, insert and connect nodes to create your workflow. Each connection between nodes becomes a typed edge. Click a node to configure its inputs and outputs, observe the data contract between steps, and ensure downstream nodes receive the properties they expect. ### Examples and templates Agent Builder provides templates for common workflow patterns. Start with a template to see how nodes work together, or start from scratch. Here's a homework helper workflow. It uses agents to take questions, reframe them for better answers, route them to other specialized agents, and return an answer. ![prompts chat](https://cdn.openai.com/API/docs/images/homework-helper2.png) ### Available nodes Nodes are the building blocks for agents. To see all available nodes and their configuration options, see the [node reference documentation](https://developers.openai.com/api/docs/guides/node-reference). ### Preview and debug As you build, you can test your workflow by using the **Preview** feature. Here, you can interactively run your workflow, attach sample files, and observe the execution of each node. ### Safety and risks Building agent workflows comes with risks, like prompt injection and data leakage. See [safety in building agents](https://developers.openai.com/api/docs/guides/agent-builder-safety) to learn about and help mitigate the risks of agent workflows. ### Evaluate your workflow Run [trace graders](https://developers.openai.com/api/docs/guides/trace-grading) inside of Agent Builder. In the top navigation, click **Evaluate**. Here, you can select a trace (or set of traces) and run custom graders to assess overall workflow performance. ## Publish your workflow Agent Builder autosaves your work as you go. When you're happy with your workflow, publish it to create a new major version that acts as a snapshot. You can then use your workflow in [ChatKit](https://developers.openai.com/api/docs/guides/chatkit), an OpenAI framework for embedding chat experiences. You can create new versions or specify an older version in your API calls. ## Deploy in your product When you're ready to implement the agent workflow you created, click **Code** in the top navigation. You have two options for implementing your workflow in production: **ChatKit**: Follow the [ChatKit quickstart](https://developers.openai.com/api/docs/guides/chatkit) and pass in your workflow ID to embed this workflow into your application. If you're not sure, we recommend this option. **Advanced integration**: Copy the workflow code and use it anywhere. You can run ChatKit on your own infrastructure and use the Agents SDK to build and customize agent chat experiences. ## Next steps Now that you've created an agent workflow, bring it into your product with ChatKit. - [ChatKit quickstart](https://developers.openai.com/api/docs/guides/chatkit) → - [Advanced integration](https://developers.openai.com/api/docs/guides/custom-chatkit) → --- # Agent definitions An agent is the core unit of an SDK-based workflow. It packages a model, instructions, and optional runtime behavior such as tools, guardrails, MCP servers, handoffs, and structured outputs. ## What belongs on an agent Use agent configuration for decisions that are intrinsic to that specialist: | Property | Use it for | Read next | | ----------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------- | ---------------------------------------------------------------------------------------- | | `name` | Human-readable identity in traces and tool/handoff surfaces | This page | | `instructions` | The job, constraints, and style for that agent | This page | | `prompt` | Stored prompt configuration for Responses-based runs | [Models and providers](https://developers.openai.com/api/docs/guides/agents/models) | | `model` and model settings | Choosing the model and tuning behavior | [Models and providers](https://developers.openai.com/api/docs/guides/agents/models) | | `tools` | Capabilities the agent can call directly | [Using tools](https://developers.openai.com/api/docs/guides/tools#usage-in-the-agents-sdk) | | `handoffDescription` in TypeScript or `handoff_description` in Python | Hinting when another agent should delegate here | [Orchestration and handoffs](https://developers.openai.com/api/docs/guides/agents/orchestration) | | `handoffs` | Delegating to another agent | [Orchestration and handoffs](https://developers.openai.com/api/docs/guides/agents/orchestration) | | `outputType` in TypeScript or `output_type` in Python | Returning structured output instead of plain text | This page | | Guardrails and approvals | Validation, blocking, and review flows | [Guardrails and human review](https://developers.openai.com/api/docs/guides/agents/guardrails-approvals) | | MCP servers and hosted MCP tools | Attaching MCP-backed capabilities | [Integrations and observability](https://developers.openai.com/api/docs/guides/agents/integrations-observability#mcp) | ## Start with one focused agent Define the smallest agent that can own a clear task. Add more agents only when you need separate ownership, different instructions, different tool surfaces, or different approval policies. Define a single agent ```javascript import { Agent, tool } from "@openai/agents"; import { z } from "zod"; const getWeather = tool({ name: "get_weather", description: "Return the weather for a given city.", parameters: z.object({ city: z.string() }), async execute({ city }) { return `The weather in ${city} is sunny.`; }, }); const agent = new Agent({ name: "Weather bot", instructions: "You are a helpful weather bot.", model: "gpt-6-astra", tools: [getWeather], }); ``` ```python from agents import Agent, function_tool @function_tool def get_weather(city: str) -> str: """Return the weather for a given city.""" return f"The weather in {city} is sunny." agent = Agent( name="Weather bot", instructions="You are a helpful weather bot.", model="gpt-6-astra", tools=[get_weather], ) ``` ## Shape instructions, handoffs, and outputs Three configuration choices deserve extra care: - Start with static `instructions`. When the guidance depends on the current user, tenant, or runtime context, switch to a dynamic instructions callback instead of stitching strings together at the call site. - Keep `handoffDescription` in TypeScript or `handoff_description` in Python short and concrete so routing agents know when to pick this specialist. - Use `outputType` in TypeScript or `output_type` in Python when downstream code needs typed data rather than free-form prose. Return structured output ```javascript import { Agent, run } from "@openai/agents"; import { z } from "zod"; const calendarEvent = z.object({ name: z.string(), date: z.string(), participants: z.array(z.string()), }); const agent = new Agent({ name: "Calendar extractor", instructions: "Extract calendar events from text.", outputType: calendarEvent, }); const result = await run(agent, "Dinner with Priya and Sam on Friday."); console.log(result.finalOutput); ``` ```python import asyncio from pydantic import BaseModel from agents import Agent, Runner class CalendarEvent(BaseModel): name: str date: str participants: list[str] agent = Agent( name="Calendar extractor", instructions="Extract calendar events from text.", output_type=CalendarEvent, ) async def main() -> None: result = await Runner.run( agent, "Dinner with Priya and Sam on Friday.", ) print(result.final_output) if __name__ == "__main__": asyncio.run(main()) ``` Use `prompt` when you want to reference a stored prompt configuration from the Responses API instead of embedding the entire system prompt in code. ## Keep local context separate from model context The SDK lets you pass application state and dependencies into a run without sending them to the model. Use this for data like authenticated user info, database clients, loggers, and helper functions. Pass local context to tools ```javascript import { Agent, run, tool } from "@openai/agents"; import { z } from "zod"; const fetchUserAge = tool({ name: "fetch_user_age", description: "Return the age of the current user.", parameters: z.object({}), // TypeScript users can type this as RunContext<{ name: string; uid: number }>. async execute(_args, runContext) { return `User ${runContext?.context.name} is 47 years old`; }, }); const agent = new Agent({ name: "Assistant", tools: [fetchUserAge], }); const result = await run(agent, "What is the age of the user?", { context: { name: "John", uid: 123 }, }); console.log(result.finalOutput); ``` ```python import asyncio from dataclasses import dataclass from agents import Agent, RunContextWrapper, Runner, function_tool @dataclass class UserInfo: name: str uid: int @function_tool async def fetch_user_age(wrapper: RunContextWrapper[UserInfo]) -> str: """Fetch the age of the current user.""" return f"The user {wrapper.context.name} is 47 years old." agent = Agent[UserInfo]( name="Assistant", tools=[fetch_user_age], ) async def main() -> None: result = await Runner.run( agent, "What is the age of the user?", context=UserInfo(name="John", uid=123), ) print(result.final_output) if __name__ == "__main__": asyncio.run(main()) ``` The important boundary is: - Conversation history is what the model sees. - Run context is what your code sees. If the model needs a fact, put it in instructions, input, retrieval, or a tool. If only your runtime needs it, keep it in local context. ## When to split one agent into several Split an agent when one specialist shouldn't own the full reply or when separate capabilities are materially different. Common reasons are: - A specialist needs a different tool or MCP surface. - A specialist needs a different approval policy or guardrail. - One branch of the workflow needs a different model or output style. - You want explicit routing in traces rather than a single large prompt. ## Next steps Once one specialist is defined cleanly, move to the guide that matches the next design question. [Models and providers Choose models, defaults, and transport strategy for this agent.](https://developers.openai.com/api/docs/guides/agents/models) [Using tools Add capabilities the agent can call directly.](https://developers.openai.com/api/docs/guides/tools#usage-in-the-agents-sdk) [Orchestration and handoffs Choose how specialists collaborate once one agent is no longer enough.](https://developers.openai.com/api/docs/guides/agents/orchestration) [Running agents Understand the runtime loop, state, and streaming behavior.](https://developers.openai.com/api/docs/guides/agents/running-agents) --- # Agents Agents can plan and complete tasks using tools, work with other agents, and maintain context across steps. Choose a runtime based on where you want orchestration to run and who should manage the state between tasks. ## Choose your starting point | You want to | Start here | | ------------------------------------------------------------------------------------ | ------------------------------------------------------ | | Run an agent with the Codex harness managed by OpenAI | [Agents API](https://developers.openai.com/api/docs/guides/agents-api/quickstart) | | Control the agent loop in your application with reusable agents, tools, and handoffs | [Agents SDK](https://developers.openai.com/api/docs/guides/agents/quickstart) | | Work directly with model responses and control your integration | [Responses API](https://developers.openai.com/api/docs/guides/migrate-to-responses) | | Add an embedded chat experience | [ChatKit](https://developers.openai.com/api/docs/guides/chatkit) | ## Compare agent runtime options | | Agents API | Agents SDK | Responses API | | ------------------------ | ------------------------------------------------------------------------------- | ------------------------------------------------------------------- | --------------------------------------------------------- | | **Use for** | Long-running tasks where OpenAI manages the agent and saves its progress | Building agents with custom tools and workflows in your application | Calling models directly or building an agent from scratch | | Where the agent runs | OpenAI runs a managed Codex harness | The SDK runs inside your application | Your application, with optional hosted orchestration | | Agent integration effort | Low | Medium | High | | State between tasks | Saved session configuration, turns, and items | Your storage and SDK sessions, or Responses conversation state | Manual history, response chaining, or Conversations | | Tool execution | Service-connected tools, application function handlers, and an optional sandbox | Tools and integrations configured in your application | Hosted tools and tools your application runs | | Execution environment | OpenAI hosted sandbox, self-hosted sandbox, or no sandbox | Your runtime and sandbox provider integrations | Your own execution environment | | Start here | [Agents API overview](https://developers.openai.com/api/docs/guides/agents-api/overview) | [Agents SDK overview](https://developers.openai.com/api/docs/guides/agents/sdk) | [Responses guide](https://developers.openai.com/api/docs/guides/migrate-to-responses) | The Agents API runs the Codex harness and manages the underlying agent infrastructure so you can focus on what your agents do. It includes automatic context compaction, multi-agent orchestration, programmatic tool calling, and support for MCP servers. See [Architecture](https://developers.openai.com/api/docs/guides/agents-api/architecture). The Agents SDK gives your application control over deployment, storage, approvals, and runtime integration. Its runner handles the agent loop and handoffs. See [Running agents](https://developers.openai.com/api/docs/guides/agents/running-agents). ## Add tools, skills, and prompt caching Tool design, reusable skills, and prompt caching apply across agent workflows. Their configuration and lifecycle can differ by API. - Start with [Using tools](https://developers.openai.com/api/docs/guides/tools) for function calling, MCP, and hosted capabilities. - Read [Programmatic Tool Calling](https://developers.openai.com/api/docs/guides/tools-programmatic-tool-calling) for orchestration with JavaScript and the configuration for each API. - Use [Skills](https://developers.openai.com/api/docs/guides/tools-skills) for reusable instructions and the supported loading mechanisms. - Read [Prompt caching](https://developers.openai.com/api/docs/guides/prompt-caching) for shared caching behavior, then [Agents API observability and usage](https://developers.openai.com/api/docs/guides/agents-api/observability) for session accounting. An Agents API session, an SDK session, a Responses conversation, and a sandbox are different resources. Follow the state and cleanup instructions for the runtime you choose. --- # Agents API The Agents API gives your application access to the Codex harness through an OpenAI-managed API. OpenAI manages sessions, orchestration, context compaction, and recovery while your application provides tools and chooses its execution environment. Agents can operate in a sandbox where they can execute code, edit files, connect to MCP servers, and produce artifacts. ## Pricing Model usage is billed at the selected model's [API rates](https://developers.openai.com/api/docs/pricing). OpenAI tools use their [standard rates](https://developers.openai.com/api/docs/pricing#built-in-tools), and OpenAI-hosted sandboxes use standard [container rates](https://developers.openai.com/api/docs/pricing#built-in-tools). ## Try an example Try these complete examples: - [Create and run a directory-tree script](https://developers.openai.com/api/docs/guides/agents-api/quickstart#1-run-a-task) in an OpenAI-hosted sandbox. - [Compare release notes with subagents](https://developers.openai.com/api/docs/guides/agents-api/multi-agent#example-compare-release-notes) and combine their findings into one answer. Explore complete applications: - [Incident response agent](https://developers.openai.com/showcase/agents-api-sev-bot): investigate alerts and request approval for recovery actions. - [Slack bot](https://developers.openai.com/showcase/agents-api-slack-bot): investigate requests using connected workplace tools. - [Data analyst](https://developers.openai.com/showcase/agents-api-data-analyst): answer warehouse questions with read-only SQL. - [GitHub issue investigator](https://developers.openai.com/showcase/agents-api-github-issues): reproduce reported bugs and share findings on GitHub. - [Document reviewer](https://developers.openai.com/showcase/agents-api-document-review): review documents with policy skills and specialist agents. ## Core concepts The Agents API is built around four main concepts: - **Agent:** The model, instructions, tools, and MCP servers available to the agent. - **Environment:** An optional sandbox or computer where the agent accesses files, loads skills, and runs commands. - **Session:** A durable instance of an agent that works on tasks and responds to input. - **Events and items:** The inputs sent to an agent and the output produced during a session. ### A session from start to finish Start with an OpenAI-hosted sandbox in the [quickstart](https://developers.openai.com/api/docs/guides/agents-api/quickstart): 1. **Create a session.** Configure the agent; OpenAI provisions its environment. 2. **Give it a task.** User input starts a turn of work once the environment is ready. 3. **Follow progress.** Stream output or use webhooks to learn when the agent finishes or needs input. 4. **Continue or steer.** Send another task to the same session, or guide the agent during its current turn. With an OpenAI-hosted session, your application sends input and receives events, while OpenAI runs the agent and provisions and manages its sandbox. See [environment options](https://developers.openai.com/api/docs/guides/agents-api/configuration#environment-settings) for setup and limitations. Your application starts sessions and receives events and output from the Agents API. OpenAI runs the managed Codex harness and provisions and manages its sandbox. ## What the managed harness provides The managed Codex harness supports: - Running commands and code in a sandbox. - Applying relevant skills and instructions. - Connecting to external data through tools or MCP. - Steering the agent while it works. - Summarizing previous work to manage its context window. - Breaking work into subtasks and delegating to subagents. - Resuming a session where it left off. Check the [quickstart prerequisites](https://developers.openai.com/api/docs/guides/agents-api/quickstart#prerequisites) for API-key permissions and SDK setup. Configure these capabilities when you create a session: Configure managed-harness capabilities ```javascript import OpenAI from "openai"; const client = new OpenAI(); const session = await client.beta.agents.sessions.create({ agent: { model: "gpt-6-astra", instructions: "Use the OpenAI documentation MCP and web search to answer technical questions accurately. Delegate independent research tasks to subagents when useful.", tools: [ { type: "programmatic_tool_calling" }, { type: "mcp", server_label: "openai_docs", transport: { type: "http", server_url: "https://developers.openai.com/mcp", }, }, { type: "web_search" }, ], multi_agent: { enabled: true, max_concurrent_subagents: 4 }, }, environment: { type: "self_hosted", workspace_directory: "/workspace", capability_directories: ["/workspace/capabilities/skills"], }, input: [ { role: "user", content: [ { type: "input_text", text: "Research how to connect an MCP server to an OpenAI agent, check for recent updates, and summarize the recommended setup.", }, ], }, ], }); console.log(session.id); ``` ```python from openai import OpenAI client = OpenAI() session = client.beta.agents.sessions.create( agent={ "model": "gpt-6-astra", "instructions": "Use the OpenAI documentation MCP and web search to answer technical questions accurately. Delegate independent research tasks to subagents when useful.", "tools": [ {"type": "programmatic_tool_calling"}, { "type": "mcp", "server_label": "openai_docs", "transport": { "type": "http", "server_url": "https://developers.openai.com/mcp", }, }, {"type": "web_search"}, ], "multi_agent": {"enabled": True, "max_concurrent_subagents": 4}, }, environment={ "type": "self_hosted", "workspace_directory": "/workspace", "capability_directories": ["/workspace/capabilities/skills"], }, input=[ { "role": "user", "content": [ { "type": "input_text", "text": "Research how to connect an MCP server to an OpenAI agent, check for recent updates, and summarize the recommended setup.", } ], } ], ) print(session.id) ``` ```go import ( "context" "fmt" "github.com/openai/openai-go/v3" ) ctx := context.Background() client := openai.NewClient() session, err := client.Beta.Agents.Sessions.New(ctx, openai.BetaAgentSessionNewParams{Agent: openai.BetaAgentSessionNewParamsAgent{Model: openai.String("gpt-6-astra"), Instructions: openai.String("Use the OpenAI documentation MCP and web search to answer technical questions accurately. Delegate independent research tasks to subagents when useful."), Tools: []openai.AgentToolParamUnion{openai.AgentToolParamUnion{OfParamProgrammaticToolCalling: &openai.AgentToolParamProgrammaticToolCalling{}}, openai.AgentToolParamUnion{OfParamMcp: &openai.AgentToolParamMcp{ServerLabel: "openai_docs", Transport: openai.McpTransportParamUnion{OfParamHTTP: &openai.McpTransportParamHTTP{ServerURL: "https://developers.openai.com/mcp"}}}}, openai.AgentToolParamUnion{OfParamWebSearch: &openai.AgentToolParamWebSearch{}}}, MultiAgent: openai.MultiAgentConfigParam{Enabled: true, MaxConcurrentSubagents: openai.Int(4)}}, Environment: openai.EnvironmentParamUnion{OfParamSelfHosted: &openai.EnvironmentParamSelfHosted{WorkspaceDirectory: "/workspace", CapabilityDirectories: []string{"/workspace/capabilities/skills"}}}, Input: openai.BetaAgentSessionNewParamsInputUnion{OfArrayOfInputMessages: []openai.AgentSessionInputMessageParam{openai.AgentSessionInputMessageParam{Content: []openai.InputContentParamUnion{openai.InputContentParamUnion{OfParamInputText: &openai.InputContentParamInputText{Text: "Research how to connect an MCP server to an OpenAI agent, check for recent updates, and summarize the recommended setup."}}}}}}}) if err != nil { panic(err) } fmt.Println(session.ID) ``` ```java import com.openai.client.OpenAIClient; import com.openai.client.okhttp.OpenAIOkHttpClient; import com.openai.models.beta.agents.AgentToolParam; import com.openai.models.beta.agents.EnvironmentParam; import com.openai.models.beta.agents.McpTransportParam; import com.openai.models.beta.agents.MultiAgentConfigParam; import com.openai.models.beta.agents.sessions.SessionCreateParams; import java.util.List; OpenAIClient client = OpenAIOkHttpClient.fromEnv(); var session = client .beta() .agents() .sessions() .create( SessionCreateParams.builder() .agent( SessionCreateParams.Agent.builder() .model("gpt-6-astra") .instructions( "Use the OpenAI documentation MCP and web search to answer" + " technical questions accurately. Delegate independent" + " research tasks to subagents when useful.") .addTool(AgentToolParam.ProgrammaticToolCalling.builder().build()) .addTool( AgentToolParam.Mcp.builder() .serverLabel("openai_docs") .transport( McpTransportParam.Http.builder() .serverUrl("https://developers.openai.com/mcp") .build()) .build()) .addTool(AgentToolParam.WebSearch.builder().build()) .multiAgent( MultiAgentConfigParam.builder() .enabled(true) .maxConcurrentSubagents(4L) .build()) .build()) .environment( EnvironmentParam.SelfHosted.builder() .workspaceDirectory("/workspace") .capabilityDirectories(List.of("/workspace/capabilities/skills")) .build()) .input( "Research how to connect an MCP server to an OpenAI agent, check for recent" + " updates, and summarize the recommended setup.") .build()); System.out.println(session.id()); ``` ```ruby require "openai" client = OpenAI::Client.new session = client.beta.agents.sessions.create( agent: { model: "gpt-6-astra", instructions: "Use the OpenAI documentation MCP and web search to answer technical questions accurately. Delegate independent research tasks to subagents when useful.", tools: [ { type: "programmatic_tool_calling" }, { type: "mcp", server_label: "openai_docs", transport: { type: "http", server_url: "https://developers.openai.com/mcp" } }, { type: "web_search" } ], multi_agent: { enabled: true, max_concurrent_subagents: 4 } }, environment: { type: "self_hosted", workspace_directory: "/workspace", capability_directories: ["/workspace/capabilities/skills"] }, input: [ { role: "user", content: [ { type: "input_text", text: "Research how to connect an MCP server to an OpenAI agent, check for recent updates, and summarize the recommended setup." } ] } ] ) puts session.id ``` ```bash curl -sS -X POST "https://api.openai.com/v1/agents/sessions" \ -H "OpenAI-Beta: agents=v1" \ -H "Authorization: Bearer $OPENAI_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "agent": { "model": "gpt-6-astra", "instructions": "Use the OpenAI documentation MCP and web search to answer technical questions accurately. Delegate independent research tasks to subagents when useful.", "tools": [ { "type": "programmatic_tool_calling" }, { "type": "mcp", "server_label": "openai_docs", "transport": { "type": "http", "server_url": "https://developers.openai.com/mcp" } }, { "type": "web_search" } ], "multi_agent": { "enabled": true, "max_concurrent_subagents": 4 } }, "environment": { "type": "self_hosted", "workspace_directory": "/workspace", "capability_directories": ["/workspace/capabilities/skills"] }, "input": [ { "role": "user", "content": [ { "type": "input_text", "text": "Research how to connect an MCP server to an OpenAI agent, check for recent updates, and summarize the recommended setup." } ] } ] }' ``` For a runtime comparison, see the [Agents overview](https://developers.openai.com/api/docs/guides/agents#compare-agent-runtimes). The Agents API retains session state so you can continue work across turns without rebuilding the conversation context. You can delete sessions and published artifacts when you no longer need them. The Agents API currently supports data residency only in the United States and does not support Zero Data Retention (ZDR). Choosing a self-hosted sandbox does not make the Agents API ZDR-eligible. See [Data controls in the OpenAI platform](https://developers.openai.com/api/docs/guides/your-data#storage-requirements-and-retention-controls-per-endpoint) for details on data residency and retention. --- # Agents API quickstart Build a coding assistant that writes `tree.py`, runs it, and shows a directory tree. OpenAI manages the agent, its conversation, and the sandbox where it works. ## Prerequisites Create an [application API key](https://platform.openai.com/api-keys) in your OpenAI Platform project. Grant `api.agents.read` and `api.agents.write` for session operations, plus `api.responses.write` for model inference, then export it: ```bash export OPENAI_API_KEY="your-api-key" ``` Keep this key outside the agent's sandbox. See [OpenAI-hosted sandboxes](https://developers.openai.com/api/docs/guides/agents-api/environments/openai-hosted#configure-the-sandbox) for sandbox configuration and limits. Requests require the `OpenAI-Beta: agents=v1` header. The OpenAI SDKs add it automatically; include it explicitly when using cURL. ## 1. Run a task Choose a language, install the OpenAI SDK, and run the example. The SDK examples use the `beta.agents` namespace. The request creates a session, submits a task, and streams progress. Python Install or update the Python SDK: ```bash pip install --upgrade openai ``` Save the example as `quickstart.py`: Create and run tree.py ```python from openai import OpenAI with OpenAI() as client: with client.beta.agents.sessions.create( agent={ "model": "gpt-6-astra", "instructions": "Write clean code, run it, and report the actual output.", }, environment={"type": "openai_hosted"}, input="Create tree.py, a Python script that prints a readable tree of the files in the current directory. Run it and show me the output.", stream=True, ) as events: for event in events: print(event.to_json(indent=None), flush=True) ``` Run it from your terminal: ```bash python quickstart.py ``` JavaScript Install the JavaScript SDK: ```bash npm install openai ``` Save the example as `quickstart.mjs`: Create and run tree.py ```javascript import OpenAI from "openai"; const client = new OpenAI(); const events = await client.beta.agents.sessions.create({ agent: { model: "gpt-6-astra", instructions: "Write clean code, run it, and report the actual output.", }, environment: { type: "openai_hosted" }, input: "Create tree.py, a Python script that prints a readable tree of the files in the current directory. Run it and show me the output.", stream: true, }); try { for await (const event of events) { console.log(JSON.stringify(event)); } } finally { events.controller.abort(); } ``` Run it from your terminal: ```bash node quickstart.mjs ``` Go In a new directory, create a Go module and install the SDK: ```bash go mod init agents-quickstart go get github.com/openai/openai-go/v3@latest ``` Save the example as `main.go`: Create and run tree.py ```go import ( "context" "fmt" "github.com/openai/openai-go/v3" ) ctx := context.Background() client := openai.NewClient() events := client.Beta.Agents.Sessions.NewStreaming(ctx, openai.BetaAgentSessionNewParams{ Agent: openai.BetaAgentSessionNewParamsAgent{ Model: openai.String("gpt-6-astra"), Instructions: openai.String("Write clean code, run it, and report the actual output."), }, Environment: openai.EnvironmentParamUnion{OfParamOpenAIHosted: &openai.EnvironmentParamOpenAIHosted{}}, Input: openai.BetaAgentSessionNewParamsInputUnion{ OfString: openai.String("Create tree.py, a Python script that prints a readable tree of the files in the current directory. Run it and show me the output."), }, }) defer events.Close() if events.Err() != nil { panic(events.Err()) } for events.Next() { event := events.Current() fmt.Println(event.RawJSON()) } if err := events.Err(); err != nil { panic(err) } ``` Run it from your terminal: ```bash go run . ``` Java Add the OpenAI SDK to your Maven project's `pom.xml`: ```xml com.openai openai-java ${apiReferencePackageVersions.java} ``` Save the example as `src/main/java/AgentsApiSessionsStreamConversationExample.java`: Create and run tree.py ```java import com.fasterxml.jackson.databind.json.JsonMapper; import com.openai.client.OpenAIClient; import com.openai.client.okhttp.OpenAIOkHttpClient; import com.openai.core.http.StreamResponse; import com.openai.models.beta.agents.AgentSessionEvent; import com.openai.models.beta.agents.EnvironmentParam; import com.openai.models.beta.agents.sessions.SessionCreateParams; OpenAIClient client = OpenAIOkHttpClient.fromEnv(); var json = new JsonMapper(); try (StreamResponse events = client .beta() .agents() .sessions() .createStreaming( SessionCreateParams.builder() .agent( SessionCreateParams.Agent.builder() .model("gpt-6-astra") .instructions("Write clean code, run it, and report the actual output.") .build()) .environment(EnvironmentParam.OpenAIHosted.builder().build()) .input( "Create tree.py, a Python script that prints a readable tree of the files" + " in the current directory. Run it and show me the output.") .build())) { var iterator = events.stream().iterator(); while (iterator.hasNext()) { var event = iterator.next(); System.out.println(json.writeValueAsString(event)); } } ``` Run it from your terminal: ```bash mvn compile exec:java -Dexec.mainClass=AgentsApiSessionsStreamConversationExample ``` Ruby Install the Ruby SDK: ```bash gem install openai ``` Save the example as `quickstart.rb`: Create and run tree.py ```ruby require "openai" require "json" client = OpenAI::Client.new events = client.beta.agents.sessions.create_streaming( agent: { model: "gpt-6-astra", instructions: "Write clean code, run it, and report the actual output." }, environment: { type: "openai_hosted" }, input: "Create tree.py, a Python script that prints a readable tree of the files in the current directory. Run it and show me the output." ) begin events.each do |event| puts JSON.generate(event.to_h) end ensure events.close end ``` Run it from your terminal: ```bash ruby quickstart.rb ``` cURL Use cURL from your terminal; no SDK installation is needed: Create and run tree.py ```bash curl --no-buffer --fail-with-body https://api.openai.com/v1/agents/sessions \\\n -H "OpenAI-Beta: agents=v1" \\\n -H "Authorization: Bearer $OPENAI_API_KEY" \\\n -H "Content-Type: application/json" \\\n -d \'{\n "agent": {\n "model": "gpt-6-astra",\n "instructions": "Write clean code, run it, and report the actual output."\n },\n "environment": { "type": "openai_hosted" },\n "input": "Create tree.py, a Python script that prints a readable tree of the files in the current directory. Run it and show me the output.",\n "stream": true\n }\' ``` ## 2. Follow progress The terminal shows streamed events. The SDK examples print JSON; cURL shows the raw event stream. On a successful run, the agent creates `tree.py`, executes it, and reports a directory tree containing that file. Other files and output depend on the sandbox. Look for `agent.session.turn.completed`, then check the agent's reported execution result. A completed turn does not guarantee every tool succeeded. Events ending in `turn.failed`, `turn.cancelled`, or `session.failed` indicate failure or cancellation; `agent.session.idle` alone does not mean success. If the stream disconnects early, [retrieve the session and its saved items](https://developers.openai.com/api/docs/guides/agents-api/sessions#how-to-recover-a-disconnected-stream) before retrying. ## 3. Continue the session Save the `session_id` from the events. Use it to [send a follow-up](https://developers.openai.com/api/docs/guides/agents-api/sessions#send-input) such as “Add a maximum-depth option to `tree.py`, run it, and show me the output.” Open the event stream before sending follow-up input so you don't miss early events. ## 4. Clean up Keep the session for more tasks, or delete it when you're done. [Save any files you need](https://developers.openai.com/api/docs/guides/agents-api/environments/files) first. Replace the illustrative `sess_123` value in the example with the session ID you saved. Python Delete the session ```python # Replace the illustrative IDs and URLs below with your own resource values. from openai import OpenAI def delete_session(client: OpenAI, session_id: str): return client.beta.agents.sessions.delete(session_id) if __name__ == "__main__": result = delete_session(OpenAI(), "sess_123") print(result.to_json()) ``` JavaScript Delete the session ```javascript // Replace the illustrative IDs and URLs below with your own resource values. import OpenAI from "openai"; async function deleteSession(client, sessionId) { return client.beta.agents.sessions.delete(sessionId); } const result = await deleteSession(new OpenAI(), "sess_123"); console.log(result); ``` Go Delete the session ```go // Replace the illustrative IDs and URLs below with your own resource values. package main import ( "context" "fmt" "github.com/openai/openai-go/v3" ) func deleteSession(ctx context.Context, client *openai.Client, sessionID string) (*openai.AgentSessionDeleted, error) { return client.Beta.Agents.Sessions.Delete(ctx, sessionID) } func main() { client := openai.NewClient() result, err := deleteSession(context.Background(), &client, "sess_123") if err != nil { panic(err) } fmt.Println(result) } ``` Java Delete the session ```java // Replace the illustrative IDs and URLs below with your own resource values. import com.openai.client.OpenAIClient; import com.openai.client.okhttp.OpenAIOkHttpClient; import com.openai.models.beta.agents.AgentSessionDeleted; import com.openai.models.beta.agents.sessions.SessionDeleteParams; public final class AgentsApiSessionsDeleteSessionExample { public static AgentSessionDeleted deleteSession(OpenAIClient client, String sessionId) { return client .beta() .agents() .sessions() .delete(SessionDeleteParams.builder().sessionId(sessionId).build()); } public static void main(String[] args) { var result = deleteSession(OpenAIOkHttpClient.fromEnv(), "sess_123"); System.out.println(result); } } ``` Ruby Delete the session ```ruby # Replace the illustrative IDs and URLs below with your own resource values. require "openai" def delete_session(client, session_id) client.beta.agents.sessions.delete(session_id) end puts delete_session(OpenAI::Client.new, "sess_123") ``` cURL Delete the session ```bash curl -X DELETE "https://api.openai.com/v1/agents/sessions/sess_123" \\\n -H "OpenAI-Beta: agents=v1" \\\n -H "Authorization: Bearer $OPENAI_API_KEY" ``` ## Next steps - [Explore example applications](https://developers.openai.com/api/docs/guides/agents-api/overview#try-an-example). - [Configure an OpenAI-hosted sandbox](https://developers.openai.com/api/docs/guides/agents-api/environments/openai-hosted): add packages and input files, control network access, and download artifacts. - [Compare release notes with subagents](https://developers.openai.com/api/docs/guides/agents-api/multi-agent#example-compare-release-notes). - [Work with files and artifacts](https://developers.openai.com/api/docs/guides/agents-api/environments/files). - [Choose an environment](https://developers.openai.com/api/docs/guides/agents-api/configuration#environment-settings), or [connect your own sandbox](https://developers.openai.com/api/docs/guides/agents-api/environments/self-hosted). --- # Agents SDK Agents can plan and complete tasks using tools, work with other agents, and maintain context across steps. ## Get your first agent running Start with the [Agents SDK quickstart](https://developers.openai.com/api/docs/guides/agents/quickstart) to install the SDK, define one agent, and run it. Once that works, return here to choose the next capability your application needs. ## Get the Agents SDK Use the GitHub repositories for more examples, issues, and language-specific reference details. [TypeScript SDK Open the TypeScript SDK repository on GitHub.](https://github.com/openai/openai-agents-js) [Python SDK Open the Python SDK repository on GitHub.](https://github.com/openai/openai-agents-python) ## Choose your starting point | If you want to | Start here | Why | | ----------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------- | | Build a code-first agent app | [Quickstart](https://developers.openai.com/api/docs/guides/agents/quickstart) | This is the shortest path to a working SDK integration. | | Define one specialist cleanly | [Agent definitions](https://developers.openai.com/api/docs/guides/agents/define-agents) | Start here when you are still shaping the contract for a single agent. | | Choose models, defaults, and transport | [Models and providers](https://developers.openai.com/api/docs/guides/agents/models) | Use this when model choice, provider setup, or transport strategy affects the workflow. | | Understand the runtime loop and state | [Running agents](https://developers.openai.com/api/docs/guides/agents/running-agents) | This is where the agent loop, streaming, and continuation strategies live. | | Run work in a container-based environment | [Sandbox agents](https://developers.openai.com/api/docs/guides/agents/sandboxes) | Use this when the agent needs files, commands, packages, snapshots, mounts, or provider links. | | Design specialist ownership | [Orchestration and handoffs](https://developers.openai.com/api/docs/guides/agents/orchestration) | Use this when you need more than one agent and must decide who owns the reply. | | Add validation or human review | [Guardrails and human review](https://developers.openai.com/api/docs/guides/agents/guardrails-approvals) | Use this when the workflow should block or pause before risky work continues. | | Understand what a run returns | [Results and state](https://developers.openai.com/api/docs/guides/agents/results) | This page explains final output, resumable state, and next-turn surfaces. | | Add hosted tools, function tools, or MCP | [Using tools](https://developers.openai.com/api/docs/guides/tools#usage-in-the-agents-sdk) and [Integrations and observability](https://developers.openai.com/api/docs/guides/agents/integrations-observability) | Tool semantics live in the platform tools docs; SDK-specific MCP and tracing live here. | | Inspect and improve runs | [Integrations and observability](https://developers.openai.com/api/docs/guides/agents/integrations-observability) and [evaluate agent workflows](https://developers.openai.com/api/docs/guides/agent-evals) | Use traces for debugging first, then move into evaluation loops. | | Build a voice-first workflow | [Voice agents](https://developers.openai.com/api/docs/guides/voice-agents) | Use the SDK voice pipeline and realtime agent patterns. | ## Build with the SDK Use the SDK track when your server owns deployment, tool implementations, state storage, and approval decisions, while the SDK runs the agent loop and invokes those tools. That path is the best fit when you want: - typed application code in TypeScript or Python - direct control over tools, MCP servers, and runtime behavior - custom storage or server-managed conversation strategies - tight integration with existing product logic or infrastructure A typical SDK reading order is: - Start with [Quickstart](https://developers.openai.com/api/docs/guides/agents/quickstart) to get one working run on screen. - Use [Agent definitions](https://developers.openai.com/api/docs/guides/agents/define-agents) and [Models and providers](https://developers.openai.com/api/docs/guides/agents/models) to shape one specialist cleanly. - Continue to [Running agents](https://developers.openai.com/api/docs/guides/agents/running-agents), [Orchestration and handoffs](https://developers.openai.com/api/docs/guides/agents/orchestration), and [Guardrails and human review](https://developers.openai.com/api/docs/guides/agents/guardrails-approvals) as the workflow grows more complex. - Use [Results and state](https://developers.openai.com/api/docs/guides/agents/results) and [Integrations and observability](https://developers.openai.com/api/docs/guides/agents/integrations-observability) when application logic depends on the run object or deeper visibility into behavior. ## Compare agent runtime options Use the [Agents overview](https://developers.openai.com/api/docs/guides/agents#compare-agent-runtimes) to compare the Agents SDK, Agents API, and Responses API. The Agents SDK runs in your application; the Agents API runs a managed harness in OpenAI's service. --- # API deployment checklist | Contents | Expected impact | | ------------------------------------------------------------------------------- | ----------------------------------- | | [Use the Responses API](#use-the-responses-api) | Quality, cost, latency, reliability | | [Choose a GPT-5.6 model](#choose-a-gpt-56-model) | Quality, cost, latency | | [Set up `reasoning.effort`](#set-up-reasoningeffort) | Quality, cost, latency | | [Set up `text.verbosity`](#set-up-textverbosity) | Quality, cost, latency | | [Set up the assistant `phase` parameter](#set-up-the-assistant-phase-parameter) | Quality, cost | | [Use `tool_search`](#use-toolsearch) | Cost, latency | | [Use Programmatic Tool Calling](#use-programmatic-tool-calling) | Quality, cost, latency | | [Use Multi-agent for parallel work](#use-multi-agent-for-parallel-work) | Quality, cost, latency | | [Leverage built-in tools](#leverage-built-in-tools) | Quality | | [Leverage compaction](#leverage-compaction) | Cost | | [Optimize prompt caching](#optimize-prompt-caching) | Latency, cost | | [Use `reasoning.encrypted_content`](#use-reasoningencryptedcontent) | Quality, latency | | [Set image detail intentionally](#set-image-detail-intentionally) | Quality, cost, latency | | [Send a safety identifier](#send-a-safety-identifier) | Safety, reliability | | [Use `background=True`](#use-backgroundtrue) | Resuming work | | [Use WebSocket mode](#use-websocket-mode) | Latency | ## Use the Responses API **Always start** with the [Responses API](https://developers.openai.com/api/docs/guides/migrate-to-responses). It is OpenAI's flagship API and the best place to access the newest model behavior, built-in tools, stateful workflows, and agent features. ## Choose a GPT-5.6 model Choose a [GPT-5.6 model](https://developers.openai.com/api/docs/guides/latest-model?model=gpt-5.6) for the workload instead of routing every request to the most capable tier. Use `gpt-5.6` or `gpt-5.6-sol` for flagship capability, `gpt-5.6-terra` for strong performance at a lower price, and `gpt-5.6-luna` for efficient, high-volume workloads. When migrating, preserve the current model's workload role and effective reasoning effort for the first comparison. Run representative evals before changing prompts or adding new capabilities. Compare task success, latency, input, output, reasoning, and cache-write tokens, and cost per successful task. ## Set up `reasoning.effort` Use `reasoning.effort` to decide how much thinking the model should do before it answers. For GPT-5.6 models, the supported values are `none`, `low`, `medium`, `high`, `xhigh`, and `max`. The default is `medium`. Lower effort is faster and uses fewer reasoning tokens. Higher effort gives the model more time for planning, debugging, synthesis, and multi-step tradeoffs. Use `low` when the job is mostly extraction, routing, classification, or a routine rewrite. Use `medium` or `high` when the model needs to diagnose a problem, compare options, write a plan, or reason through code. Use `xhigh` or `max` only when representative evals show that the quality gain justifies the extra latency and cost. When migrating from GPT-5.5 or GPT-5.4, start with the current effort and compare the same setting with one level lower. GPT-5.6 can often maintain or improve quality with fewer reasoning tokens, so the lower setting may also reduce latency and cost. For the hardest quality-first workloads, also compare [`reasoning.mode: "pro"`](https://developers.openai.com/api/docs/guides/reasoning#reasoning-mode) with standard mode at the same effort. Reasoning mode and effort are independent. Pro mode can improve reliability by applying more model work before returning a single final answer, but it increases latency and token usage. Tune reasoning effort for the task ```javascript import OpenAI from "openai"; const openai = new OpenAI(); const prompt = [ "Our CI job started failing after a dependency bump.", "", "Error:", "TypeError: Timeout.__init__() got an unexpected keyword argument 'connect'", "", "Identify the likeliest root cause and the smallest safe fix.", ].join("\n"); const response = await openai.responses.create({ model: "gpt-6-astra", reasoning: { effort: "xhigh", mode: "pro" }, input: prompt, }); console.log(response.output_text); ``` ```python from openai import OpenAI client = OpenAI() prompt = """ Our CI job started failing after a dependency bump. Error: TypeError: Timeout.__init__() got an unexpected keyword argument 'connect' Identify the likeliest root cause and the smallest safe fix. """ response = client.responses.create( model="gpt-6-astra", reasoning={"effort": "xhigh", "mode": "pro"}, input=prompt, ) print(response.output_text) ``` ```go package main import ( "context" "fmt" "strings" "github.com/openai/openai-go/v3" "github.com/openai/openai-go/v3/responses" "github.com/openai/openai-go/v3/shared" ) func main() { client := openai.NewClient() prompt := strings.Join([]string{ "Our CI job started failing after a dependency bump.", "", "Error:", "TypeError: Timeout.__init__() got an unexpected keyword argument 'connect'", "", "Identify the likeliest root cause and the smallest safe fix.", }, "\n") reasoning := shared.ReasoningParam{Effort: shared.ReasoningEffortXhigh} reasoning.SetExtraFields(map[string]any{"mode": "pro"}) response, err := client.Responses.New(context.Background(), responses.ResponseNewParams{ Model: "gpt-6-astra", Reasoning: reasoning, Input: responses.ResponseNewParamsInputUnion{OfString: openai.String(prompt)}, }) if err != nil { panic(err) } fmt.Println(response.OutputText()) } ``` ```java import com.openai.client.OpenAIClient; import com.openai.client.okhttp.OpenAIOkHttpClient; import com.openai.core.JsonValue; import com.openai.models.Reasoning; import com.openai.models.ReasoningEffort; import com.openai.models.responses.ResponseCreateParams; ResponseCreateParams params = ResponseCreateParams.builder() .model("gpt-6-astra") .input( "Our CI job started failing after a dependency bump. Error: TypeError: Timeout.__init__() got an unexpected keyword argument 'connect'. Identify the likeliest root cause and the smallest safe fix.") .reasoning( Reasoning.builder() .effort(ReasoningEffort.XHIGH) .putAdditionalProperty("mode", JsonValue.from("pro")) .build()) .build(); client.responses().create(params).output().stream() .flatMap(item -> item.message().stream()) .flatMap(message -> message.content().stream()) .flatMap(content -> content.outputText().stream()) .forEach(text -> System.out.println(text.text())); ``` ```ruby require "openai" client = OpenAI::Client.new prompt = <<~PROMPT Our CI job started failing after a dependency bump. Error: TypeError: Timeout.__init__() got an unexpected keyword argument 'connect' Identify the likeliest root cause and the smallest safe fix. PROMPT response = client.responses.create( model: "gpt-6-astra", reasoning: { effort: :xhigh, mode: :pro }, input: prompt ) puts(response.output_text) ``` ## Set up `text.verbosity` `text.verbosity` is the main lever for balancing brevity against completeness. Use lower verbosity when the product needs a quick, compact answer, and higher verbosity when the response needs richer explanation, clearer structure, or complete context. Lower verbosity means fewer output tokens, so the model generates less and returns output faster. For coding, `medium` and `high` tend to produce longer, more organized output with clearer structure. `low` keeps the answer tighter and more minimal. GPT-5.6 tends to be more concise by default than GPT-5.5. When migrating, check whether broad instructions like "Be concise" still help. In some cases, they may make responses too brief. Keep them only when they still help, and prefer using `text.verbosity` to control the default level of detail; then use the prompt to specify required content, structure, and a more specific length, if applicable. Set lower verbosity for compact output ```javascript import OpenAI from "openai"; const openai = new OpenAI(); const incident = [ "Summarize this incident for the next on-call engineer.", "- checkout latency spiked from 220 ms to 4.8 s", "- only us-east-1 was affected", "- rollback is complete", "- likely trigger: cache stampede after deploy", ].join("\n"); const response = await openai.responses.create({ model: "gpt-6-astra", text: { verbosity: "low" }, input: incident, }); console.log(response.output_text); ``` ```python from openai import OpenAI client = OpenAI() response = client.responses.create( model="gpt-6-astra", text={"verbosity": "low"}, input=""" Summarize this incident for the next on-call engineer. - checkout latency spiked from 220 ms to 4.8 s - only us-east-1 was affected - rollback is complete - likely trigger: cache stampede after deploy """, ) print(response.output_text) ``` ```go package main import ( "context" "fmt" "strings" "github.com/openai/openai-go/v3" "github.com/openai/openai-go/v3/responses" ) func main() { client := openai.NewClient() incident := strings.Join([]string{ "Summarize this incident for the next on-call engineer.", "- checkout latency spiked from 220 ms to 4.8 s", "- only us-east-1 was affected", "- rollback is complete", "- likely trigger: cache stampede after deploy", }, "\n") response, err := client.Responses.New(context.Background(), responses.ResponseNewParams{ Model: "gpt-6-astra", Text: responses.ResponseTextConfigParam{Verbosity: "low"}, Input: responses.ResponseNewParamsInputUnion{OfString: openai.String(incident)}, }) if err != nil { panic(err) } fmt.Println(response.OutputText()) } ``` ```java import com.openai.client.OpenAIClient; import com.openai.client.okhttp.OpenAIOkHttpClient; import com.openai.models.responses.ResponseCreateParams; import com.openai.models.responses.ResponseTextConfig; ResponseCreateParams params = ResponseCreateParams.builder() .model("gpt-6-astra") .input( "Summarize this incident for the next on-call engineer: checkout latency spiked from 220 ms to 4.8 s, only us-east-1 was affected, rollback is complete, and the likely trigger was a cache stampede.") .text(ResponseTextConfig.builder().verbosity(ResponseTextConfig.Verbosity.LOW).build()) .build(); client.responses().create(params).output().stream() .flatMap(item -> item.message().stream()) .flatMap(message -> message.content().stream()) .flatMap(content -> content.outputText().stream()) .forEach(text -> System.out.println(text.text())); ``` ```ruby require "openai" client = OpenAI::Client.new incident = <<~INCIDENT Summarize this incident for the next on-call engineer. - checkout latency spiked from 220 ms to 4.8 s - only us-east-1 was affected - rollback is complete - likely trigger: cache stampede after deploy INCIDENT response = client.responses.create( model: "gpt-6-astra", text: { verbosity: :low }, input: incident ) puts(response.output_text) ``` ## Set up the assistant `phase` parameter `phase` is a label on assistant messages in the conversation history. It indicates to the model whether a prior assistant message was an intermediate working commentary or the final answer. Use `phase: "commentary"` for progress updates, pre-tool-call notes, and other in-between messages. Use `phase: "final_answer"` for the completed response. The assistant might say something like: Assistant commentary message ```json { "role": "assistant", "phase": "commentary", "content": "I'm checking the logs and comparing them to the last successful deploy." } ``` That is not the answer. It is a progress note. Later, the assistant might say: Assistant final answer message ```json { "role": "assistant", "phase": "final_answer", "content": "The deploy failed because the migration referenced a column that does not exist in production." } ``` This is useful in long-running or tool-heavy workflows where the assistant may produce visible progress updates before it finishes. When you send that history back on follow-up requests for `gpt-5.3-codex` and later models, **preserve and resend `phase`** on assistant messages so the model can distinguish progress updates from the final result. This helps reduce early stopping, making the agent more likely to continue until it reaches the final answer. ## Use `tool_search` Instead of loading the full tool catalog into every request, use [tool search](https://developers.openai.com/api/docs/guides/tools-tool-search): add `{"type": "tool_search"}` and mark expensive tool definitions with `defer_loading: true`. The model can then load the subset it needs at runtime. At request start, the model only sees the search tool name and description. If the model decides it needs a deferred tool, it runs tool search, and only then are the deferred tool definitions loaded into context. Only then will the model call them. This saves tokens and preserves cache performance. Tool search has two modes: - **Hosted tool search** is the simpler option. Use it when you already know which tools could be available for the request. - **Client-executed tool search** is for cases where your app has to decide what tools are available, like based on the user's tenant, project, permissions, or internal registry. **Start with hosted tool search** unless your app really needs to control discovery itself. Group your tools by user intent. Use namespaces or MCP servers when you can. It is easier for the model to choose between a few clear groups than a long flat list of functions. We recommend keeping each namespace under about 10 functions for optimal token efficiency and model performance. Keep namespace descriptions short and discriminative. Put the detailed instructions inside the deferred tool definitions. Avoid making one giant namespace for everything. Use hosted tool search with deferred tools ```javascript import OpenAI from "openai"; const openai = new OpenAI(); const billingNamespace = { type: "namespace", name: "billing", description: "Billing tools for invoices, payments, taxes, and credits.", tools: [ { type: "function", name: "lookup_invoice", description: "Look up invoice state, taxes, credits, and payment attempts.", parameters: { type: "object", properties: { invoice_id: { type: "string" }, }, required: ["invoice_id"], additionalProperties: false, }, strict: true, defer_loading: true, }, ], }; const crmNamespace = { type: "namespace", name: "crm", description: "CRM tools for account ownership, plans, health, and payment history.", tools: [ { type: "function", name: "get_account", description: "Fetch account owner, plan, health, and payment history.", parameters: { type: "object", properties: { account_id: { type: "string" }, }, required: ["account_id"], additionalProperties: false, }, strict: true, defer_loading: true, }, ], }; const response = await openai.responses.create({ model: "gpt-6-astra", input: "Find the right billing tool and explain why invoice INV-1043 still " + "shows overdue after a payment yesterday.", tools: [billingNamespace, crmNamespace, { type: "tool_search" }], }); console.log(response.output); ``` ```python from openai import OpenAI client = OpenAI() billing_namespace = { "type": "namespace", "name": "billing", "description": "Billing tools for invoices, payments, taxes, and credits.", "tools": [ { "type": "function", "name": "lookup_invoice", "description": "Look up invoice state, taxes, credits, and payment attempts.", "parameters": { "type": "object", "properties": { "invoice_id": {"type": "string"}, }, "required": ["invoice_id"], "additionalProperties": False, }, "strict": True, "defer_loading": True, } ], } crm_namespace = { "type": "namespace", "name": "crm", "description": "CRM tools for account ownership, plans, health, and payment history.", "tools": [ { "type": "function", "name": "get_account", "description": "Fetch account owner, plan, health, and payment history.", "parameters": { "type": "object", "properties": { "account_id": {"type": "string"}, }, "required": ["account_id"], "additionalProperties": False, }, "strict": True, "defer_loading": True, } ], } response = client.responses.create( model="gpt-6-astra", input=( "Find the right billing tool and explain why invoice INV-1043 still " "shows overdue after a payment yesterday." ), tools=[billing_namespace, crm_namespace, {"type": "tool_search"}], ) print(response.output) ``` ```go package main import ( "context" "fmt" "github.com/openai/openai-go/v3" "github.com/openai/openai-go/v3/responses" ) func main() { client := openai.NewClient() billing := namespaceTool( "billing", "Billing tools for invoices, payments, taxes, and credits.", "lookup_invoice", "Look up invoice state, taxes, credits, and payment attempts.", "invoice_id", ) crm := namespaceTool( "crm", "CRM tools for account ownership, plans, health, and payment history.", "get_account", "Fetch account owner, plan, health, and payment history.", "account_id", ) toolSearch := responses.ToolUnionParam{OfToolSearch: &responses.ToolSearchToolParam{}} response, err := client.Responses.New(context.Background(), responses.ResponseNewParams{ Model: "gpt-6-astra", Input: responses.ResponseNewParamsInputUnion{OfString: openai.String( "Find the right billing tool and explain why invoice INV-1043 still shows overdue after a payment yesterday.", )}, Tools: []responses.ToolUnionParam{billing, crm, toolSearch}, }) if err != nil { panic(err) } fmt.Println(response.Output) } func namespaceTool(namespace, namespaceDescription, name, description, argument string) responses.ToolUnionParam { parameters := map[string]any{ "type": "object", "properties": map[string]any{ argument: map[string]any{"type": "string"}, }, "required": []string{argument}, "additionalProperties": false, } function := responses.NamespaceToolToolFunctionParam{ Name: name, Description: openai.String(description), Parameters: parameters, Strict: openai.Bool(true), DeferLoading: openai.Bool(true), } return responses.ToolParamOfNamespace( namespaceDescription, namespace, []responses.NamespaceToolToolUnionParam{{OfFunction: &function}}, ) } ``` ```java import com.openai.client.OpenAIClient; import com.openai.client.okhttp.OpenAIOkHttpClient; import com.openai.core.JsonValue; import com.openai.models.responses.NamespaceTool; import com.openai.models.responses.ResponseCreateParams; import com.openai.models.responses.ToolSearchTool; import java.util.List; import java.util.Map; ResponseCreateParams params = ResponseCreateParams.builder() .model("gpt-6-astra") .input( "Find the right billing tool and explain why invoice INV-1043 still shows overdue after a payment yesterday.") .addTool( namespace( "billing", "Billing tools for invoices, payments, taxes, and credits.", "lookup_invoice", "Look up invoice state, taxes, credits, and payment attempts.", "invoice_id")) .addTool( namespace( "crm", "CRM tools for account ownership, plans, health, and payment history.", "get_account", "Fetch account owner, plan, health, and payment history.", "account_id")) .addTool(ToolSearchTool.builder().execution(ToolSearchTool.Execution.SERVER).build()) .build(); client.responses().create(params).output().forEach(System.out::println); private static NamespaceTool namespace( String name, String description, String function, String functionDescription, String argument) { return NamespaceTool.builder() .name(name) .description(description) .addTool( NamespaceTool.Tool.Function.builder() .name(function) .description(functionDescription) .deferLoading(true) .strict(true) .parameters( JsonValue.from( Map.of( "type", "object", "properties", Map.of(argument, Map.of("type", "string")), "required", List.of(argument), "additionalProperties", false))) .build()) .build(); } ``` ```ruby require "openai" def namespace_tool(name, description, function_name, function_description, argument) { type: :namespace, name: name, description: description, tools: [ { type: :function, name: function_name, description: function_description, defer_loading: true, strict: true, parameters: { type: "object", properties: { argument => { type: "string" } }, required: [argument], additionalProperties: false } } ] } end client = OpenAI::Client.new billing = namespace_tool( "billing", "Billing tools for invoices, payments, taxes, and credits.", "lookup_invoice", "Look up invoice state, taxes, credits, and payment attempts.", "invoice_id" ) crm = namespace_tool( "crm", "CRM tools for account ownership, plans, health, and payment history.", "get_account", "Fetch account owner, plan, health, and payment history.", "account_id" ) response = client.responses.create( model: "gpt-6-astra", input: "Find the right billing tool and explain why invoice INV-1043 still shows overdue after a payment yesterday.", tools: [billing, crm, { type: :tool_search }] ) puts(response.output) ``` ## Use Programmatic Tool Calling [Programmatic Tool Calling](https://developers.openai.com/api/docs/guides/tools-programmatic-tool-calling) lets GPT-5.6 write JavaScript that calls eligible tools and reduces their intermediate results inside a hosted runtime. Use it for bounded stages where code can filter, join, rank, remove duplicates, combine, or check large tool results before returning a smaller structured result to the model. Add the `programmatic_tool_calling` tool and opt in each eligible tool. Use `allowed_callers: ["programmatic"]` for program-only tools, or use `allowed_callers: ["direct", "programmatic"]` when the model may also call the tool directly. Keep calls direct when each result may change the model's next decision, an action requires approval, or the final answer must preserve citations or native artifacts. Document tool return fields and error behavior so the model can write a correct program without first inspecting a result. Your tool loop must handle `program` and `program_output` items, as well as program-issued `function_call` items and their `function_call_output` items. Preserve each `call_id`, and copy the function call's `caller` into its output so the service can resume the correct program. Test both the `program_output` and the final assistant message. A correct program result can still become an incomplete final answer. Compare task success, required evidence, total tokens, latency, and cost against the same workflow using direct tool calls. ## Use Multi-agent for parallel work [Multi-agent](https://developers.openai.com/api/docs/guides/responses-multi-agent) is a GPT-5.6 feature that lets a root agent delegate independent workstreams to subagents and synthesize their results. Use it when you can split research, analysis, or implementation into concrete, bounded tasks that use separate context and run in parallel. Set `multi_agent.enabled` to `true` in the request. For HTTP, use the beta Responses SDK with `client.beta.responses` and pass `responses_multi_agent=v1` in `betas`. For raw HTTP or WebSocket connections, send `OpenAI-Beta: responses_multi_agent=v1`. Item schemas can change while Multi-agent is in beta. Prefer one agent for short tasks, ordered chains where each step depends on the last, or work that writes to the same mutable resource. Subagents can increase token usage, so start with the default `max_concurrent_subagents` value of `3` and measure end-to-end quality, latency, and cost. For tool-heavy or long-running Multi-agent workflows, WebSocket mode can reduce continuation overhead. Before enabling Multi-agent, account for its current limitations: `/responses/compact`, `reasoning.summary`, and `max_tool_calls` are not supported. The server automatically compacts the root context and every subagent context. ## Leverage built-in tools [Built-in tools](https://developers.openai.com/api/docs/guides/tools) are native capabilities of the API. Instead of building every tool yourself, you can give the model access to tools that already work inside the Responses API. The model can then decide when to use them. OpenAI keeps adding more native tools, so start with built-in tools when they fit your workflow. Build custom tools when native options do not cover the task. Current built-in tools and related tool options include: - **Web search**: Search the web for up-to-date information - **File search**: Search uploaded files or vector stores - **Code interpreter**: Run Python for analysis, math, charts, and file processing - **Shell**: Run shell commands in a hosted container or your own runtime - **Computer use**: Operate a UI through screenshots, clicks, typing, and scrolling - **Image generation**: Generate or edit images - **MCP/connectors**: Connect the model to external services and tools - **Skills**: Attach reusable instruction bundles and workflow files - **Apply patch**: Make structured code edits Model quality is another reason to prefer them. Built-in tools are in-distribution for our post-training, meaning that the models are trained and evaluated around these tool shapes, behaviors, and outputs. With built-in tools, OpenAI models support better tool selection, cleaner execution, and fewer failures than with new tools. ## Leverage compaction [Compaction](https://developers.openai.com/api/docs/guides/compaction) is a context engineering tool: it decides what information the model carries forward across many turns. In long-running agents, the problem is not just, "Will I hit the context limit?" It is that old messages, tool logs, retries, and stale details crowd out the state the model needs. Compaction gives you a controlled way to reduce context size while preserving state needed for subsequent turns. After a meaningful milestone, like finishing a debugging phase or narrowing a root cause, you can compact the prior window and continue from the compacted output. This keeps the model sharp because the next turn is built around the important state, not every intermediate reasoning, failed command, and obsolete branch of reasoning. You can use compaction in two ways: - **Let the server handle it**: if you use `previous_response_id`, turn on `context_management` with a `compact_threshold`. The server will automatically compact the conversation when it gets too large. You keep sending only the newest user message. - **Do it yourself**: if you manage the full input array yourself, call `client.responses.compact()`. It gives back a smaller context window. Use that returned output directly in the next `responses.create()` call. **Do not edit the compacted output.** It is not a human summary, but the machine state that helps the model continue. Pass it forward as-is, then add the next user message. Continue from compacted response state ```javascript import OpenAI from "openai"; import { toResponseInputItems } from "openai/lib/responses/ResponseInputItems"; const openai = new OpenAI(); // Full window collected from a long debugging session: // user messages, assistant outputs, tool calls, and tool outputs. const longWindow = sessionItems; const compacted = await openai.responses.compact({ model: "gpt-6-astra", input: longWindow, }); const nextResponse = await openai.responses.create({ model: "gpt-6-astra", store: false, input: [ // Preserve replayable compacted items. ...toResponseInputItems(compacted.output), { type: "message", role: "user", content: "We found the bad cache invalidation path. Write the fix plan " + "and the verification checklist.", }, ], }); console.log(nextResponse.output_text); ``` ```python from openai import OpenAI client = OpenAI() # Full window collected from a long debugging session: # user messages, assistant outputs, tool calls, and tool outputs. long_window = session_items compacted = client.responses.compact( model="gpt-6-astra", input=long_window, ) next_response = client.responses.create( model="gpt-6-astra", store=False, input=[ *compacted.output, # Use compact output as-is. { "type": "message", "role": "user", "content": ( "We found the bad cache invalidation path. Write the fix plan " "and the verification checklist." ), }, ], ) print(next_response.output_text) ``` ```go package main import ( "context" "encoding/json" "fmt" "github.com/openai/openai-go/v3" "github.com/openai/openai-go/v3/responses" ) func main() { client := openai.NewClient() longWindow := []responses.ResponseInputItemUnionParam{ responses.ResponseInputItemParamOfMessage("Find the cache invalidation bug in this debugging session.", responses.EasyInputMessageRoleUser), } compacted, err := client.Responses.Compact(context.Background(), responses.ResponseCompactParams{ Model: "gpt-6-astra", Input: responses.ResponseCompactParamsInputUnion{OfResponseInputItemArray: longWindow}, }) if err != nil { panic(err) } input := append(outputAsInput(compacted.Output), responses.ResponseInputItemParamOfMessage( "We found the bad cache invalidation path. Write the fix plan and the verification checklist.", responses.EasyInputMessageRoleUser, ), ) nextResponse, err := client.Responses.New(context.Background(), responses.ResponseNewParams{ Model: "gpt-6-astra", Store: openai.Bool(false), Input: responses.ResponseNewParamsInputUnion{OfInputItemList: input}, }) if err != nil { panic(err) } fmt.Println(nextResponse.OutputText()) } func outputAsInput(output []responses.ResponseOutputItemUnion) []responses.ResponseInputItemUnionParam { input := make([]responses.ResponseInputItemUnionParam, 0, len(output)) for _, item := range output { var converted responses.ResponseInputItemUnion if err := json.Unmarshal([]byte(item.RawJSON()), &converted); err != nil { panic(err) } input = append(input, converted.ToParam()) } return input } ``` ```java import com.openai.client.OpenAIClient; import com.openai.client.okhttp.OpenAIOkHttpClient; import com.openai.models.responses.EasyInputMessage; import com.openai.models.responses.ResponseCompactParams; import com.openai.models.responses.ResponseCompactionItemParam; import com.openai.models.responses.ResponseCreateParams; import com.openai.models.responses.ResponseInputItem; import java.util.ArrayList; var compacted = client .responses() .compact( ResponseCompactParams.builder() .model("gpt-6-astra") .input("Find the cache invalidation bug in this debugging session.") .build()); var input = new ArrayList(); for (var item : compacted.output()) { item.message().map(ResponseInputItem::ofResponseOutputMessage).ifPresent(input::add); item.reasoning().map(ResponseInputItem::ofReasoning).ifPresent(input::add); item.compaction() .map( value -> ResponseInputItem.ofCompaction( ResponseCompactionItemParam.builder() .id(value.id()) .encryptedContent(value.encryptedContent()) .build())) .ifPresent(input::add); } input.add( ResponseInputItem.ofEasyInputMessage( EasyInputMessage.builder() .role(EasyInputMessage.Role.USER) .content( "We found the bad cache invalidation path. Write the fix plan and the verification checklist.") .build())); client .responses() .create( ResponseCreateParams.builder() .model("gpt-6-astra") .inputOfResponse(input) .store(false) .build()) .output() .stream() .flatMap(item -> item.message().stream()) .flatMap(message -> message.content().stream()) .flatMap(content -> content.outputText().stream()) .forEach(text -> System.out.println(text.text())); ``` ```ruby require "openai" client = OpenAI::Client.new long_window = [ { role: :user, content: "Find the cache invalidation bug in this debugging session." } ] compacted = client.responses.compact( model: "gpt-6-astra", input: long_window ) input = compacted.output.dup input << { role: :user, content: "We found the bad cache invalidation path. Write the fix plan and the verification checklist." } response = client.responses.create( model: "gpt-6-astra", store: false, input: input ) puts(response.output_text) ``` ## Optimize prompt caching [Prompt caching](https://developers.openai.com/api/docs/guides/prompt-caching) automatically reduces latency and cost when requests reuse the same long prefix. Put stable instructions, examples, and reference material first, followed by dynamic user-specific content. Keep tool definitions and ordering stable, and append new conversation turns without rewriting earlier context. GPT-5.6 introduced explicit prompt caching. Implicit caching remains the default, but GPT-5.6 models and later model families also support explicit cache breakpoints and request-wide cache policy. If a changing suffix comes after a stable prefix, add an explicit `prompt_cache_breakpoint` at the reusable boundary. Set `prompt_cache_options.mode` to `explicit` only when the request should use only the breakpoints you provide and no implicit breakpoint. Earlier models continue to use automatic prompt caching only. On GPT-5.6 models and later model families, cache writes cost 1.25× the uncached input token rate. Log `cached_tokens` and `cache_write_tokens`, then compare write volume with later cache reads to measure net cost and tune breakpoint placement. Use a stable `prompt_cache_key` for requests that share a reusable prefix to help route related requests to the same cache and optimize cache hit rates on models before GPT-5.6. For busy groups, follow the [guidance for distributing traffic across more keys](https://developers.openai.com/api/docs/guides/prompt-caching#prompt-cache-keys). On GPT-5.6 and later, `prompt_cache_key` is optional: you can achieve optimal cache hit rates without it. You can use it to maintain separate cache accounting for customers, users, or workspaces. This can make cached token usage and billing easier to explain for each group. Assign a distinct key to each customer and keep it stable across that customer's related requests. Separate keys also help prevent cache-hit probing across customers. See [Separate cache accounting with keys](https://developers.openai.com/api/docs/guides/prompt-caching#separate-prompts-with-cache-keys). Maintain separate cache accounting for a customer ```javascript import OpenAI from "openai"; const openai = new OpenAI(); const instructions = [ "You are the support agent for Acme.", "Follow the Acme support policy and escalation rubric.", "Use the same tone, safety rules, and tool plan for each ticket.", ].join("\n"); const response = await openai.responses.create({ model: "gpt-6-astra", prompt_cache_key: "tenant-acme-support-agent", instructions, input: "Summarize the current escalation for the on-call lead.", }); console.log(response.output_text); ``` ```python from openai import OpenAI client = OpenAI() instructions = """ You are the support agent for Acme. Follow the Acme support policy and escalation rubric. Use the same tone, safety rules, and tool plan for each ticket. """ response = client.responses.create( model="gpt-6-astra", prompt_cache_key="tenant-acme-support-agent", instructions=instructions, input="Summarize the current escalation for the on-call lead.", ) print(response.output_text) ``` ```go package main import ( "context" "fmt" "strings" "github.com/openai/openai-go/v3" "github.com/openai/openai-go/v3/responses" ) func main() { client := openai.NewClient() instructions := strings.Join([]string{ "You are the support agent for Acme.", "Follow the Acme support policy and escalation rubric.", "Use the same tone, safety rules, and tool plan for each ticket.", }, "\n") response, err := client.Responses.New(context.Background(), responses.ResponseNewParams{ Model: "gpt-6-astra", PromptCacheKey: openai.String("tenant-acme-support-agent"), Instructions: openai.String(instructions), Input: responses.ResponseNewParamsInputUnion{OfString: openai.String("Summarize the current escalation for the on-call lead.")}, }) if err != nil { panic(err) } fmt.Println(response.OutputText()) } ``` ```java import com.openai.client.OpenAIClient; import com.openai.client.okhttp.OpenAIOkHttpClient; import com.openai.models.responses.ResponseCreateParams; ResponseCreateParams params = ResponseCreateParams.builder() .model("gpt-6-astra") .instructions( "You are the support agent for Acme.\n" + "Follow the Acme support policy and escalation rubric.\n" + "Use the same tone, safety rules, and tool plan for each ticket.") .input("Summarize the current escalation for the on-call lead.") .promptCacheKey("tenant-acme-support-agent") .build(); client.responses().create(params).output().stream() .flatMap(item -> item.message().stream()) .flatMap(message -> message.content().stream()) .flatMap(content -> content.outputText().stream()) .forEach(text -> System.out.println(text.text())); ``` ```csharp using OpenAI.Responses; #pragma warning disable OPENAI001 string key = Environment.GetEnvironmentVariable("OPENAI_API_KEY")!; ResponsesClient client = new(key); CreateResponseOptions options = new() { Model = "gpt-6-astra", PromptCacheKey = "tenant-acme-support-agent", Instructions = "Follow the Acme support policy and escalation rubric.", }; options.InputItems.Add( ResponseItem.CreateUserMessageItem("Summarize the current escalation for the on-call lead.") ); ResponseResult response = await client.CreateResponseAsync(options); Console.WriteLine(response.GetOutputText()); ``` ```ruby require "openai" client = OpenAI::Client.new instructions = <<~INSTRUCTIONS You are the support agent for Acme. Follow the Acme support policy and escalation rubric. Use the same tone, safety rules, and tool plan for each ticket. INSTRUCTIONS response = client.responses.create( model: "gpt-6-astra", prompt_cache_key: "tenant-acme-support-agent", instructions: instructions, input: "Summarize the current escalation for the on-call lead." ) puts(response.output_text) ``` ## Use `reasoning.encrypted_content` GPT-5.6 can [preserve reasoning across calls](https://developers.openai.com/api/docs/guides/reasoning#preserve-reasoning-across-calls). Use `reasoning.context: "all_turns"` when the task's goals, assumptions, and priorities remain stable. Use `current_turn` when earlier reasoning is no longer relevant and might anchor the model to an outdated approach. If you omit `reasoning.context` or set it to `auto`, inspect the response's `reasoning.context` field to confirm the effective mode. [Persisted reasoning](https://developers.openai.com/api/docs/guides/reasoning#keeping-reasoning-items-in-context) works only when earlier reasoning items are available. Use `previous_response_id` for stored responses. If your [Zero Data Retention (ZDR)](https://developers.openai.com/api/docs/guides/your-data#zero-data-retention) requirements do not allow storing response data, encrypted reasoning content enables a stateless handoff. Reasoning items in the response output include encrypted reasoning content by default. You can access the encrypted reasoning content from each reasoning item's `encrypted_content` property. Your app does not need to understand that value. It just keeps each reasoning item exactly as returned and sends it back during the next turn, so the model can use it to continue the workflow. Pass encrypted reasoning between stateless turns ```javascript import OpenAI from "openai"; import { toResponseInputItems } from "openai/lib/responses/ResponseInputItems"; const openai = new OpenAI(); const history = [ { role: "user", content: "Investigate why invoice INV-1043 has mismatched tax totals.", }, ]; const first = await openai.responses.create({ model: "gpt-6-astra", store: false, reasoning: { effort: "medium", context: "current_turn" }, input: history, }); history.push(...toResponseInputItems(first.output)); history.push({ role: "user", content: "Now write the customer-facing explanation in plain English.", }); const second = await openai.responses.create({ model: "gpt-6-astra", store: false, reasoning: { effort: "medium", context: "all_turns" }, input: history, }); console.log(second.output_text); ``` ```python from openai import OpenAI client = OpenAI() history = [ { "role": "user", "content": "Investigate why invoice INV-1043 has mismatched tax totals.", } ] first = client.responses.create( model="gpt-6-astra", store=False, reasoning={"effort": "medium", "context": "current_turn"}, input=history, ) history.extend(item.model_dump(exclude={"status"}) for item in first.output) history.append( { "role": "user", "content": "Now write the customer-facing explanation in plain English.", } ) second = client.responses.create( model="gpt-6-astra", store=False, reasoning={"effort": "medium", "context": "all_turns"}, input=history, ) print(second.output_text) ``` ```go package main import ( "context" "encoding/json" "fmt" "github.com/openai/openai-go/v3" "github.com/openai/openai-go/v3/responses" "github.com/openai/openai-go/v3/shared" ) func main() { client := openai.NewClient() history := []responses.ResponseInputItemUnionParam{ responses.ResponseInputItemParamOfMessage("Investigate why invoice INV-1043 has mismatched tax totals.", responses.EasyInputMessageRoleUser), } first, err := client.Responses.New(context.Background(), responses.ResponseNewParams{ Model: "gpt-6-astra", Store: openai.Bool(false), Reasoning: shared.ReasoningParam{Effort: shared.ReasoningEffortMedium, Context: shared.ReasoningContextCurrentTurn}, Include: []responses.ResponseIncludable{responses.ResponseIncludableReasoningEncryptedContent}, Input: responses.ResponseNewParamsInputUnion{OfInputItemList: history}, }) if err != nil { panic(err) } history = append(history, outputAsInput(first.Output)...) history = append(history, responses.ResponseInputItemParamOfMessage( "Now write the customer-facing explanation in plain English.", responses.EasyInputMessageRoleUser, )) second, err := client.Responses.New(context.Background(), responses.ResponseNewParams{ Model: "gpt-6-astra", Store: openai.Bool(false), Reasoning: shared.ReasoningParam{Effort: shared.ReasoningEffortMedium, Context: shared.ReasoningContextAllTurns}, Input: responses.ResponseNewParamsInputUnion{OfInputItemList: history}, }) if err != nil { panic(err) } fmt.Println(second.OutputText()) } func outputAsInput(output []responses.ResponseOutputItemUnion) []responses.ResponseInputItemUnionParam { input := make([]responses.ResponseInputItemUnionParam, 0, len(output)) for _, item := range output { var converted responses.ResponseInputItemUnion if err := json.Unmarshal([]byte(item.RawJSON()), &converted); err != nil { panic(err) } input = append(input, converted.ToParam()) } return input } ``` ```java import com.openai.client.OpenAIClient; import com.openai.client.okhttp.OpenAIOkHttpClient; import com.openai.core.JsonValue; import com.openai.models.Reasoning; import com.openai.models.responses.EasyInputMessage; import com.openai.models.responses.ResponseCreateParams; import com.openai.models.responses.ResponseIncludable; import com.openai.models.responses.ResponseInputItem; import java.util.ArrayList; var history = new ArrayList(); history.add( ResponseInputItem.ofEasyInputMessage( EasyInputMessage.builder() .role(EasyInputMessage.Role.USER) .content("Investigate why invoice INV-1043 has mismatched tax totals.") .build())); var first = client .responses() .create( ResponseCreateParams.builder() .model("gpt-6-astra") .inputOfResponse(history) .store(false) .reasoning( Reasoning.builder() .effort(com.openai.models.ReasoningEffort.MEDIUM) .putAdditionalProperty("context", JsonValue.from("current_turn")) .build()) .addInclude(ResponseIncludable.of("reasoning.encrypted_content")) .build()); first.output().stream() .map(item -> JsonValue.from(item).convert(ResponseInputItem.class)) .forEach(history::add); history.add( ResponseInputItem.ofEasyInputMessage( EasyInputMessage.builder() .role(EasyInputMessage.Role.USER) .content("Now write the customer-facing explanation in plain English.") .build())); client .responses() .create( ResponseCreateParams.builder() .model("gpt-6-astra") .inputOfResponse(history) .store(false) .reasoning( Reasoning.builder() .effort(com.openai.models.ReasoningEffort.MEDIUM) .putAdditionalProperty("context", JsonValue.from("all_turns")) .build()) .build()) .output() .stream() .flatMap(item -> item.message().stream()) .flatMap(message -> message.content().stream()) .flatMap(content -> content.outputText().stream()) .forEach(text -> System.out.println(text.text())); ``` ```ruby require "openai" client = OpenAI::Client.new history = [ { role: :user, content: "Investigate why invoice INV-1043 has mismatched tax totals." } ] first = client.responses.create( model: "gpt-6-astra", store: false, reasoning: { effort: :medium, context: :current_turn }, include: ["reasoning.encrypted_content"], input: history ) history.concat(first.output) history << { role: :user, content: "Now write the customer-facing explanation in plain English." } second = client.responses.create( model: "gpt-6-astra", store: false, reasoning: { effort: :medium, context: :all_turns }, input: history ) puts(second.output_text) ``` ## Set image detail intentionally On GPT-5.6 models, omitted image `detail` and `detail: "auto"` use the same sizing behavior as `original`. The service preserves the input dimensions, except that images larger than 65,535 pixels on either side are scaled down to fit that limit. The API rejects images that still exceed the [30,000-patch limit](https://developers.openai.com/api/docs/guides/images-vision#image-input-requirements), rather than resizing them to fit it. Large images can use more input tokens and add latency as a result. Choose [`detail`](https://developers.openai.com/api/docs/guides/images-vision#choose-an-image-detail-level) for the task. Resize the image, use `low` when fine visual detail is not important, or use `high` for standard high-fidelity image understanding. Keep `original` for large, dense, coordinate-sensitive, OCR, localization, or visual-inspection tasks where the extra detail improves quality. Measure worst-case image tokens and latency before deployment. ## Send a safety identifier If your application serves individual end users, send a stable, privacy-preserving [`safety_identifier`](https://developers.openai.com/api/docs/guides/safety-best-practices#implement-safety-identifiers) with each request. It helps OpenAI detect misuse and gives your team a stable way to trace policy violations. It also reduces the chance that one user's misuse disrupts access for your broader organization. Hash the user's username or email address instead of sending identifying information. For logged-out experiences, use a stable session ID. ## Use `background=True` Use [`background=True`](https://developers.openai.com/api/docs/guides/background) for requests that may take a long time. Instead of keeping the client connection open, the API starts a job and returns an ID. Your app can poll that job until it finishes, fails, or is canceled. Use it for large analyses, long tool runs, or work that needs status and retry behavior. Run and poll a background response ```javascript // Replace the illustrative IDs and URLs below with your own resource values. import OpenAI from "openai"; const openai = new OpenAI(); const logBundleFileId = "file_123"; let job = await openai.responses.create({ model: "gpt-6-astra", background: true, store: false, input: "Analyze this large log bundle and cluster the primary failure modes.", tools: [ { type: "code_interpreter", container: { type: "auto", file_ids: [logBundleFileId], }, }, ], }); while (["queued", "in_progress"].includes(job.status)) { await new Promise((resolve) => setTimeout(resolve, 2000)); job = await openai.responses.retrieve(job.id); } console.log(job.output_text); ``` ```python # Replace the illustrative IDs and URLs below with your own resource values. from openai import OpenAI import time client = OpenAI() log_bundle_file_id = "file_123" job = client.responses.create( model="gpt-6-astra", background=True, store=False, input="Analyze this large log bundle and cluster the primary failure modes.", tools=[ { "type": "code_interpreter", "container": { "type": "auto", "file_ids": [log_bundle_file_id], }, } ], ) while job.status in {"queued", "in_progress"}: time.sleep(2) job = client.responses.retrieve(job.id) print(job.output_text) ``` ```go package main import ( "context" "fmt" "time" "github.com/openai/openai-go/v3" "github.com/openai/openai-go/v3/responses" ) func main() { client := openai.NewClient() tool := responses.ToolParamOfCodeInterpreter(responses.ToolCodeInterpreterContainerCodeInterpreterContainerAutoParam{ FileIDs: []string{"file_abc123"}, }) job, err := client.Responses.New(context.Background(), responses.ResponseNewParams{ Model: "gpt-6-astra", Background: openai.Bool(true), Store: openai.Bool(false), Input: responses.ResponseNewParamsInputUnion{OfString: openai.String("Analyze this large log bundle and cluster the primary failure modes.")}, Tools: []responses.ToolUnionParam{tool}, }) if err != nil { panic(err) } for job.Status == responses.ResponseStatusQueued || job.Status == responses.ResponseStatusInProgress { time.Sleep(2 * time.Second) job, err = client.Responses.Get(context.Background(), job.ID, responses.ResponseGetParams{}) if err != nil { panic(err) } } fmt.Println(job.OutputText()) } ``` ```java import com.openai.client.OpenAIClient; import com.openai.client.okhttp.OpenAIOkHttpClient; import com.openai.models.responses.ResponseCreateParams; import com.openai.models.responses.ResponseStatus; import com.openai.models.responses.Tool; String fileId = "file_abc123"; ResponseCreateParams params = ResponseCreateParams.builder() .model("gpt-6-astra") .input("Analyze this large log bundle and cluster the primary failure modes.") .background(true) .store(false) .addCodeInterpreterTool( Tool.CodeInterpreter.Container.CodeInterpreterToolAuto.builder() .addFileId(fileId) .build()) .build(); var response = client.responses().create(params); while (response.status().filter(ResponseStatus.QUEUED::equals).isPresent() || response.status().filter(ResponseStatus.IN_PROGRESS::equals).isPresent()) { Thread.sleep(1000); response = client.responses().retrieve(response.id()); } if (response.status().filter(ResponseStatus.COMPLETED::equals).isEmpty()) { throw new IllegalStateException( "Research ended with status: " + response.status().orElseThrow()); } response.output().stream() .flatMap(item -> item.message().stream()) .flatMap(message -> message.content().stream()) .flatMap(content -> content.outputText().stream()) .forEach(text -> System.out.println(text.text())); ``` ```ruby require "openai" client = OpenAI::Client.new job = client.responses.create( model: "gpt-6-astra", background: true, store: false, input: "Analyze this large log bundle and cluster the primary failure modes.", tools: [ { type: :code_interpreter, container: { type: :auto, file_ids: ["file_abc123"] } } ] ) while [:queued, :in_progress].include?(job.status) sleep(2) job = client.responses.retrieve(job.id) end puts(job.output_text) ``` You can combine it with `stream=True` for progress events, but the first event may take longer than a normal request. From the UI perspective, background mode indicates, "This is running; here is the status; the result will appear here when it's ready." ## Use WebSocket mode [WebSocket mode](https://developers.openai.com/api/docs/guides/websocket-mode) is built for long-running, tool-call-heavy workflows where you keep a persistent connection open and continue by sending only new input items plus `previous_response_id`. For runs with 20 or more tool calls, this approach is roughly 40% faster end-to-end. **How this works**: The first message will look like a normal Responses request: model, instructions, tools, and user input. The server streams events back. If the model asks for a tool, your app runs the tool. Then, instead of sending a new HTTP request, you send another `response.create` event on the same socket with the prior `previous_response_id` and the new item. That is where the latency win comes from. In plain HTTP, every follow-up is a fresh request. In WebSocket mode, the connection stays open and the most recent response state stays warm in memory on that connection. When the next turn continues from that response, the backend has to do less setup work. If your workflow is one request, one answer, then **keep HTTP**. If your workflow behaves like a long-running agent, try WebSocket mode. A single WebSocket connection handles one in-flight response at a time, so parallel work needs multiple connections. Connections currently top out at 60 minutes. Continuation uses the same `previous_response_id` semantics as HTTP mode, with a connection-local cache for the most recent response. Note: WebSocket mode works with ZDR because your data is not stored to disk, only stored in memory. The Python sample uses `pip install "openai[realtime]>=3.8.0"`. The JavaScript sample uses `npm install openai@^7.10.0 ws`. The Ruby sample uses `gem install openai async-websocket`. Start a Responses API WebSocket session ```javascript import OpenAI from "openai"; import { ResponsesWS } from "openai/resources/responses/ws"; const openai = new OpenAI(); const ws = new ResponsesWS(openai); ws.on("event", (event) => { console.log(event.type); if ( event.type === "response.completed" || event.type === "response.failed" || event.type === "response.incomplete" ) { ws.close(); } }); ws.on("error", (error) => { console.error(error); ws.close(); }); ws.send({ type: "response.create", model: "gpt-6-astra", store: false, input: [ { type: "message", role: "user", content: [ { type: "input_text", text: "Find the flaky test in this run, call the tools you need, " + "and keep going until you can explain the root cause.", }, ], }, ], tools: [testLogTool, codeSearchTool], }); ``` ```python from openai import OpenAI client = OpenAI() with client.responses.connect() as connection: # Use the same typed parameters as client.responses.create(...). connection.response.create( model="gpt-6-astra", store=False, input=[ { "type": "message", "role": "user", "content": [ { "type": "input_text", "text": ( "Find the flaky test in this run, call the tools " "you need, and keep going until you can explain " "the root cause." ), } ], } ], tools=[test_log_tool, code_search_tool], ) first_event = connection.recv() print(first_event.type) ``` ```ruby require "async" require "openai" require "json" def wait_for_response(connection) while (event = connection.receive) case event.type.to_s when "response.completed" then return event.response when "response.failed", "response.incomplete", "error" raise "Response failed: #{event.to_json}" end end raise "Connection closed before the response finished" end test_log_tool = { type: "function", name: "search_test_logs", description: "Search test logs.", parameters: { type: "object", properties: { query: { type: "string" } }, required: ["query"], additionalProperties: false }, strict: true } code_search_tool = { type: "function", name: "search_code", description: "Search source code.", parameters: { type: "object", properties: { query: { type: "string" } }, required: ["query"], additionalProperties: false }, strict: true } client = OpenAI::Client.new Sync do |task| task.with_timeout(120) do client.responses.connect(request_options: { timeout: 10 }) do |connection| connection.response.create( stream_id: "main", model: "gpt-6-astra", store: false, input: [ { role: "user", content: "Find the flaky test in this run, call the tools you need, and keep going until you can explain the root cause." } ], tools: [test_log_tool, code_search_tool] ) puts(JSON.pretty_generate(wait_for_response(connection).output.map(&:to_h))) end end end ``` ## Final takeaway Responses API is the foundation for building smarter, more capable OpenAI applications. The real advantage is that it lets developers move from one-off prompts to durable, tool-using, context-aware workflows that can adapt to the complexity of the task. Follow this guide to see higher performance in real deployments. --- # Apply Patch The `apply_patch` tool lets GPT-5.1 create, update, and delete files in your codebase using structured diffs. Instead of just suggesting edits, the model emits patch operations that your application applies and then reports back on, enabling iterative, multi-step code editing workflows. ## When to use Some common scenarios where you would use apply_patch: - **Multi-file refactors** – Rename symbols, extract helpers, or reorganize modules across many files at once. - **Bug fixes** – Have the model both diagnose issues and emit precise patches. - **Tests & docs generation** – Create new test files, fixtures, and documentation alongside code changes. - **Migrations & mechanical edits** – Apply repetitive, structured updates (API migrations, type annotations, formatting fixes, etc.). If you can describe your repo and desired change in text, apply_patch can usually generate the corresponding diffs. ## Use apply patch tool with Responses API At a high level, using `apply_patch` with the Responses API looks like this: 1. **Call the Responses API with the `apply_patch` tool** - Provide the model with context about available files (or a summary) in your `input`, or give the model tools for exploring your file system. - Enable the tool with `tools=[{"type": "apply_patch"}]`. 2. **Let the model return one or more patch operations** - The Response output includes one or more `apply_patch_call` objects. - Each call describes a single file operation: create, update, or delete. 3. **Apply patches in your environment** - Run a patch harness or script that: - Interprets the `operation` diff for each `apply_patch_call`. - Applies the patch to your working directory or repo. - Records whether each patch succeeded and any logs or error messages. 4. **Report patch results back to the model** - Call the Responses API again, either with `previous_response_id` or by passing back your conversation items into `input`. - Include an `apply_patch_call_output` event for each `call_id`, with a `status` and optional `output` string. - Keep `tools=[{"type": "apply_patch"}]` so the model can continue editing if needed. 5. **Let the model continue or explain changes** - The model may issue more `apply_patch_call` operations, or - Provide a human-facing explanation of what it changed and why. ## Example: Renaming a function with Apply Patch Tool **Step 1: Ask the model to plan and emit patches** Ask the model to plan and emit patches ```javascript const response = await client.responses.create({ model: "gpt-6-astra", input: fileContext, tools: [{ type: "apply_patch" }], }); const patchCalls = response.output.filter( (item) => item.type === "apply_patch_call" ); ``` ```python from openai import OpenAI client = OpenAI() # For brevity, we are including file context in the example input. # Most agentic use cases should instead equip the model with tools # for exploring file system state. RESPONSE_INPUT = """ The user has the following files: ===== lib/fib.py def fib(n): if n <= 1: return n return fib(n-1) + fib(n-2) ===== run.py from lib.fib import fib def main(): print(fib(42)) You are a helpful coding assistant that should assist the user with whatever they ask. User query: Help me rename the fib() function to fibonacci() """ response = client.responses.create( model="gpt-6-astra", input=RESPONSE_INPUT, tools=[{"type": "apply_patch"}], ) # response.output may contain multiple apply_patch_call entries, e.g.: # - update lib/fib.py # - update run.py patch_calls = [ item.model_dump() for item in response.output if item.type == "apply_patch_call" ] ``` ```go response, err := client.Responses.New(context.Background(), responses.ResponseNewParams{ Model: "gpt-6-astra", Input: responses.ResponseNewParamsInputUnion{OfString: openai.String(responseInput)}, Tools: []responses.ToolUnionParam{{OfApplyPatch: &responses.ApplyPatchToolParam{}}}, }) if err != nil { panic(err) } patchCalls := make([]responses.ResponseOutputItemUnion, 0) for _, item := range response.Output { if item.Type == "apply_patch_call" { patchCalls = append(patchCalls, item) } } ``` ```java import com.openai.client.OpenAIClient; import com.openai.client.okhttp.OpenAIOkHttpClient; import com.openai.models.responses.ApplyPatchTool; import com.openai.models.responses.ResponseCreateParams; ResponseCreateParams params = ResponseCreateParams.builder() .model("gpt-6-astra") .input( "Rename fib() to fibonacci() in lib/fib.py and update run.py to use the new name.") .addTool(ApplyPatchTool.builder().build()) .build(); client.responses().create(params).output().stream() .flatMap(item -> item.applyPatchCall().stream()) .forEach(System.out::println); ``` ```ruby require "openai" client = OpenAI::Client.new response = client.responses.create( model: "gpt-6-astra", input: "Rename fib() to fibonacci() in lib/fib.py and update run.py to use the new name.", tools: [{ type: :apply_patch }] ) patch_calls = response.output.select { |item| item.type == :apply_patch_call } puts(patch_calls) ``` **Example `apply_patch_call` object** Example apply_patch_call object ```json { "id": "apc_08f3d96c87a585390069118b594f7481a088b16cda7d9415fe", "type": "apply_patch_call", "status": "completed", "call_id": "call_Rjsqzz96C5xzPb0jUWJFRTNW", "operation": { "type": "update_file", "diff": " @@ -def fib(n): +def fibonacci(n): if n <= 1: return n - return fib(n-1) + fib(n-2) + return fibonacci(n-1) + fibonacci(n-2), ", "path": "lib/fib.py" } } ``` **Step 2: Apply the patch and send results back** Apply the patch and return results ```javascript const results = patchCalls.map((call) => { const { success, output } = applyOperation(call.operation); return { type: "apply_patch_call_output", call_id: call.call_id, status: success ? "completed" : "failed", output, }; }); const followup = await client.responses.create({ model: "gpt-6-astra", previous_response_id: response.id, input: results, tools: [{ type: "apply_patch" }], }); console.log(followup.output_text); ``` ```python from apply_patch_harness import apply_operation # your implementation results = [] for call in patch_calls: op = call["operation"] success, maybe_log_output = apply_operation(op) results.append( { "type": "apply_patch_call_output", "call_id": call["call_id"], "status": "completed" if success else "failed", "output": maybe_log_output, } ) followup = client.responses.create( model="gpt-6-astra", previous_response_id=response.id, input=results, tools=[{"type": "apply_patch"}], ) ``` ```go results := make(responses.ResponseInputParam, 0, len(patchCalls)) for _, call := range patchCalls { success, logOutput := applyOperation(call.Operation) status := "completed" if !success { status = "failed" } result := responses.ResponseInputItemParamOfApplyPatchCallOutput(call.CallID, status) result.OfApplyPatchCallOutput.Output = openai.String(logOutput) results = append(results, result) } _, err = client.Responses.New(context.Background(), responses.ResponseNewParams{ Model: "gpt-6-astra", PreviousResponseID: openai.String(response.ID), Input: responses.ResponseNewParamsInputUnion{OfInputItemList: results}, Tools: []responses.ToolUnionParam{{OfApplyPatch: &responses.ApplyPatchToolParam{}}}, }) if err != nil { panic(err) } ``` ```java import com.openai.client.OpenAIClient; import com.openai.client.okhttp.OpenAIOkHttpClient; import com.openai.models.responses.ApplyPatchTool; import com.openai.models.responses.ResponseCreateParams; import com.openai.models.responses.ResponseInputItem; import java.util.List; ResponseCreateParams params = ResponseCreateParams.builder() .model("gpt-6-astra") .inputOfResponse( List.of( ResponseInputItem.ofApplyPatchCallOutput( ResponseInputItem.ApplyPatchCallOutput.builder() .callId(System.getenv("OPENAI_EXAMPLE_APPLY_PATCH_CALL_ID")) .status(ResponseInputItem.ApplyPatchCallOutput.Status.COMPLETED) .output("Patch applied successfully.") .build()))) .previousResponseId(System.getenv("OPENAI_EXAMPLE_PREVIOUS_RESPONSE_ID")) .addTool(ApplyPatchTool.builder().build()) .build(); client.responses().create(params).output().stream() .flatMap(item -> item.message().stream()) .flatMap(message -> message.content().stream()) .flatMap(content -> content.outputText().stream()) .forEach(text -> System.out.println(text.text())); ``` ```ruby require "openai" client = OpenAI::Client.new response_id = ENV.fetch("OPENAI_RESPONSE_ID") patch_call_id = ENV.fetch("OPENAI_APPLY_PATCH_CALL_ID") response = client.responses.create( model: "gpt-6-astra", previous_response_id: response_id, input: [ { type: :apply_patch_call_output, call_id: patch_call_id, status: :completed, output: "Patch applied successfully." } ], tools: [{ type: :apply_patch }] ) puts(response.output_text) ``` If a patch fails (for example, file not found), set `status: "failed"` and include a helpful `output` string so the model can recover: Report a failed apply_patch call ```json { "type": "apply_patch_call_output", "call_id": "call_cNWm41dB3RyQcLNOVTIPBWZU", "status": "failed", "output": "Could not apply patch to lib/foo.py — file not found on disk" } ``` ## Apply patch operations | Operation Type | Purpose | Payload | | -------------- | ---------------------------------- | ---------------------------------------------------------------- | | `create_file` | Create a new file at `path`. | `diff` is a V4A diff representing the full file contents. | | `update_file` | Modify an existing file at `path`. | `diff` is a V4A diff with additions, deletions, or replacements. | | `delete_file` | Remove a file at `path`. | No `diff`; delete the file entirely. | Your patch harness is responsible for interpreting the V4A diff format and applying changes. For reference implementations, see the [Python Agents SDK](https://github.com/openai/openai-agents-python/blob/main/src/agents/apply_diff.py) or [TypeScript Agents SDK](https://github.com/openai/openai-agents-js/blob/main/packages/agents-core/src/utils/applyDiff.ts) code. ## Implementing the patch harness When using the `apply_patch` tool, you don’t provide an input schema; the model knows how to construct `operation` objects. Your job is to: 1. **Parse operations from the Response** - Scan the Response for items with `type: "apply_patch_call"`. - For each call, inspect `operation.type`, `operation.path`, and any potential `diff`. 2. **Apply file operations** - For `create_file` and `update_file`, apply the V4A diff to the file system or in-memory workspace. - For `delete_file`, remove the file at `path`. - Record whether each operation succeeded and any logs or error messages. 3. **Return `apply_patch_call_output` events** - For each `call_id`, emit exactly one `apply_patch_call_output` event with: - `status: "completed"` if the operation was applied successfully. - `status: "failed"` if you encountered an error (include a short human-readable `output` string). ### Safety and robustness - **Path validation**: Prevent directory traversal and restrict edits to allowed directories. - **Backups**: Consider backing up files (or working in a scratch copy) before applying patches. - **Error handling**: Always return a `failed` status with an informative `output` string when patches cannot be applied. - **Atomicity**: Decide whether you want “all-or-nothing” semantics (rollback if any patch fails) or per-file success/failure. ## Use the apply patch tool with the Agents SDK Alternatively, you can use the [Agents SDK](https://developers.openai.com/api/docs/guides/tools#usage-in-the-agents-sdk) to use the apply patch tool. You'll still have to implement the harness that handles the actual file operations but you can use the `applyDiff` function to handle the diff processing. Use the apply patch tool with the Agents SDK ```javascript import { applyDiff, Agent, run, applyPatchTool } from "@openai/agents"; class WorkspaceEditor { async createFile(operation) { // convert the diff to the file content const content = applyDiff("", operation.diff, "create"); // write the file content to the file system return { status: "completed", output: `Created ${operation.path}` }; } async updateFile(operation) { // read the file content from the file system const current = ""; // convert the diff to the new file content const newContent = applyDiff(current, operation.diff); // write the updated file content to the file system return { status: "completed", output: `Updated ${operation.path}` }; } async deleteFile(operation) { // delete the file from the file system return { status: "completed", output: `Deleted ${operation.path}` }; } } const editor = new WorkspaceEditor(); const agent = new Agent({ name: "Patch Assistant", model: "gpt-6-astra", instructions: "You can edit files inside the /tmp directory using the apply_patch tool.", tools: [ applyPatchTool({ editor, // could also be a function for you to determine if approval is needed needsApproval: true, onApproval: async (_ctx, _approvalItem) => { // create your own approval logic return { approve: true }; }, }), ], }); const result = await run( agent, "Create tasks.md with a shopping checklist of 5 entries." ); console.log(`\nFinal response:\n${result.finalOutput}`); ``` ```python from agents import Agent, ApplyPatchTool, Runner, apply_diff class WorkspaceEditor: async def create_file(self, operation): # convert the diff to the file content content = apply_diff("", operation.diff, mode="create") # write the file content to the file system return {"status": "completed", "output": f"Created {operation.path}"} async def update_file(self, operation): # read the file content from the file system current = "" # convert the diff to the new file content new_content = apply_diff(current, operation.diff) # write the updated file content to the file system return {"status": "completed", "output": f"Updated {operation.path}"} async def delete_file(self, operation): # delete the file from the file system return {"status": "completed", "output": f"Deleted {operation.path}"} editor = WorkspaceEditor() agent = Agent( name="Patch Assistant", model="gpt-6-astra", instructions="You can edit files inside the /tmp directory using the apply_patch tool.", tools=[ ApplyPatchTool( editor=editor, # could also be a function for you to determine if approval is needed needs_approval=True, # Implement your own approval logic on_approval=lambda _ctx, _approval_item: {"approve": True}, ), ], ) async def main(): result = await Runner.run( agent, input="Create tasks.md with a shopping checklist of 5 entries.", ) print(f"\nFinal response:\n{result.final_output}") if __name__ == "__main__": import asyncio asyncio.run(main()) ``` You can find full working examples on GitHub. [Apply patch tool example - TypeScript Example of how to use the apply patch tool with the Agents SDK in TypeScript](https://github.com/openai/openai-agents-js/blob/main/examples/tools/applyPatch.ts) [Apply patch tool example - Python Example of how to use the apply patch tool with the Agents SDK in Python](https://github.com/openai/openai-agents-python/blob/main/examples/tools/apply_patch.py) ## Handling common errors Use `status: "failed"` plus a clear `output` message to help the model recover. File not found File not found error ```json { "type": "apply_patch_call_output", "call_id": "call_abc", "status": "failed", "output": "Error: File not found at path 'lib/baz.py'" } ``` Patch conflict Patch conflict error ```json { "type": "apply_patch_call_output", "call_id": "call_abc", "status": "failed", "output": "Error: Invalid Context:\n@@ def fib(n):" } ``` The model can then adjust future diffs (for example, by re-reading a file in your prompt or simplifying a change) based on these error messages. ## Best practices - **Give clear file context** - When you call the Responses API, include either an inline snapshot of your files (as in the example), or give the model tools for exploring your filesystem (like the `shell` tool). - **Consider using with the `shell` tool** - When used in conjunction with the `shell` tool, the model can explore file system directories, read files, and grep for keywords, enabling agentic file discovery and editing. - **Encourage small, focused diffs** - In your system instructions, nudge the model toward minimal, targeted edits rather than huge rewrites. - **Make sure changes apply cleanly** - After a series of patches, run your tests or linters and share failures back in the next `input` so the model can fix them. ## Usage notes
API Availability Supported models
[Responses](https://developers.openai.com/api/reference/resources/responses) [Chat Completions](https://developers.openai.com/api/reference/resources/chat) [Assistants](https://developers.openai.com/api/reference/resources/beta/subresources/assistants) [GPT-5.5](https://developers.openai.com/api/docs/models/gpt-5.5) [GPT-5.4](https://developers.openai.com/api/docs/models/gpt-5.4) [GPT-5.2](https://developers.openai.com/api/docs/models/gpt-5.2) [GPT-5.1](https://developers.openai.com/api/docs/models/gpt-5.1)
--- # Architecture OpenAI runs the agent harness. Your application sends it work and receives results. Add an environment when the agent needs compute or files. ## The pieces - **Harness:** The OpenAI-hosted Codex instance that runs the model and tool loop and maintains the agent's session. - **Environment:** Where the agent runs commands, executes code, and works with files. An environment can be a remote sandbox, your laptop, a Docker container, or an AWS Lambda function. - **Application server:** Your code that connects the agent to your product. It submits tasks, receives events, and handles function tools. When you provide the environment, your code also manages its lifecycle. Start with the pieces your task needs. The harness can work without an environment, and your application can receive progress through streaming or webhooks. ## Start without an environment An agent that answers questions or uses tools to access external services may not need its own compute or files. Set `environment.type` to `none`. This fragment shows the environment setting. Session creation also needs an agent and initial input: ```json { "environment": { "type": "none" } } ``` Your application sends input to a session. The harness calls the model, uses the configured tools, and returns results. OpenAI maintains the session for later work. The harness can call remote MCP tools directly. For [function tools](https://developers.openai.com/api/docs/guides/agents-api/tools/functions), your code receives each call, runs the function, and returns its result. Without an environment, the built-in Bash and apply-patch tools, workspace files, and executor MCPs are unavailable. With no sandbox, the application supplies function tools or a virtual shell, and the Agents API can call remote MCP servers. There is no executor or built-in shell. The optional virtual runtime shown here provides files and shell commands through your application's function tools. ## Add an OpenAI-hosted environment When the agent needs to run scripts, edit files, or create artifacts, set `environment.type` to `openai_hosted`. OpenAI creates and manages a sandbox for the session. You configure the packages, files, and network access the agent needs. The harness runs commands in the sandbox directly. Your application continues to send tasks, receive events, and handle any function tools. An application starts sessions and receives events from the Agents API, which runs the managed Codex harness and exchanges tool calls and results with a sandbox. The application controls compute only for self-hosted sandboxes. The dashed arrow applies only when you manage the environment yourself, as described below. See [OpenAI-hosted environments](https://developers.openai.com/api/docs/guides/agents-api/environments/openai-hosted) for configuration options. ## Connect your own environment Use `environment.type: "self_hosted"` when the agent needs your infrastructure, private network, or custom software. Your code starts the environment and connects an executor to the session. The executor runs the commands and tools that the harness requests. Your application manages the connection and lifecycle without forwarding each command. You own provisioning, reconnection, shutdown, and any files you need to preserve. Your application server or a webhook handler can manage this work. The application creates a self-hosted session, starts compute, and connects an executor. It receives events and checks the turn outcome before stopping compute. Before stopping compute, coordinate incoming work and confirm that no execution is pending. See [Connect a sandbox](https://developers.openai.com/api/docs/guides/agents-api/environments/self-hosted) and [Sandbox lifecycle](https://developers.openai.com/api/docs/guides/agents-api/environments/lifecycle) for setup and shutdown requirements. ## Receive progress and results With any environment choice, you can use either or both of these: - **Streaming:** Receive detailed events as the agent works, such as output to display in your product. - **Webhooks:** Receive session state changes without keeping a stream open. Your handler can retrieve results, run function tools, or manage a self-hosted environment. Function tools need a handler that receives calls and returns results. If that handler is unavailable, the agent can remain waiting for a result. Failures in your event or lifecycle handlers can also interrupt progress updates or environment management. See [Session events](https://developers.openai.com/api/docs/guides/agents-api/sessions) and [Webhooks](https://developers.openai.com/api/docs/guides/agents-api/sessions/webhooks) for integration details. --- # Assistants migration guide The Assistants API was officially sunset on August 26, 2026, and is no longer available. Use the [Responses API](https://developers.openai.com/api/docs/guides/migrate-to-responses) for new integrations. Thank you to everyone who used the Assistants API. We appreciate everything you built and the feedback you shared along the way. Use this guide to migrate your integration to the [Responses API](https://developers.openai.com/api/docs/guides/migrate-to-responses). Responses are simpler—send input items and get output items back. With the Responses API, you also get better performance and new features like [deep research](https://developers.openai.com/api/docs/guides/deep-research), [MCP](https://developers.openai.com/api/docs/guides/tools-connectors-mcp), and [computer use](https://developers.openai.com/api/docs/guides/tools-computer-use). This change also lets you manage conversations instead of passing back `previous_response_id`. ### What's changed?
Before Now Why?
`Assistants` `Prompts` Prompts hold configuration (model, tools, instructions) and are easier to version and update
`Threads` `Conversations` Streams of items instead of just messages
`Runs` `Responses` Responses send input items or use a conversation object and receive output items; tool call loops are explicitly managed
`Run steps` `Items` Generalized objects—can be messages, tool calls, outputs, and more
## From assistants to prompts Assistants were persistent API objects that bundled model choice, instructions, and tool declarations—created and managed entirely through the API. Their replacement, prompts, can only be created in the dashboard, where you can version them as you develop your product. ### Why this is helpful - **Portability and versioning**: You can snapshot, review, diff, and roll back prompt specs. You can also version a prompt, so your code can just point the latest version. - **Separation of concerns**: Your application code now handles orchestration (history pruning, tool loop, retries) while your prompt focuses on high‑level behavior and constraints (system guidance, tool availability, structured output schema, temperature defaults). - **Realtime compatibility**: The same prompt configuration can be reused when you connect through the Realtime API, giving you a single definition of behavior across chat, streaming, and low‑latency interactive sessions. - **Tool and output consistency**: Using prompts, every Responses or Realtime session you start inherits a consistent contract because prompts encapsulate tool schemas and structured output expectations. ### Practical migration steps 1. Identify each existing Assistant’s _instruction + tool_ bundle. 2. In the dashboard, recreate that bundle as a named prompt. 3. Store the prompt ID (or its exported spec) in source control so application code can refer to a stable identifier. 4. During rollout, run A/B tests by swapping prompt IDs—no need to create or delete assistant objects programmatically. Think of a prompt as a **versioned behavioral profile** to plug into either Responses or Realtime API. --- ## From threads to conversations A thread was a collection of messages stored server-side. Threads could _only_ store messages. Conversations store items, which can include messages, tool calls, tool outputs, and other data. ### Request example #### Python #### Go ### Response example #### Thread object ```json { "id": "thread_CrXtCzcyEQbkAcXuNmVSKFs1", "object": "thread", "created_at": 1752855924, "metadata": { "user_id": "peter_le_fleur" }, "tool_resources": {} } ``` #### Conversation object ```json { "id": "conv_68542dc602388199a30af27d040cefd4087a04b576bfeb24", "object": "conversation", "created_at": 1752855924, "metadata": { "user_id": "peter_le_fleur" } } ``` --- ## From runs to responses Runs were asynchronous processes that executed against threads. See the example below. Responses are simpler: provide a set of input items to execute, and get a list of output items back. Responses are designed to be used alone, but you can also use them with prompt and conversation objects for storing context and configuration. ### Request example #### Python #### Go ### Response example #### Run object ```json { "id": "run_FKIpcs5ECSwuCmehBqsqkORj", "assistant_id": "asst_8fVY45hU3IM6creFkVi5MBKB", "cancelled_at": null, "completed_at": 1752857327, "created_at": 1752857322, "expires_at": null, "failed_at": null, "incomplete_details": null, "instructions": null, "last_error": null, "max_completion_tokens": null, "max_prompt_tokens": null, "metadata": {}, "model": "gpt-4.1", "object": "thread.run", "parallel_tool_calls": true, "required_action": null, "response_format": "auto", "started_at": 1752857324, "status": "completed", "thread_id": "thread_CrXtCzcyEQbkAcXuNmVSKFs1", "tool_choice": "auto", "tools": [], "truncation_strategy": { "type": "auto", "last_messages": null }, "usage": { "completion_tokens": 130, "prompt_tokens": 34, "total_tokens": 164, "prompt_token_details": { "cached_tokens": 0 }, "completion_tokens_details": { "reasoning_tokens": 0 } }, "temperature": 1.0, "top_p": 1.0, "tool_resources": {}, "reasoning_effort": null } ``` #### Response object ```json { "id": "resp_687a7b53036c819baad6012d58b39bcb074adcd9e24850fc", "created_at": 1752857427, "conversation": { "id": "conv_689667905b048191b4740501625afd940c7533ace33a2dab" }, "error": null, "incomplete_details": null, "instructions": null, "metadata": {}, "model": "gpt-5.5", "object": "response", "output": [ { "id": "msg_687a7b542948819ba79e77e14791ef83074adcd9e24850fc", "content": [ { "annotations": [], "text": "The \"5 Ds of Dodgeball\" are a humorous set of rules made famous by the 2004 comedy film **\"Dodgeball: A True Underdog Story.\"** In the movie, dodgeball coach Patches O’Houlihan teaches these basics to his team. The **5 Ds** are:\n\n1. **Dodge**\n2. **Duck**\n3. **Dip**\n4. **Dive**\n5. **Dodge** (yes, dodge is listed twice for emphasis!)\n\nIn summary: \n> **“If you can dodge a wrench, you can dodge a ball!”**\n\nThese 5 Ds are not official competitive rules, but have become a fun and memorable pop culture reference for the sport of dodgeball.", "type": "output_text", "logprobs": [] } ], "role": "assistant", "status": "completed", "type": "message" } ], "parallel_tool_calls": true, "temperature": 1.0, "tool_choice": "auto", "tools": [], "top_p": 1.0, "background": false, "max_output_tokens": null, "previous_response_id": null, "reasoning": { "effort": null, "generate_summary": null, "summary": null }, "service_tier": "scale", "status": "completed", "text": { "format": { "type": "text" } }, "truncation": "disabled", "usage": { "input_tokens": 17, "input_tokens_details": { "cached_tokens": 0 }, "output_tokens": 150, "output_tokens_details": { "reasoning_tokens": 0 }, "total_tokens": 167 }, "user": null, "max_tool_calls": null, "store": true, "top_logprobs": 0 } ``` --- ## Migrating your integration Follow the migration steps below to move from the Assistants API to the Responses API, without losing any feature support. ### 1. Create prompts from your assistants 1. Identify the most important assistant objects in your application. 1. Find these in the dashboard and click `Create prompt`. This will create a prompt object out of each existing assistant object. Reusable prompt objects are also being deprecated. If you use this migration path, review the [prompts deprecation timeline](https://developers.openai.com/api/docs/deprecations#2026-06-03-reusable-prompts) before adopting prompt objects in a long-lived integration. ### 2. Move new user chats over to conversations and responses Start new chats with the Conversations API and Responses API. To preserve earlier conversation history, use messages already stored by your application. The example below shows how thread history could be migrated before the sunset. The Assistants API call that retrieves thread messages no longer works; use your stored messages instead. ```python # Replace the illustrative IDs and URLs below with your own resource values. from openai import OpenAI openai = OpenAI() messages = [] thread_id = "thread_123" for page in openai.beta.threads.messages.list( thread_id=thread_id, order="asc" ).iter_pages(): messages += page.data items = [] for m in messages: item = {"role": m.role} item_content = [] for content in m.content: match content.type: case "text": item_content_type = "input_text" if m.role == "user" else "output_text" item_content += [ {"type": item_content_type, "text": content.text.value} ] case "image_url": item_content += [ { "type": "input_image", "image_url": content.image_url.url, "detail": content.image_url.detail, } ] item |= {"content": item_content} items.append(item) # create a conversation with your converted items conversation = openai.conversations.create(items=items) ``` ```ruby # Replace the illustrative IDs and URLs below with your own resource values. require "openai" client = OpenAI::Client.new thread_id = "thread_123" messages = client.beta.threads.messages.list(thread_id, order: :asc) items = [] messages.auto_paging_each do |message| content = message.content.filter_map do |part| case part when OpenAI::Models::Beta::Threads::TextContentBlock type = if message.role == OpenAI::Models::Beta::Threads::Message::Role::USER :input_text else :output_text end { type: type, text: part.text.value } when OpenAI::Models::Beta::Threads::ImageURLContentBlock { type: :input_image, image_url: part.image_url.url, detail: part.image_url.detail } end end items << { role: message.role, content: content } end conversation = client.conversations.create( items: items ) puts(conversation.id) ``` ## Comparing full examples Here are a few examples of integrations using both the Assistants API and the Responses API so you can see how they compare. ### User chat app Assistants API ```python # Replace the illustrative IDs and URLs below with your own resource values. threads_by_session: dict[str, str] = {} @app.post("/messages") async def message(message: Message): thread_id = threads_by_session.get(message.session_id) if thread_id is None: thread_id = openai.beta.threads.create().id threads_by_session[message.session_id] = thread_id openai.beta.threads.messages.create( thread_id=thread_id, role="user", content=message.content, ) example_assistant_id = "asst_123" run = openai.beta.threads.runs.create( assistant_id=example_assistant_id, thread_id=thread_id, ) while run.status in ("queued", "in_progress"): await asyncio.sleep(1) run = openai.beta.threads.runs.retrieve( thread_id=thread_id, run_id=run.id, ) messages = openai.beta.threads.messages.list( order="desc", limit=1, thread_id=thread_id, ) return {"content": messages.data[0].content} ``` ```ruby # Replace the illustrative IDs and URLs below with your own resource values. require "openai" client = OpenAI::Client.new assistant_id = "asst_123" threads_by_session = {} handle_message = lambda do |session_id:, content:| thread_id = threads_by_session[session_id] unless thread_id thread_id = client.beta.threads.create.id threads_by_session[session_id] = thread_id end client.beta.threads.messages.create( thread_id, role: :user, content: content ) run = client.beta.threads.runs.create( thread_id, assistant_id: assistant_id ) while [:queued, :in_progress].include?(run.status) sleep(1) run = client.beta.threads.runs.retrieve(run.id, thread_id: thread_id) end messages = client.beta.threads.messages.list( thread_id, order: :desc, limit: 1 ) { content: messages.data&.first&.content } end puts( handle_message.call( session_id: "example-session", content: "What are the five Ds of dodgeball?" ) ) ``` Responses API ```javascript // Replace the illustrative IDs and URLs below with your own resource values. import express from "express"; import OpenAI from "openai"; const app = express(); const client = new OpenAI(); const conversationsBySession = new Map(); app.use(express.json()); app.post("/messages", async (request, response) => { const { content, session_id: sessionId } = request.body ?? {}; if ( typeof content !== "string" || !content.trim() || typeof sessionId !== "string" || !sessionId.trim() ) { response.status(400).json({ error: "content and session_id must be non-empty strings.", }); return; } let conversationIdPromise = conversationsBySession.get(sessionId); if (!conversationIdPromise) { conversationIdPromise = client.conversations .create() .then((conversation) => conversation.id) .catch((error) => { conversationsBySession.delete(sessionId); throw error; }); conversationsBySession.set(sessionId, conversationIdPromise); } const conversationId = await conversationIdPromise; const promptId = "pmpt_123"; const result = await client.responses.create({ prompt: { id: promptId }, input: [{ role: "user", content }], conversation: conversationId, }); response.json({ content: result.output_text }); }); app.listen(Number(process.env.OPENAI_EXAMPLE_PORT ?? 8000), "127.0.0.1"); ``` ```python # Replace the illustrative IDs and URLs below with your own resource values. conversations_by_session: dict[str, str] = {} @app.post("/messages") async def message(message: Message): conversation_id = conversations_by_session.get(message.session_id) if conversation_id is None: conversation_id = openai.conversations.create().id conversations_by_session[message.session_id] = conversation_id example_prompt_id = "pmpt_123" response = openai.responses.create( prompt={"id": example_prompt_id}, input=[{"role": "user", "content": message.content}], conversation=conversation_id, ) return {"content": response.output_text} ``` ```ruby # Replace the illustrative IDs and URLs below with your own resource values. require "openai" client = OpenAI::Client.new conversations_by_session = {} handle_message = lambda do |session_id:, content:| conversation_id = conversations_by_session[session_id] unless conversation_id conversation_id = client.conversations.create.id conversations_by_session[session_id] = conversation_id end response = client.responses.create( prompt: { id: "pmpt_123" }, input: [ { role: :user, content: content } ], conversation: conversation_id ) { content: response.output_text } end puts( handle_message.call( session_id: "example-session", content: "What are the five Ds of dodgeball?" ) ) ``` --- # Async tool calling Async tool calling lets the model continue working after it calls a tool, without waiting for that tool's result. Use it to start slow lookup requests early, answer independent parts of a request, and provide results when your application has them. ## How async tools work A normal [function call](https://developers.openai.com/api/docs/guides/function-calling) pauses the model's turn to wait for a tool response. Set `async: true` on a function or custom tool definition to let the model continue working after issuing that call, before your application returns the output. Your application still executes the tool. Async tools don't move execution to OpenAI or manage your background jobs. This differs from [Background mode](https://developers.openai.com/api/docs/guides/background), which runs response generation asynchronously. Async tool calling lets the model continue working while your application runs a tool. When a job finishes, include its output in a later Responses request. Use the original API `call_id` to match the result to its call: | Tool type | Call item | Output item | | --------- | ------------------ | ------------------------- | | Function | `function_call` | `function_call_output` | | Custom | `custom_tool_call` | `custom_tool_call_output` | ## Call an async tool Add `async: true` to the tool definition. The corresponding call items in `response.output` include `async: true`. Run a weather lookup in the background ```javascript import OpenAI from "openai"; const client = new OpenAI(); const model = "gpt-6-astra"; const tools = [ { type: "function", name: "get_weather", description: "Read a demo weather snapshot for a city.", async: true, strict: true, parameters: { type: "object", properties: { city: { type: "string" } }, required: ["city"], additionalProperties: false, }, }, ]; async function getWeather(city) { const snapshots = { Paris: { city: "Paris", temperature_c: 22, condition: "Clear", source: "demo weather snapshot", }, }; if (typeof city !== "string" || !Object.hasOwn(snapshots, city)) { throw new Error(`No demo weather snapshot for ${city}.`); } return snapshots[city]; } const instructions = "Start the weather lookup and answer the independent packing question " + "without waiting. Use the demo weather result when it arrives; never invent it."; let response = await client.responses.create({ model, tools, instructions, input: "Check the demo weather snapshot for Paris. Meanwhile, " + "list three essentials for any city trip.", }); const call = response.output.find((item) => item.type === "function_call"); if (!call || call.name !== "get_weather") { throw new Error("The response did not include a weather call."); } const { city } = JSON.parse(call.arguments); let latestResponseId = response.id; // Calling an async function starts the application's job immediately. const job = getWeather(city).catch((error) => ({ error: error.message })); if (!call.async) { // Ordinary synchronous calls must finish before the model resumes. await job; } console.log(response.output); // Independent work or conversation turns can happen here. // Update latestResponseId after each continuation. const result = await job; response = await client.responses.create({ model, tools, instructions, previous_response_id: latestResponseId, input: [ { type: "function_call_output", call_id: call.call_id, output: JSON.stringify(result), }, ], }); latestResponseId = response.id; console.log(response.output); ``` ```python import json from concurrent.futures import ThreadPoolExecutor from openai import OpenAI from openai.types.responses import FunctionToolParam def get_weather(city): # Demo data. Replace this function with your weather service. weather = { "Paris": { "city": "Paris", "temperature_c": 22, "condition": "Clear", "source": "demo weather snapshot", } } return weather[city] worker = ThreadPoolExecutor() def main(): client = OpenAI() model = "gpt-6-astra" tools: list[FunctionToolParam] = [ { "type": "function", "name": "get_weather", "description": "Read the demo weather snapshot for a city.", "async": True, "strict": True, "parameters": { "type": "object", "properties": {"city": {"type": "string"}}, "required": ["city"], "additionalProperties": False, }, }, ] instructions = ( "Start the weather lookup and answer the independent packing " "question without waiting. Use the actual tool result when it " "arrives; never invent it. Identify the weather as demo data." ) response = client.responses.create( model=model, tools=tools, instructions=instructions, input=( "Check the demo weather in Paris. Meanwhile, " "list three essentials for any city trip." ), ) call = next(item for item in response.output if item.type == "function_call") arguments = json.loads(call.arguments) if call.name != "get_weather" or arguments != {"city": "Paris"}: raise ValueError("Expected a weather lookup for Paris") latest_response_id = response.id if call.async_: job = worker.submit(get_weather, **arguments) print(response.output_text) # Independent work or conversation turns can happen here. # Update latest_response_id after each continuation. result = job.result() else: result = get_weather(**arguments) response = client.responses.create( model=model, tools=tools, instructions=instructions, previous_response_id=latest_response_id, input=[ { "type": "function_call_output", "call_id": call.call_id, "output": json.dumps(result), }, ], ) print(response.output_text) if __name__ == "__main__": try: main() finally: worker.shutdown(wait=True) ``` ```go package main import ( "context" "encoding/json" "fmt" "github.com/openai/openai-go/v3" "github.com/openai/openai-go/v3/responses" ) type weatherArguments struct { City string `json:"city"` } type weatherSnapshot struct { City string `json:"city"` TemperatureC int `json:"temperature_c"` Condition string `json:"condition"` Source string `json:"source"` } func getWeather(city string) weatherSnapshot { // Demo data. Replace this function with your weather service. if city != "Paris" { panic("No demo weather snapshot for " + city) } return weatherSnapshot{ City: city, TemperatureC: 22, Condition: "Clear", Source: "demo weather snapshot", } } func main() { client := openai.NewClient() ctx := context.Background() tool := responses.ToolParamOfFunction("get_weather", map[string]any{ "type": "object", "properties": map[string]any{"city": map[string]string{"type": "string"}}, "required": []string{"city"}, "additionalProperties": false, }, true) tool.OfFunction.Description = openai.String("Read the demo weather snapshot for a city.") tool.OfFunction.Async = openai.Bool(true) tools := []responses.ToolUnionParam{tool} instructions := "Start the weather lookup and answer the independent packing question " + "without waiting. Use the actual tool result when it arrives; never invent it. " + "Identify the weather as demo data." response, err := client.Responses.New(ctx, responses.ResponseNewParams{ Model: "gpt-6-astra", Tools: tools, Instructions: openai.String(instructions), Input: responses.ResponseNewParamsInputUnion{OfString: openai.String("Check the demo weather in Paris. Meanwhile, list three essentials for any city trip.")}, }) if err != nil { panic(err) } var call responses.ResponseFunctionToolCall for _, item := range response.Output { if item.Type == "function_call" && item.AsFunctionCall().Name == "get_weather" { call = item.AsFunctionCall() break } } if call.CallID == "" { panic("The response did not include a weather call.") } var arguments weatherArguments if err := json.Unmarshal([]byte(call.Arguments), &arguments); err != nil { panic(err) } latestResponseID := response.ID var result weatherSnapshot if call.Async { job := make(chan weatherSnapshot, 1) go func() { job <- getWeather(arguments.City) }() fmt.Println(response.OutputText()) // Independent work or conversation turns can happen here. // Update latestResponseID after each continuation. result = <-job } else { result = getWeather(arguments.City) } output, err := json.Marshal(result) if err != nil { panic(err) } functionOutput := responses.ResponseInputItemParamOfFunctionCallOutput(string(output)) functionOutput.OfFunctionCallOutput.CallID = openai.String(call.CallID) response, err = client.Responses.New(ctx, responses.ResponseNewParams{ Model: "gpt-6-astra", Tools: tools, Instructions: openai.String(instructions), PreviousResponseID: openai.String(latestResponseID), Input: responses.ResponseNewParamsInputUnion{OfInputItemList: responses.ResponseInputParam{functionOutput}}, }) if err != nil { panic(err) } fmt.Println(response.OutputText()) } ``` ```java import com.fasterxml.jackson.annotation.JsonProperty; import com.fasterxml.jackson.databind.ObjectMapper; import com.openai.core.JsonValue; import com.openai.models.responses.FunctionTool; import com.openai.models.responses.Response; import com.openai.models.responses.ResponseCreateParams; import com.openai.models.responses.ResponseFunctionToolCall; import com.openai.models.responses.ResponseInputItem; import java.util.List; import java.util.Map; import java.util.concurrent.CompletableFuture; record WeatherArguments(String city) {} record WeatherSnapshot( String city, @JsonProperty("temperature_c") int temperatureC, String condition, String source) {} static WeatherSnapshot getWeather(String city) { // Demo data. Replace this function with your weather service. if (!city.equals("Paris")) { throw new IllegalArgumentException("No demo weather snapshot for " + city); } return new WeatherSnapshot(city, 22, "Clear", "demo weather snapshot"); } FunctionTool tool = FunctionTool.builder() .name("get_weather") .description("Read the demo weather snapshot for a city.") .async(true) .strict(true) .parameters( FunctionTool.Parameters.builder() .putAdditionalProperty("type", JsonValue.from("object")) .putAdditionalProperty( "properties", JsonValue.from(Map.of("city", Map.of("type", "string")))) .putAdditionalProperty("required", JsonValue.from(List.of("city"))) .putAdditionalProperty("additionalProperties", JsonValue.from(false)) .build()) .build(); String instructions = "Start the weather lookup and answer the independent packing question without waiting. Use the actual tool result when it arrives; never invent it. Identify the weather as demo data."; Response response = client .responses() .create( ResponseCreateParams.builder() .model("gpt-6-astra") .addTool(tool) .instructions(instructions) .input( "Check the demo weather in Paris. Meanwhile, list three essentials for any city trip.") .build()); ResponseFunctionToolCall call = response.output().stream() .flatMap(item -> item.functionCall().stream()) .filter(item -> item.name().equals("get_weather")) .findFirst() .orElseThrow( () -> new IllegalStateException("The response did not include a weather call.")); WeatherArguments arguments = call.arguments(WeatherArguments.class); String latestResponseId = response.id(); WeatherSnapshot result; if (call.async().orElse(false)) { CompletableFuture job = CompletableFuture.supplyAsync(() -> getWeather(arguments.city())); System.out.println(response.output()); // Independent work or conversation turns can happen here. // Update latestResponseId after each continuation. result = job.join(); } else { result = getWeather(arguments.city()); } response = client .responses() .create( ResponseCreateParams.builder() .model("gpt-6-astra") .addTool(tool) .instructions(instructions) .previousResponseId(latestResponseId) .inputOfResponse( List.of( ResponseInputItem.ofFunctionCallOutput( ResponseInputItem.FunctionCallOutput.builder() .callId(call.callId()) .output(new ObjectMapper().writeValueAsString(result)) .build()))) .build()); response.output().stream() .flatMap(item -> item.message().stream()) .flatMap(message -> message.content().stream()) .flatMap(content -> content.outputText().stream()) .forEach(text -> System.out.println(text.text())); ``` ```ruby require "json" require "openai" def get_weather(city) # Demo data. Replace this function with your weather service. raise "No demo weather snapshot for #{city}" unless city == "Paris" { city: city, temperature_c: 22, condition: "Clear", source: "demo weather snapshot" } end client = OpenAI::Client.new tools = [ OpenAI::Models::Responses::FunctionTool.new( name: "get_weather", description: "Read the demo weather snapshot for a city.", async: true, strict: true, parameters: { type: "object", properties: { city: { type: "string" } }, required: ["city"], additionalProperties: false } ) ] instructions = "Start the weather lookup and answer the independent packing question " \ "without waiting. Use the actual tool result when it arrives; never invent it. " \ "Identify the weather as demo data." response = client.responses.create( model: "gpt-6-astra", tools: tools, instructions: instructions, input: "Check the demo weather in Paris. Meanwhile, list three essentials for any city trip." ) call = response.output.find do |item| item.is_a?(OpenAI::Models::Responses::ResponseFunctionToolCall) && item.name == "get_weather" end unless call.is_a?(OpenAI::Models::Responses::ResponseFunctionToolCall) raise "The response did not include a weather call." end city = JSON.parse(call.arguments).fetch("city") latest_response_id = response.id result = if call.async job = Thread.new { get_weather(city) } puts(response.output_text) # Independent work or conversation turns can happen here. # Update latest_response_id after each continuation. job.value else get_weather(city) end response = client.responses.create( model: "gpt-6-astra", tools: tools, instructions: instructions, previous_response_id: latest_response_id, input: [ OpenAI::Models::Responses::ResponseInputItem::FunctionCallOutput.new( call_id: call.call_id, output: JSON.generate(result) ) ] ) puts(response.output_text) ``` The response can contain both the async call and an answer. If other conversation turns happen before the job finishes, update `latest_response_id` to continue from the latest response while keeping the original tool `call_id`. For earlier dispatch with [streaming](https://developers.openai.com/api/docs/guides/streaming-responses), start the job when its complete call item arrives while you continue consuming the response. ## Add a wait tool A wait tool lets the model choose when it needs a pending result. For example, it can launch two price lookup requests, work on something independent, and wait only when it's ready to compare prices. Add a `task_handle` argument to each async tool. The model assigns a handle to each call, and your application binds it to the original API `call_id` and the running job. Keep handles unique throughout the conversation, including completed tasks and repeated lookup requests. Define the wait tool as an ordinary synchronous function: omit `async` or set it to `false`. Its schema and behavior belong to your application. `wait_for_tasks` isn't a built-in Responses tool. Use these definitions in the request's `tools` array: ```json [ { "type": "function", "name": "lookup_price", "async": true, "description": "Look up a product price in the background. Choose a fresh task_handle unique within this conversation, including completed tasks.", "strict": true, "parameters": { "type": "object", "properties": { "sku": { "type": "string" }, "task_handle": { "type": "string" } }, "required": ["sku", "task_handle"], "additionalProperties": false } }, { "type": "function", "name": "wait_for_tasks", "description": "Wait for selected tasks whose results you need. Pass a nonempty list of distinct task_handles from your earlier lookup_price calls. Results arrive on their original calls; this tool returns status only. Do not wait again for results that have already arrived.", "strict": true, "parameters": { "type": "object", "properties": { "task_handles": { "type": "array", "items": { "type": "string" } } }, "required": ["task_handles"], "additionalProperties": false } } ] ``` ### Register each job Register and start each launch before processing a dependent wait. Calls can arrive together or across responses. The following illustrative output items show two launches and a wait that depends on both: ```json [ { "type": "function_call", "name": "lookup_price", "async": true, "call_id": "call_widget", "arguments": "{\"sku\":\"WIDGET\",\"task_handle\":\"widget_price_1\"}" }, { "type": "function_call", "name": "lookup_price", "async": true, "call_id": "call_gadget", "arguments": "{\"sku\":\"GADGET\",\"task_handle\":\"gadget_price_1\"}" }, { "type": "function_call", "name": "wait_for_tasks", "call_id": "call_wait", "arguments": "{\"task_handles\":[\"widget_price_1\",\"gadget_price_1\"]}" } ] ``` Your application's registry binds each handle to its original call and running job: | Task handle | Original call ID | Job | | ---------------- | ---------------- | ------------------- | | `widget_price_1` | `call_widget` | WIDGET price lookup | | `gadget_price_1` | `call_gadget` | GADGET price lookup | Keep the registry for the entire conversation to prevent reuse of a completed task's handle. ### Deliver results before wait status Resolve the requested handles in the registry and await only those jobs. Return each newly completed result on its original `call_id`, then return status on the wait call's own `call_id`. This order gives the model the results when it resumes. For example, send these output items in the next request's `input` array. The prices are illustrative: ```json [ { "type": "function_call_output", "call_id": "call_widget", "output": "{\"task_handle\":\"widget_price_1\",\"price_cents\":1200,\"currency\":\"USD\"}" }, { "type": "function_call_output", "call_id": "call_gadget", "output": "{\"task_handle\":\"gadget_price_1\",\"price_cents\":1500,\"currency\":\"USD\"}" }, { "type": "function_call_output", "call_id": "call_wait", "output": "{\"status\":\"completed\",\"completed_task_handles\":[\"widget_price_1\",\"gadget_price_1\"]}" } ] ``` Set `previous_response_id` to the latest response ID, and include the tools and instructions in the continuation request. Your application can also deliver results as they become available, without a wait call. Only use the wait tool when the model's next step depends on results that haven't arrived. ## Compatibility Async tool calling is supported by GPT-6 Astra and later models. Async execution applies to function and custom tools that your application runs. It doesn't apply to hosted built-in tools. Use direct tool calls; don't configure async tools for [programmatic tool calling](https://developers.openai.com/api/docs/guides/tools-programmatic-tool-calling). In [Multi-agent mode](https://developers.openai.com/api/docs/guides/responses-multi-agent), don't combine async tools with parallel tool calls. --- # Audio and voice For a new conversational voice application, start with **[GPT-Live](https://developers.openai.com/api/docs/guides/live)**. It can listen while speaking and keep the conversation moving while a backend agent reasons, uses tools, or completes a task. Connect your first conversation with the [WebRTC quickstart](https://developers.openai.com/api/docs/guides/voice-webrtc?api=live), then write a short [Live prompt](https://developers.openai.com/api/docs/guides/live-prompting). If you already have a Realtime application or text agent, follow [Migrate to GPT-Live](https://developers.openai.com/api/docs/guides/live-migration). ## Choose another audio workflow Use the Realtime API when you need its session and tool model. For transcription, translation, or speech generation without a conversational agent, choose the dedicated API below. | Build | Start here | What you control | | ------------------------------------------------------------------ | ---------------------------------------------------------------------------- | ----------------------------------------------------------------- | | A speech-to-speech agent using the Realtime session and tool model | [Realtime API](https://developers.openai.com/api/docs/guides/realtime) | Audio turns, session state, tools, and interruptions. | | A voice interface for an existing text agent | [Voice agents](https://developers.openai.com/api/docs/guides/voice-agents#build-a-chained-voice-workflow) | Speech-to-text, the text-agent workflow, then text-to-speech. | | A transcript of an audio file | [File transcription](https://developers.openai.com/api/docs/guides/speech-to-text) | File uploads, bounded requests, and supported transcript formats. | | Live captions without assistant speech | [Live transcription](https://developers.openai.com/api/docs/guides/realtime-transcription) | Streaming audio and incremental transcript events. | | Continuous speech translation | [Live translation](https://developers.openai.com/api/docs/guides/realtime-translation) | A dedicated translation session, not a voice-agent turn loop. | | Narration or generated speech | [Text to speech](https://developers.openai.com/api/docs/guides/text-to-speech) | Text, voice, and output format. | | Audio input or output in an existing chat app | [Audio in Chat Completions](https://developers.openai.com/api/docs/guides/audio-chat-completions) | Bounded multimodal chat requests. | ## Build with voice Use [Voice agents](https://developers.openai.com/api/docs/guides/voice-agents) to compare architectures. Start with the prompting guide for [GPT-Live](https://developers.openai.com/api/docs/guides/live-prompting) or [Realtime](https://developers.openai.com/api/docs/guides/voice-prompting). Then use the shared guides for [custom voices](https://developers.openai.com/api/docs/guides/custom-voices), [evaluation](https://developers.openai.com/api/docs/guides/voice-agents#evaluate-your-voice-agent), and [cost optimization](https://developers.openai.com/api/docs/guides/voice-latency-cost). Each guide distinguishes model- or API-specific behavior. ## Choose a connection For browser audio, start with [WebRTC](https://developers.openai.com/api/docs/guides/voice-webrtc). For server audio pipelines, use [WebSockets](https://developers.openai.com/api/docs/guides/voice-websockets). For phone calls, see [Telephony and SIP](https://developers.openai.com/api/docs/guides/voice-sip). A [server-side control connection](https://developers.openai.com/api/docs/guides/voice-server-controls) lets a trusted backend observe and control a media session. Select your API on each connection page. Sharing a transport does not make GPT-Live and Realtime handshakes, credentials, or event formats interchangeable. Check the connection guide for prerequisites and setup instructions. ## Add audio to your existing application The Chat Completions examples now live in [Audio in Chat Completions](https://developers.openai.com/api/docs/guides/audio-chat-completions). For a browser voice-agent starter, use the [GPT-Live WebRTC quickstart](https://developers.openai.com/api/docs/guides/voice-webrtc?api=live). --- # Audio in Chat Completions If you already have a text-based LLM application with the [Chat Completions endpoint](https://developers.openai.com/api/reference/resources/chat), you may want to add audio capabilities. For example, if your chat application supports text input, you can add audio input and output: include `audio` in the `modalities` array and use an audio model, like [`gpt-audio-1.5`](https://developers.openai.com/api/docs/models/gpt-audio-1.5). The [Responses API](https://developers.openai.com/api/reference/resources/responses) docs currently describe text and image inputs with text outputs. For this audio-chat pattern, use Chat Completions with an audio-capable model. Audio output from model Create a human-like audio response to a prompt ```javascript import { writeFileSync } from "node:fs"; import OpenAI from "openai"; const openai = new OpenAI(); // Generate an audio response to the given prompt const response = await openai.chat.completions.create({ model: "gpt-audio-1.5", modalities: ["text", "audio"], audio: { voice: "alloy", format: "wav" }, messages: [ { role: "user", content: "Is a golden retriever a good family dog?", }, ], store: true, }); // Inspect returned data console.log(response.choices[0]); // Write audio data to a file writeFileSync( "dog.wav", Buffer.from(response.choices[0].message.audio.data, "base64"), { encoding: "utf-8" } ); ``` ```python import base64 from openai import OpenAI client = OpenAI() completion = client.chat.completions.create( model="gpt-audio-1.5", modalities=["text", "audio"], audio={"voice": "alloy", "format": "wav"}, messages=[{"role": "user", "content": "Is a golden retriever a good family dog?"}], ) print(completion.choices[0]) wav_bytes = base64.b64decode(completion.choices[0].message.audio.data) with open("dog.wav", "wb") as f: f.write(wav_bytes) ``` ```go package main import ( "context" "encoding/base64" "fmt" "os" "github.com/openai/openai-go/v3" ) func main() { client := openai.NewClient() response, err := client.Chat.Completions.New(context.Background(), openai.ChatCompletionNewParams{ Model: "gpt-audio-1.5", Modalities: []string{"text", "audio"}, Audio: openai.ChatCompletionAudioParam{ Voice: openai.ChatCompletionAudioParamVoiceUnion{OfString: openai.String("alloy")}, Format: openai.ChatCompletionAudioParamFormatWAV, }, Messages: []openai.ChatCompletionMessageParamUnion{openai.UserMessage("Is a golden retriever a good family dog?")}, }) if err != nil { panic(err) } fmt.Println(response.Choices[0]) audio, err := base64.StdEncoding.DecodeString(response.Choices[0].Message.Audio.Data) if err != nil { panic(err) } if err := os.WriteFile("dog.wav", audio, 0o600); err != nil { panic(err) } } ``` ```java import com.openai.client.OpenAIClient; import com.openai.client.okhttp.OpenAIOkHttpClient; import com.openai.models.chat.completions.ChatCompletionAudioParam; import com.openai.models.chat.completions.ChatCompletionCreateParams; import java.io.IOException; import java.nio.file.Files; import java.nio.file.Path; import java.util.Base64; ChatCompletionCreateParams params = ChatCompletionCreateParams.builder() .model("gpt-audio-1.5") .addUserMessage("Is a golden retriever a good family dog?") .addModality(ChatCompletionCreateParams.Modality.TEXT) .addModality(ChatCompletionCreateParams.Modality.AUDIO) .audio( ChatCompletionAudioParam.builder() .voice("alloy") .format(ChatCompletionAudioParam.Format.WAV) .build()) .store(true) .build(); var message = client.chat().completions().create(params).choices().get(0).message(); var audio = message.audio().orElseThrow(() -> new IllegalStateException("No audio output returned")); Files.write(Path.of("dog.wav"), Base64.getDecoder().decode(audio.data())); message.content().ifPresent(System.out::println); ``` ```csharp using OpenAI.Chat; #pragma warning disable OPENAI001 string key = Environment.GetEnvironmentVariable("OPENAI_API_KEY")!; ChatClient client = new("gpt-audio-1.5", key); ChatCompletionOptions options = new() { ResponseModalities = ChatResponseModalities.Text | ChatResponseModalities.Audio, AudioOptions = new(ChatOutputAudioVoice.Alloy, ChatOutputAudioFormat.Wav), StoredOutputEnabled = true, }; ChatCompletion completion = await client.CompleteChatAsync( [new UserChatMessage("Is a golden retriever a good family dog?")], options ); if (completion.OutputAudio is not ChatOutputAudio audio) { throw new InvalidOperationException("No audio output was returned."); } Console.WriteLine(audio.Transcript); await File.WriteAllBytesAsync("dog.wav", audio.AudioBytes.ToArray()); ``` ```ruby require "base64" require "openai" client = OpenAI::Client.new completion = client.chat.completions.create( model: "gpt-audio-1.5", messages: [ { role: :user, content: "Is a golden retriever a good family dog?" } ], modalities: [:text, :audio], audio: { voice: :alloy, format: :wav }, store: true ) audio = completion.choices.fetch(0).message.audio or raise "No audio returned" File.binwrite("dog.wav", Base64.strict_decode64(audio.data)) ``` ```bash curl "https://api.openai.com/v1/chat/completions" \ -H "Content-Type: application/json" \ -H "Authorization: Bearer $OPENAI_API_KEY" \ -d '{ "model": "gpt-audio-1.5", "modalities": ["text", "audio"], "audio": { "voice": "alloy", "format": "wav" }, "messages": [ { "role": "user", "content": "Is a golden retriever a good family dog?" } ] }' ``` Audio input to model Use audio inputs for prompting a model ```javascript import OpenAI from "openai"; const openai = new OpenAI(); // Fetch an audio file and convert it to a base64 string const url = "https://cdn.openai.com/API/docs/audio/alloy.wav"; const audioResponse = await fetch(url); const buffer = await audioResponse.arrayBuffer(); const base64str = Buffer.from(buffer).toString("base64"); const response = await openai.chat.completions.create({ model: "gpt-audio-1.5", modalities: ["text", "audio"], audio: { voice: "alloy", format: "wav" }, messages: [ { role: "user", content: [ { type: "text", text: "What is in this recording?" }, { type: "input_audio", input_audio: { data: base64str, format: "wav" }, }, ], }, ], store: true, }); console.log(response.choices[0]); ``` ```python import base64 import requests from openai import OpenAI client = OpenAI() # Fetch the audio file and convert it to a base64 encoded string url = "https://cdn.openai.com/API/docs/audio/alloy.wav" response = requests.get(url) response.raise_for_status() wav_data = response.content encoded_string = base64.b64encode(wav_data).decode("utf-8") completion = client.chat.completions.create( model="gpt-audio-1.5", modalities=["text", "audio"], audio={"voice": "alloy", "format": "wav"}, messages=[ { "role": "user", "content": [ {"type": "text", "text": "What is in this recording?"}, { "type": "input_audio", "input_audio": {"data": encoded_string, "format": "wav"}, }, ], }, ], ) print(completion.choices[0].message) ``` ```go package main import ( "context" "encoding/base64" "fmt" "os" "github.com/openai/openai-go/v3" ) func main() { audio, err := os.ReadFile("fixtures/audio.wav") if err != nil { panic(err) } client := openai.NewClient() response, err := client.Chat.Completions.New(context.Background(), openai.ChatCompletionNewParams{ Model: "gpt-audio-1.5", Modalities: []string{"text", "audio"}, Audio: openai.ChatCompletionAudioParam{ Voice: openai.ChatCompletionAudioParamVoiceUnion{OfString: openai.String("alloy")}, Format: openai.ChatCompletionAudioParamFormatWAV, }, Messages: []openai.ChatCompletionMessageParamUnion{openai.UserMessage([]openai.ChatCompletionContentPartUnionParam{ openai.TextContentPart("What is in this recording?"), openai.InputAudioContentPart(openai.ChatCompletionContentPartInputAudioInputAudioParam{ Data: base64.StdEncoding.EncodeToString(audio), Format: "wav", }), })}, }) if err != nil { panic(err) } fmt.Println(response.Choices[0]) } ``` ```java import com.openai.client.OpenAIClient; import com.openai.client.okhttp.OpenAIOkHttpClient; import com.openai.models.chat.completions.ChatCompletionAudioParam; import com.openai.models.chat.completions.ChatCompletionContentPart; import com.openai.models.chat.completions.ChatCompletionContentPartInputAudio; import com.openai.models.chat.completions.ChatCompletionContentPartText; import com.openai.models.chat.completions.ChatCompletionCreateParams; import com.openai.models.chat.completions.ChatCompletionUserMessageParam; import java.io.IOException; import java.nio.file.Files; import java.nio.file.Path; import java.util.Base64; import java.util.List; String encodedAudio = Base64.getEncoder() .encodeToString( Files.readAllBytes(Path.of(System.getenv("OPENAI_EXAMPLE_AUDIO_PATH")))); ChatCompletionCreateParams params = ChatCompletionCreateParams.builder() .model("gpt-audio-1.5") .addMessage( ChatCompletionUserMessageParam.builder() .contentOfArrayOfContentParts( List.of( ChatCompletionContentPart.ofText( ChatCompletionContentPartText.builder() .text("What is in this recording?") .build()), ChatCompletionContentPart.ofInputAudio( ChatCompletionContentPartInputAudio.builder() .inputAudio( ChatCompletionContentPartInputAudio.InputAudio.builder() .data(encodedAudio) .format( ChatCompletionContentPartInputAudio.InputAudio .Format.WAV) .build()) .build()))) .build()) .addModality(ChatCompletionCreateParams.Modality.TEXT) .addModality(ChatCompletionCreateParams.Modality.AUDIO) .audio( ChatCompletionAudioParam.builder() .voice("alloy") .format(ChatCompletionAudioParam.Format.WAV) .build()) .store(true) .build(); client.chat().completions().create(params).choices().stream() .flatMap(choice -> choice.message().content().stream()) .forEach(System.out::println); ``` ```csharp using OpenAI.Chat; #pragma warning disable OPENAI001 string key = Environment.GetEnvironmentVariable("OPENAI_API_KEY")!; ChatClient client = new("gpt-audio-1.5", key); BinaryData audio = BinaryData.FromBytes( await File.ReadAllBytesAsync("audio.wav") ); UserChatMessage message = new( [ ChatMessageContentPart.CreateTextPart("What is in this recording?"), ChatMessageContentPart.CreateInputAudioPart( audio, ChatInputAudioFormat.Wav ), ] ); ChatCompletionOptions options = new() { ResponseModalities = ChatResponseModalities.Text | ChatResponseModalities.Audio, AudioOptions = new(ChatOutputAudioVoice.Alloy, ChatOutputAudioFormat.Wav), StoredOutputEnabled = true, }; ChatCompletion completion = await client.CompleteChatAsync([message], options); if (completion.OutputAudio is not ChatOutputAudio audioOutput) { throw new InvalidOperationException("No audio output was returned."); } Console.WriteLine(audioOutput.Transcript); ``` ```ruby require "base64" require "openai" client = OpenAI::Client.new audio = Base64.strict_encode64(File.binread("audio.wav")) completion = client.chat.completions.create( model: "gpt-audio-1.5", messages: [ { role: :user, content: [ { type: :text, text: "What is in this recording?" }, { type: :input_audio, input_audio: { data: audio, format: :wav } } ] } ], modalities: [:text, :audio], audio: { voice: :alloy, format: :wav }, store: true ) puts(completion.choices.fetch(0).message.content) ``` ```bash curl "https://api.openai.com/v1/chat/completions" \ -H "Content-Type: application/json" \ -H "Authorization: Bearer $OPENAI_API_KEY" \ -d '{ "model": "gpt-audio-1.5", "modalities": ["text", "audio"], "audio": { "voice": "alloy", "format": "wav" }, "messages": [ { "role": "user", "content": [ { "type": "text", "text": "What is in this recording?" }, { "type": "input_audio", "input_audio": { "data": "", "format": "wav" } } ] } ] }' ``` --- # Background mode Agents like [Codex](https://openai.com/index/introducing-codex/) and [Deep Research](https://openai.com/index/introducing-deep-research/) show that reasoning models can take several minutes to solve complex problems. Background mode enables you to execute long-running tasks on models like GPT-5.2 and GPT-5.2 Pro reliably, without having to worry about timeouts or other connectivity issues. Background mode kicks off these tasks asynchronously, and developers can poll response objects to check status over time. To start response generation in the background, make an API request with `background` set to `true`: Background requests from Zero Data Retention (ZDR) projects run with `store=false`. Response data is temporarily stored to disk for roughly 10 minutes to enable asynchronous execution and polling. For projects using [Modified Abuse Monitoring](https://developers.openai.com/api/docs/guides/your-data#modified-abuse-monitoring), including enhanced Modified Abuse Monitoring, foreground requests follow standard retention when `store` is omitted or set to `true`. Background responses are retained after the polling period only when `store=true` is explicitly provided. If `store` is omitted or set to `false` for a background request, the response is deleted after roughly 10 minutes. Generate a response in the background ```bash curl https://api.openai.com/v1/responses \ -H "Content-Type: application/json" \ -H "Authorization: Bearer $OPENAI_API_KEY" \ -d '{ "model": "gpt-6-astra", "input": "Write a very long novel about otters in space.", "background": true }' ``` ```javascript import OpenAI from "openai"; const client = new OpenAI(); const resp = await client.responses.create({ model: "gpt-6-astra", input: "Write a very long novel about otters in space.", background: true, }); console.log(resp.status); ``` ```python from openai import OpenAI client = OpenAI() resp = client.responses.create( model="gpt-6-astra", input="Write a very long novel about otters in space.", background=True, ) print(resp.status) ``` ```go package main import ( "context" "fmt" "github.com/openai/openai-go/v3" "github.com/openai/openai-go/v3/responses" ) func main() { client := openai.NewClient() response, err := client.Responses.New(context.Background(), responses.ResponseNewParams{ Model: "gpt-6-astra", Background: openai.Bool(true), Input: responses.ResponseNewParamsInputUnion{ OfString: openai.String("Write a very long novel about otters in space."), }, }) if err != nil { panic(err) } fmt.Println(response.Status) } ``` ```java import com.openai.client.OpenAIClient; import com.openai.client.okhttp.OpenAIOkHttpClient; import com.openai.models.responses.ResponseCreateParams; ResponseCreateParams params = ResponseCreateParams.builder() .model("gpt-6-astra") .input("Write a detailed market analysis.") .background(true) .build(); var response = client.responses().create(params); System.out.println(response.status().orElseThrow()); ``` ```csharp using OpenAI.Responses; #pragma warning disable OPENAI001 string key = Environment.GetEnvironmentVariable("OPENAI_API_KEY")!; ResponsesClient client = new(key); CreateResponseOptions options = new() { Model = "gpt-6-astra", BackgroundModeEnabled = true, }; options.InputItems.Add( ResponseItem.CreateUserMessageItem("Write a very long novel about otters in space.") ); ResponseResult response = await client.CreateResponseAsync(options); Console.WriteLine(response.Status); ``` ```ruby require "openai" client = OpenAI::Client.new response = client.responses.create( model: "gpt-6-astra", input: "Write a detailed market analysis.", background: true ) puts(response.status) ``` ## Polling background responses To check the status of background requests, use the GET endpoint for Responses. Keep polling while the request is in the queued or in_progress state. When it leaves these states, it has reached a final (terminal) state. Retrieve a response executing in the background ```bash curl https://api.openai.com/v1/responses/resp_123 \ -H "Content-Type: application/json" \ -H "Authorization: Bearer $OPENAI_API_KEY" ``` ```javascript import OpenAI from "openai"; const client = new OpenAI(); let resp = await client.responses.create({ model: "gpt-6-astra", input: "Write a very long novel about otters in space.", background: true, }); while (resp.status === "queued" || resp.status === "in_progress") { console.log("Current status: " + resp.status); await new Promise((resolve) => setTimeout(resolve, 2000)); // wait 2 seconds resp = await client.responses.retrieve(resp.id); } console.log("Final status: " + resp.status + "\nOutput:\n" + resp.output_text); ``` ```python from openai import OpenAI from time import sleep client = OpenAI() resp = client.responses.create( model="gpt-6-astra", input="Write a very long novel about otters in space.", background=True, ) while resp.status in {"queued", "in_progress"}: print(f"Current status: {resp.status}") sleep(2) resp = client.responses.retrieve(resp.id) print(f"Final status: {resp.status}\nOutput:\n{resp.output_text}") ``` ```go package main import ( "context" "fmt" "time" "github.com/openai/openai-go/v3" "github.com/openai/openai-go/v3/responses" ) func main() { client := openai.NewClient() response, err := client.Responses.New(context.Background(), responses.ResponseNewParams{ Model: "gpt-6-astra", Background: openai.Bool(true), Input: responses.ResponseNewParamsInputUnion{ OfString: openai.String("Write a very long novel about otters in space."), }, }) if err != nil { panic(err) } for response.Status == "queued" || response.Status == "in_progress" { fmt.Println("Current status:", response.Status) time.Sleep(2 * time.Second) response, err = client.Responses.Get(context.Background(), response.ID, responses.ResponseGetParams{}) if err != nil { panic(err) } } fmt.Printf("Final status: %s\nOutput:\n%s\n", response.Status, response.OutputText()) } ``` ```java import com.openai.client.OpenAIClient; import com.openai.client.okhttp.OpenAIOkHttpClient; import com.openai.models.responses.ResponseCreateParams; import com.openai.models.responses.ResponseStatus; ResponseCreateParams params = ResponseCreateParams.builder() .model("gpt-6-astra") .input("Write a very long novel about otters in space.") .background(true) .build(); var response = client.responses().create(params); while (response.status().filter(ResponseStatus.QUEUED::equals).isPresent() || response.status().filter(ResponseStatus.IN_PROGRESS::equals).isPresent()) { System.out.println("Current status: " + response.status().orElseThrow()); Thread.sleep(1000); response = client.responses().retrieve(response.id()); } System.out.println("Final status: " + response.status().orElseThrow()); response.output().stream() .flatMap(item -> item.message().stream()) .flatMap(message -> message.content().stream()) .flatMap(content -> content.outputText().stream()) .forEach(text -> System.out.println(text.text())); ``` ```csharp using OpenAI.Responses; #pragma warning disable OPENAI001 string key = Environment.GetEnvironmentVariable("OPENAI_API_KEY")!; ResponsesClient client = new(key); CreateResponseOptions options = new() { Model = "gpt-6-astra", BackgroundModeEnabled = true, }; options.InputItems.Add( ResponseItem.CreateUserMessageItem("Write a very long novel about otters in space.") ); ResponseResult created = await client.CreateResponseAsync(options); ResponseResult response = await client.GetResponseAsync(created.Id); while (response.Status is ResponseStatus.Queued or ResponseStatus.InProgress) { await Task.Delay(TimeSpan.FromSeconds(1)); response = await client.GetResponseAsync(response.Id); } if (response.Status != ResponseStatus.Completed) { throw new InvalidOperationException($"Background response ended with status: {response.Status}"); } Console.WriteLine($"Status: {response.Status}"); Console.WriteLine(response.GetOutputText()); ``` ```ruby require "openai" client = OpenAI::Client.new response = client.responses.create( model: "gpt-6-astra", input: "Write a very long novel about otters in space.", background: true ) while [:queued, :in_progress].include?(response.status) puts("Current status: #{response.status}") sleep(2) response = client.responses.retrieve(response.id) end puts("Final status: #{response.status}") puts(response.output_text) ``` ## Cancelling a background response You can also cancel an in-flight response like this: Cancel an ongoing response ```bash curl -X POST https://api.openai.com/v1/responses/resp_123/cancel \ -H "Content-Type: application/json" \ -H "Authorization: Bearer $OPENAI_API_KEY" ``` ```javascript import OpenAI from "openai"; const client = new OpenAI(); const resp = await client.responses.cancel("resp_123"); console.log(resp.status); ``` ```python import os from openai import OpenAI response_id = os.environ["OPENAI_RESPONSE_ID"] client = OpenAI() resp = client.responses.cancel(response_id) print(resp.status) ``` ```go package main import ( "context" "fmt" "github.com/openai/openai-go/v3" ) func main() { client := openai.NewClient() canceled, err := client.Responses.Cancel(context.Background(), "resp_123") if err != nil { panic(err) } fmt.Println(canceled.Status) } ``` ```java import com.openai.client.OpenAIClient; import com.openai.client.okhttp.OpenAIOkHttpClient; String responseId = "resp_123"; var response = client.responses().cancel(responseId); System.out.println(response.status()); ``` ```csharp using OpenAI.Responses; #pragma warning disable OPENAI001 string key = Environment.GetEnvironmentVariable("OPENAI_API_KEY")!; ResponsesClient client = new(key); // Replace this illustrative ID with the background response to cancel. string responseId = "resp_123"; ResponseResult response = await client.CancelResponseAsync(responseId); Console.WriteLine(response.Status); ``` ```ruby require "openai" client = OpenAI::Client.new response = client.responses.cancel("resp_123") puts(response.status) ``` Cancelling twice is idempotent - subsequent calls simply return the final `Response` object. ## Streaming a background response You can create a background Response and start streaming events from it right away. This may be helpful if you expect the client to drop the stream and want the option of picking it back up later. To do this, create a Response with both `background` and `stream` set to `true`. You will want to keep track of a "cursor" corresponding to the `sequence_number` you receive in each streaming event. Currently, the time to first token you receive from a background response is higher than what you receive from a synchronous one. We are working to reduce this latency gap in the coming weeks. Generate and stream a background response ```bash curl https://api.openai.com/v1/responses \ -H "Content-Type: application/json" \ -H "Authorization: Bearer $OPENAI_API_KEY" \ -d '{ "model": "gpt-6-astra", "input": "Write a very long novel about otters in space.", "background": true, "stream": true }' // To resume: curl "https://api.openai.com/v1/responses/resp_123?stream=true&starting_after=42" \ -H "Content-Type: application/json" \ -H "Authorization: Bearer $OPENAI_API_KEY" ``` ```javascript import OpenAI from "openai"; const client = new OpenAI(); const stream = await client.responses.create({ model: "gpt-6-astra", input: "Write a very long novel about otters in space.", background: true, stream: true, }); let cursor = null; for await (const event of stream) { console.log(event); cursor = event.sequence_number; } // If the connection drops, you can resume streaming from the last cursor (SDK support coming soon): // const resumedStream = await client.responses.stream(resp.id, { starting_after: cursor }); // for await (const event of resumedStream) { ... } ``` ```python from openai import OpenAI client = OpenAI() # Fire off an async response but also start streaming immediately stream = client.responses.create( model="gpt-6-astra", input="Write a very long novel about otters in space.", background=True, stream=True, ) cursor = None for event in stream: print(event) cursor = event.sequence_number # If your connection drops, the response continues running and you can reconnect: # SDK support for resuming the stream is coming soon. # for event in client.responses.stream(resp.id, starting_after=cursor): # print(event) ``` ```go package main import ( "context" "fmt" "github.com/openai/openai-go/v3" "github.com/openai/openai-go/v3/responses" ) func main() { client := openai.NewClient() stream := client.Responses.NewStreaming(context.Background(), responses.ResponseNewParams{ Model: "gpt-6-astra", Background: openai.Bool(true), Input: responses.ResponseNewParamsInputUnion{ OfString: openai.String("Write a very long novel about otters in space."), }, }) var cursor int64 var responseID string for stream.Next() { event := stream.Current() fmt.Println(event.Type) cursor = event.SequenceNumber if event.Response.ID != "" { responseID = event.Response.ID } } if err := stream.Err(); err != nil { panic(err) } fmt.Printf("response %s last cursor %d\n", responseID, cursor) // If the connection drops, resume streaming from the last cursor: // resumed := client.Responses.GetStreaming( // context.Background(), // responseID, // responses.ResponseGetParams{StartingAfter: openai.Int(cursor)}, // ) // for resumed.Next() { // fmt.Println(resumed.Current().Type) // } } ``` ```java import com.fasterxml.jackson.databind.json.JsonMapper; import com.openai.client.OpenAIClient; import com.openai.client.okhttp.OpenAIOkHttpClient; import com.openai.core.http.StreamResponse; import com.openai.models.responses.ResponseCreateParams; import com.openai.models.responses.ResponseRetrieveParams; import com.openai.models.responses.ResponseStreamEvent; import java.util.concurrent.atomic.AtomicBoolean; import java.util.concurrent.atomic.AtomicLong; import java.util.concurrent.atomic.AtomicReference; ResponseCreateParams params = ResponseCreateParams.builder() .model("gpt-6-astra") .input("Write a very long novel about otters in space.") .background(true) .build(); AtomicLong lastSequenceNumber = new AtomicLong(-1); AtomicReference responseId = new AtomicReference<>(""); AtomicBoolean streamCompleted = new AtomicBoolean(false); JsonMapper json = new JsonMapper(); try (StreamResponse stream = client.responses().createStreaming(params)) { stream.stream() .forEach( event -> { lastSequenceNumber.set(json.valueToTree(event).path("sequence_number").asLong()); event .created() .ifPresent( created -> { responseId.set(created.response().id()); System.out.println("response.created"); }); event .outputTextDelta() .ifPresent( delta -> { System.out.println("response.output_text.delta"); }); event .completed() .ifPresent( completed -> { streamCompleted.set(true); System.out.println("response.completed"); }); }); } System.out.println( "Response " + responseId.get() + "; last sequence number " + lastSequenceNumber.get()); if (!streamCompleted.get()) { try (StreamResponse resumed = client .responses() .retrieveStreaming( ResponseRetrieveParams.builder() .responseId(responseId.get()) .startingAfter(lastSequenceNumber.get()) .build())) { resumed.stream() .forEach( event -> event.outputTextDelta().ifPresent(delta -> System.out.println(delta.delta()))); } } ``` ```csharp using OpenAI.Responses; #pragma warning disable OPENAI001 string key = Environment.GetEnvironmentVariable("OPENAI_API_KEY")!; ResponsesClient client = new(key); CreateResponseOptions options = new() { Model = "gpt-6-astra", BackgroundModeEnabled = true, StreamingEnabled = true, }; options.InputItems.Add( ResponseItem.CreateUserMessageItem("Write a very long novel about otters in space.") ); string? responseId = null; int lastSequenceNumber = -1; bool completed = false; void HandleUpdate(StreamingResponseUpdate update) { lastSequenceNumber = update.SequenceNumber; switch (update) { case StreamingResponseCreatedUpdate created: responseId = created.Response.Id; break; case StreamingResponseOutputTextDeltaUpdate text: Console.Write(text.Delta); break; case StreamingResponseCompletedUpdate: completed = true; break; case StreamingResponseFailedUpdate: throw new InvalidOperationException("The background response failed."); case StreamingResponseIncompleteUpdate: throw new InvalidOperationException("The background response was incomplete."); case StreamingResponseErrorUpdate error: throw new InvalidOperationException($"The response stream failed: {error.Message}"); } } try { await foreach ( StreamingResponseUpdate update in client.CreateResponseStreamingAsync(options) ) { HandleUpdate(update); } } catch (Exception error) when (error is HttpRequestException or IOException && responseId is not null) { // The background response continues after its streaming connection is interrupted. } if (!completed) { if (responseId is null) { throw new InvalidOperationException("The response stream ended before providing its ID."); } GetResponseOptions resumeOptions = new(responseId) { StartingAfter = lastSequenceNumber, StreamingEnabled = true, }; await foreach (StreamingResponseUpdate update in client.GetResponseStreamingAsync(resumeOptions)) { HandleUpdate(update); } if (!completed) { throw new InvalidOperationException( "The resumed response stream ended before the background response completed." ); } } ``` ```ruby require "openai" client = OpenAI::Client.new stream = client.responses.stream( model: "gpt-6-astra", input: "Write a very long novel about otters in space.", background: true ) last_sequence_number = -1 response_id = "" stream.each do |event| puts(event.type) last_sequence_number = event.sequence_number || last_sequence_number if event.is_a?(OpenAI::Models::Responses::ResponseCreatedEvent) response_id = event.response.id end end puts("Response #{response_id}; last sequence number #{last_sequence_number}") # If the connection drops, resume from the last sequence number: # client.responses.stream(response_id: response_id, starting_after: last_sequence_number).each do |event| # puts(event.type) # end ``` ## Limits 1. Background requests can use `store=false`, but response data is temporarily stored to support asynchronous execution and polling. 2. To cancel a synchronous response, terminate the connection 3. You can only start a new stream from a background response if you created it with `stream=true`. --- # Batch API Learn how to use OpenAI's Batch API to send asynchronous groups of requests with 50% lower costs, a separate pool of significantly higher rate limits, and a clear 24-hour turnaround time. The service is ideal for processing jobs that don't require immediate responses. You can also [explore the API reference directly here](https://developers.openai.com/api/reference/resources/batches). ## Overview While some uses of the OpenAI Platform require you to send synchronous requests, there are many cases where requests do not need an immediate response or [rate limits](https://developers.openai.com/api/docs/guides/rate-limits) prevent you from executing a large number of queries quickly. Batch processing jobs are often helpful in use cases like: 1. Running evaluations 2. Classifying large datasets 3. Embedding content repositories 4. Queuing large offline video-render jobs The Batch API offers a straightforward set of endpoints that allow you to collect a set of requests into a single file, kick off a batch processing job to execute these requests, query for the status of that batch while the underlying requests execute, and eventually retrieve the collected results when the batch is complete. Compared to using standard endpoints directly, Batch API has: 1. **Better cost efficiency:** 50% cost discount compared to synchronous APIs 2. **Higher rate limits:** [Substantially more headroom](https://platform.openai.com/settings/organization/limits) compared to the synchronous APIs 3. **Fast completion times:** Each batch completes within 24 hours (and often more quickly) ## Getting started ### 1. Prepare your batch file Batches start with a `.jsonl` file where each line contains the details of an individual request to the API. For now, the available endpoints are: - `/v1/responses` ([Responses API](https://developers.openai.com/api/reference/resources/responses)) - `/v1/chat/completions` ([Chat Completions API](https://developers.openai.com/api/reference/resources/chat)) - `/v1/embeddings` ([Embeddings API](https://developers.openai.com/api/reference/resources/embeddings)) - `/v1/completions` ([Completions API](https://developers.openai.com/api/reference/resources/completions)) - `/v1/moderations` ([Moderation guide](https://developers.openai.com/api/docs/guides/moderation)) - `/v1/images/generations` ([Images API](https://developers.openai.com/api/reference/resources/images)) - `/v1/images/edits` ([Images API](https://developers.openai.com/api/reference/resources/images)) - `/v1/videos` ([Video generation guide](https://developers.openai.com/api/docs/guides/video-generation)) For a given input file, the parameters in each line's `body` field are the same as the parameters for the underlying endpoint. Each request must include a unique `custom_id` value, which you can use to reference results after completion. Here's an example of an input file with 2 requests. Note that each input file can only include requests to a single model. For video generation in Batch: - Batch currently supports `POST /v1/videos` only. - Batch requests for videos must use JSON, not multipart. - Upload assets ahead of time and pass supported asset references in the request body rather than using multipart uploads. - Use `input_reference` for image-guided generations in Batch. In JSON requests, pass `input_reference` as an object with either `file_id` or `image_url`. - Multipart `input_reference` uploads, including video reference inputs, aren't supported in Batch. - Batch-generated videos are available for download for up to `24` hours after the batch completes. When targeting `/v1/moderations`, include an `input` field in every request body. Batch accepts plain-text inputs and content arrays with text or image inputs using `omni-moderation-latest`. The Batch worker rejects requests that set `stream=true`, matching the synchronous moderation endpoint. ```jsonl {"custom_id": "request-1", "method": "POST", "url": "/v1/chat/completions", "body": {"model": "gpt-3.5-turbo-0125", "messages": [{"role": "system", "content": "You are a helpful assistant."},{"role": "user", "content": "Hello world!"}],"max_tokens": 1000}} {"custom_id": "request-2", "method": "POST", "url": "/v1/chat/completions", "body": {"model": "gpt-3.5-turbo-0125", "messages": [{"role": "system", "content": "You are an unhelpful assistant."},{"role": "user", "content": "Hello world!"}],"max_tokens": 1000}} ``` #### Moderation input examples Text-only request: ```jsonl { "custom_id": "moderation-text-1", "method": "POST", "url": "/v1/moderations", "body": { "model": "omni-moderation-latest", "input": "This is a harmless test sentence." } } ``` Request with text and image input: ```jsonl { "custom_id": "moderation-mm-1", "method": "POST", "url": "/v1/moderations", "body": { "model": "omni-moderation-latest", "input": [ { "type": "text", "text": "Describe this image" }, { "type": "image_url", "image_url": { "url": "https://api.nga.gov/iiif/a2e6da57-3cd1-4235-b20e-95dcaefed6c8/full/!800,800/0/default.jpg" } } ] } } ``` Prefer referencing remote assets with `image_url` (instead of base64 blobs) to keep your `.jsonl` files well below the 200 MB Batch upload limit, especially for multimodal Moderations requests. ### 2. Upload your batch input file Similar to our [Fine-tuning API](https://developers.openai.com/api/docs/guides/model-optimization), you must first upload your input file so that you can reference it correctly when kicking off batches. Upload your `.jsonl` file using the [Files API](https://developers.openai.com/api/reference/resources/files). Upload files for Batch API ```javascript import fs from "fs"; import OpenAI from "openai"; const openai = new OpenAI(); const file = await openai.files.create({ file: fs.createReadStream("fixtures/batchinput.jsonl"), purpose: "batch", }); console.log(file); ``` ```python from openai import OpenAI client = OpenAI() batch_input_file = client.files.create( file=open("batchinput.jsonl", "rb"), purpose="batch" ) print(batch_input_file) ``` ```go package main import ( "context" "fmt" "os" "github.com/openai/openai-go/v3" ) func main() { client := openai.NewClient() file, err := os.Open("batchinput.jsonl") if err != nil { panic(err) } defer file.Close() uploaded, err := client.Files.New(context.Background(), openai.FileNewParams{ File: file, Purpose: openai.FilePurposeBatch, }) if err != nil { panic(err) } fmt.Println(uploaded.ID) } ``` ```java import com.openai.client.OpenAIClient; import com.openai.client.okhttp.OpenAIOkHttpClient; import com.openai.models.files.FileCreateParams; import com.openai.models.files.FilePurpose; import java.nio.file.Path; var file = client .files() .create( FileCreateParams.builder() .file(Path.of(System.getenv("OPENAI_EXAMPLE_FILE_PATH"))) .purpose(FilePurpose.BATCH) .build()); System.out.println(file.id()); ``` ```ruby require "openai" require "pathname" client = OpenAI::Client.new file = Pathname("batchinput.jsonl") uploaded = client.files.create(file: file, purpose: :batch) puts(uploaded.id) ``` ```bash curl https://api.openai.com/v1/files \ -H "Authorization: Bearer $OPENAI_API_KEY" \ -F purpose="batch" \ -F file="@batchinput.jsonl" ``` ```bash openai files create \ --file batchinput.jsonl \ --purpose batch ``` ### 3. Create the batch Once you've successfully uploaded your input file, you can use the input File object's ID to create a batch. In this case, let's assume the file ID is `file-abc123`. For now, the completion window can only be set to `24h`. You can also provide custom metadata via an optional `metadata` parameter. Create the Batch ```javascript import OpenAI from "openai"; const openai = new OpenAI(); const batch = await openai.batches.create({ input_file_id: "file-abc123", endpoint: "/v1/chat/completions", completion_window: "24h", }); console.log(batch); ``` ```python batch = client.batches.create( input_file_id=batch_input_file.id, endpoint="/v1/chat/completions", completion_window="24h", metadata={"description": "nightly eval job"}, ) print(batch) ``` ```go package main import ( "context" "fmt" "github.com/openai/openai-go/v3" ) func main() { client := openai.NewClient() batch, err := client.Batches.New(context.Background(), openai.BatchNewParams{ InputFileID: "file-abc123", Endpoint: openai.BatchNewParamsEndpointV1ChatCompletions, CompletionWindow: openai.BatchNewParamsCompletionWindow24h, }) if err != nil { panic(err) } fmt.Println(batch.ID) } ``` ```java import com.openai.client.OpenAIClient; import com.openai.client.okhttp.OpenAIOkHttpClient; import com.openai.models.batches.BatchCreateParams; String fileId = "file-abc123"; var batch = client .batches() .create( BatchCreateParams.builder() .inputFileId(fileId) .endpoint(BatchCreateParams.Endpoint.V1_RESPONSES) .completionWindow(BatchCreateParams.CompletionWindow._24H) .build()); System.out.println(batch.id()); ``` ```ruby require "openai" client = OpenAI::Client.new batch = client.batches.create(input_file_id: "file-abc123", endpoint: "/v1/responses", completion_window: "24h") puts(batch.id) ``` ```bash curl https://api.openai.com/v1/batches \ -H "Authorization: Bearer $OPENAI_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "input_file_id": "file-abc123", "endpoint": "/v1/chat/completions", "completion_window": "24h" }' ``` ```bash openai batches create \ --input-file-id file-abc123 \ --endpoint /v1/chat/completions \ --completion-window 24h ``` This request will return a [Batch object](https://developers.openai.com/api/reference/resources/batches) with metadata about your batch: ```json { "id": "batch_abc123", "object": "batch", "endpoint": "/v1/chat/completions", "errors": null, "input_file_id": "file-abc123", "completion_window": "24h", "status": "validating", "output_file_id": null, "error_file_id": null, "created_at": 1714508499, "in_progress_at": null, "expires_at": 1714536634, "completed_at": null, "failed_at": null, "expired_at": null, "request_counts": { "total": 0, "completed": 0, "failed": 0 }, "metadata": null } ``` ### 4. Check the status of a batch You can check the status of a batch at any time, which will also return a Batch object. Check the status of a batch ```javascript import OpenAI from "openai"; const openai = new OpenAI(); const batch = await openai.batches.retrieve("batch_abc123"); console.log(batch); ``` ```python batch = client.batches.retrieve(batch.id) print(batch) ``` ```go package main import ( "context" "fmt" "github.com/openai/openai-go/v3" ) func main() { client := openai.NewClient() batch, err := client.Batches.Get(context.Background(), "batch_abc123") if err != nil { panic(err) } fmt.Println(batch.Status) } ``` ```java import com.openai.client.OpenAIClient; import com.openai.client.okhttp.OpenAIOkHttpClient; String batchId = "batch_abc123"; var batch = client.batches().retrieve(batchId); System.out.println(batch.status()); ``` ```ruby require "openai" client = OpenAI::Client.new batch = client.batches.retrieve("batch_abc123") puts(batch.status) ``` ```bash curl https://api.openai.com/v1/batches/batch_abc123 \ -H "Authorization: Bearer $OPENAI_API_KEY" \ -H "Content-Type: application/json" ``` ```bash openai batches retrieve \ --batch-id batch_abc123 ``` The status of a given Batch object can be any of the following: | Status | Description | | ------------- | ------------------------------------------------------------------------------ | | `validating` | the input file is being validated before the batch can begin | | `failed` | the input file has failed the validation process | | `in_progress` | the input file was successfully validated and the batch is currently being run | | `finalizing` | the batch has completed and the results are being prepared | | `completed` | the batch has been completed and the results are ready | | `expired` | the batch was not able to be completed within the 24-hour time window | | `cancelling` | the batch is being cancelled (may take up to 10 minutes) | | `cancelled` | the batch was cancelled | ### 5. Retrieve the results Once the batch is complete, you can download the output by making a request against the [Files API](https://developers.openai.com/api/reference/resources/files) via the `output_file_id` field from the Batch object and writing it to a file on your machine, in this case `batch_output.jsonl` Retrieving the batch results ```javascript import OpenAI from "openai"; const openai = new OpenAI(); const fileResponse = await openai.files.content("file-xyz123"); const fileContents = await fileResponse.text(); console.log(fileContents); ``` ```python # Replace the illustrative IDs and URLs below with your own resource values. from openai import OpenAI output_file_id = "file_123" client = OpenAI() file_response = client.files.content(output_file_id) print(file_response.text) ``` ```go package main import ( "context" "fmt" "io" "github.com/openai/openai-go/v3" ) func main() { client := openai.NewClient() response, err := client.Files.Content(context.Background(), "file-xyz123") if err != nil { panic(err) } defer response.Body.Close() contents, err := io.ReadAll(response.Body) if err != nil { panic(err) } fmt.Println(string(contents)) } ``` ```java import com.openai.client.OpenAIClient; import com.openai.client.okhttp.OpenAIOkHttpClient; import com.openai.core.http.HttpResponse; import java.io.IOException; import java.nio.file.Files; import java.nio.file.Path; import java.nio.file.StandardCopyOption; String fileId = "file-xyz123"; try (HttpResponse content = client.files().content(fileId)) { Files.copy( content.body(), Path.of("batch_output.jsonl"), StandardCopyOption.REPLACE_EXISTING); } ``` ```ruby require "openai" client = OpenAI::Client.new content = client.files.content("file-xyz123") puts(content.read) ``` ```bash curl https://api.openai.com/v1/files/file-xyz123/content \ -H "Authorization: Bearer $OPENAI_API_KEY" > batch_output.jsonl ``` ```bash openai files content \ --file-id file-xyz123 \ --output batch_output.jsonl ``` The output `.jsonl` file will have one response line for every successful request line in the input file. Any failed requests in the batch will have their error information written to an error file that can be found via the batch's `error_file_id`. For `/v1/videos`, a completed batch result contains video objects that have already reached a terminal state such as `completed`, `failed`, or `expired`. You can use the returned video IDs to download final assets immediately after the batch finishes. Note that the output line order **may not match** the input line order. Instead of relying on order to process your results, use the custom_id field which will be present in each line of your output file and allow you to map requests in your input to results in your output. ```jsonl {"id": "batch_req_123", "custom_id": "request-2", "response": {"status_code": 200, "request_id": "req_123", "body": {"id": "chatcmpl-123", "object": "chat.completion", "created": 1711652795, "model": "gpt-3.5-turbo-0125", "choices": [{"index": 0, "message": {"role": "assistant", "content": "Hello."}, "logprobs": null, "finish_reason": "stop"}], "usage": {"prompt_tokens": 22, "completion_tokens": 2, "total_tokens": 24}, "system_fingerprint": "fp_123"}}, "error": null} {"id": "batch_req_456", "custom_id": "request-1", "response": {"status_code": 200, "request_id": "req_789", "body": {"id": "chatcmpl-abc", "object": "chat.completion", "created": 1711652789, "model": "gpt-3.5-turbo-0125", "choices": [{"index": 0, "message": {"role": "assistant", "content": "Hello! How can I assist you today?"}, "logprobs": null, "finish_reason": "stop"}], "usage": {"prompt_tokens": 20, "completion_tokens": 9, "total_tokens": 29}, "system_fingerprint": "fp_3ba"}}, "error": null} ``` The output file will automatically be deleted 30 days after the batch is complete. ### 6. Cancel a batch If necessary, you can cancel an ongoing batch. The batch's status will change to `cancelling` until in-flight requests are complete (up to 10 minutes), after which the status will change to `cancelled`. Cancelling a batch ```javascript import OpenAI from "openai"; const openai = new OpenAI(); const batch = await openai.batches.cancel("batch_abc123"); console.log(batch); ``` ```python # Replace the illustrative IDs and URLs below with your own resource values. from openai import OpenAI batch_id = "batch_123" client = OpenAI() client.batches.cancel(batch_id) ``` ```go package main import ( "context" "fmt" "github.com/openai/openai-go/v3" ) func main() { client := openai.NewClient() batch, err := client.Batches.Cancel(context.Background(), "batch_abc123") if err != nil { panic(err) } fmt.Println(batch.Status) } ``` ```java import com.openai.client.OpenAIClient; import com.openai.client.okhttp.OpenAIOkHttpClient; String batchId = "batch_abc123"; System.out.println(client.batches().cancel(batchId).status()); ``` ```ruby require "openai" client = OpenAI::Client.new batch = client.batches.cancel("batch_abc123") puts(batch.status) ``` ```bash curl https://api.openai.com/v1/batches/batch_abc123/cancel \ -H "Authorization: Bearer $OPENAI_API_KEY" \ -H "Content-Type: application/json" \ -X POST ``` ```bash openai batches cancel \ --batch-id batch_abc123 ``` ### 7. Get a list of all batches At any time, you can see all your batches. For users with many batches, you can use the `limit` and `after` parameters to paginate your results. Getting a list of all batches ```javascript import OpenAI from "openai"; const openai = new OpenAI(); const list = await openai.batches.list(); for await (const batch of list) { console.log(batch); } ``` ```python from openai import OpenAI client = OpenAI() client.batches.list(limit=10) ``` ```go package main import ( "context" "fmt" "github.com/openai/openai-go/v3" ) func main() { client := openai.NewClient() list := client.Batches.ListAutoPaging(context.Background(), openai.BatchListParams{Limit: openai.Int(10)}) for list.Next() { fmt.Println(list.Current().ID) } if err := list.Err(); err != nil { panic(err) } } ``` ```java import com.openai.client.OpenAIClient; import com.openai.client.okhttp.OpenAIOkHttpClient; import com.openai.models.batches.BatchListParams; client .batches() .list(BatchListParams.builder().limit(10).build()) .autoPager() .forEach(batch -> System.out.println(batch.id())); ``` ```ruby require "openai" client = OpenAI::Client.new client.batches.list(limit: 10).auto_paging_each do |batch| puts(batch.id) end ``` ```bash curl https://api.openai.com/v1/batches?limit=10 \ -H "Authorization: Bearer $OPENAI_API_KEY" \ -H "Content-Type: application/json" ``` ```bash openai batches list \ --limit 10 ``` ## Model availability The Batch API is widely available across most of our models, but not all. Please refer to the [model reference docs](https://developers.openai.com/api/docs/models) to ensure the model you're using supports the Batch API. ## Rate limits Batch API rate limits are separate from existing per-model rate limits. The Batch API has three types of rate limits: 1. **Per-batch limits:** A single batch may include up to 50,000 requests, and a batch input file can be up to 200 MB in size. Note that `/v1/embeddings` batches are also restricted to a maximum of 50,000 embedding inputs across all requests in the batch. 2. **Queued prompt tokens per model:** Each model has a maximum number of prompt tokens that can be queued for batch processing. You can find these limits on the [Platform Settings page](https://platform.openai.com/settings/organization/limits). 3. **Batch creation rate limit:** You can create up to 2,000 batches per hour. If you need to submit more requests, increase the number of requests per batch. The Batch API currently has no output-token limit. Because Batch API rate limits are a new, separate pool, **using the Batch API will not consume tokens from your standard per-model rate limits**, thereby offering you a convenient way to increase the number of requests and processed tokens you can use when querying our API. ## Batch expiration Batches that do not complete in time eventually move to an `expired` state; unfinished requests within that batch are cancelled, and any responses to completed requests are made available via the batch's output file. You will be charged for tokens consumed from any completed requests. Expired requests will be written to your error file with the message as shown below. You can use the `custom_id` to retrieve the request data for expired requests. ```jsonl {"id": "batch_req_123", "custom_id": "request-3", "response": null, "error": {"code": "batch_expired", "message": "This request could not be executed before the completion window expired."}} {"id": "batch_req_123", "custom_id": "request-7", "response": null, "error": {"code": "batch_expired", "message": "This request could not be executed before the completion window expired."}} ``` --- # Blaxel See the [application-managed](https://github.com/openai/openai-cookbook/tree/main/examples/agents_api/sandboxes/application_managed/blaxel) and [webhook-managed](https://github.com/openai/openai-cookbook/tree/main/examples/agents_api/sandboxes/webhook_managed/blaxel) examples in the OpenAI Cookbook. See [Self-hosted sandboxes](https://developers.openai.com/api/docs/guides/agents-api/environments/self-hosted) for executor setup and connection requirements. Choose a provisioning mode: - **[Application-managed](#before-you-begin):** Follow this guide to start and stop sandboxes from your application. - **[Webhook-managed](https://developers.openai.com/api/docs/guides/agents-api/environments/lifecycle#set-up-webhook-managed-sandboxes):** Deploy a handler that starts or reconnects sandboxes from OpenAI webhooks. See [Sandbox lifecycle](https://developers.openai.com/api/docs/guides/agents-api/environments/lifecycle) to compare the two modes. ## Before you begin You need an OpenAI project API key, a Blaxel API key and workspace, and the Codex CLI package. Set `OPENAI_API_KEY`, a separate restricted `OPENAI_EXECUTOR_API_KEY`, `BL_API_KEY`, and `BL_WORKSPACE` in your environment. Grant the application key `api.agents.read` and `api.agents.write` for session operations, plus `api.responses.write` for model inference. Add `api.vaults.read` and `api.vaults.write` if your application manages vaults. Create the executor's [environment key](https://developers.openai.com/api/docs/guides/agents-api/environments/self-hosted#authentication) and use the same organization, project, and user or service account for both keys. Only the restricted executor key enters the sandbox. Choose the sandbox region in your provisioning code. Use `us-was-1` if you need the Agent Drive persistence option below. ## 1. Set up the Blaxel environment Create a [self-hosted session](https://developers.openai.com/api/docs/guides/agents-api/environments/self-hosted#create-or-reuse-a-session) and save its environment ID. Use the Blaxel SDK or API to create an isolated sandbox with the configured working directory. Install the Codex CLI in the sandbox, then [start its executor](https://developers.openai.com/api/docs/guides/agents-api/environments/self-hosted#start-the-executor) with that environment ID and the restricted executor key. The Blaxel Node image uses Alpine Linux, so install `ripgrep` with `apk`. Pass the restricted executor key as `CODEX_API_KEY` only to the executor process. Set `keep_alive=True` to prevent the sandbox from scaling to zero while the executor runs. Bounded setup, executor, and sandbox timeouts prevent abandoned resources from running indefinitely. For regular use, build a Blaxel image with Codex and `ripgrep` already installed so the sandbox can connect sooner. ## 2. Run the session Use the HTTP examples in [Run and continue sessions](https://developers.openai.com/api/docs/guides/agents-api/sessions) to send input and stream the result after the Blaxel executor connects. When finished, [delete the session](https://developers.openai.com/api/docs/guides/agents-api/sessions/manage#delete-a-session) and stop the provider sandbox separately. Start the sandbox before submitting input. The turn waits for the environment to connect, and the session reports the connection through `agent.session.environment.connected` on the event stream. Use `agent.session.turn.completed` to identify a successful turn. A failed or cancelled turn can also be followed by `agent.session.idle`, so do not treat an idle session as proof that the turn succeeded. ## Optional: Persist files between sessions Use [Blaxel Agent Drive](https://docs.blaxel.ai/Agent-drive/Overview) to preserve files across sandboxes and sessions. Mount the same drive in each sandbox to share files; Agent Drive requires the `us-was-1` region and does not transfer conversation history or session state. ## References - Read [Blaxel Sandbox documentation](https://docs.blaxel.ai/Sandboxes/Overview) - Read [Blaxel Python SDK](https://docs.blaxel.ai/sdk-reference/sdk-python) - Read [Blaxel TypeScript SDK](https://docs.blaxel.ai/sdk-reference/sdk-ts) --- # Building MCP servers for plugins and API integrations [Model Context Protocol](https://modelcontextprotocol.io/introduction) (MCP) is an open protocol that's becoming the industry standard for extending AI models with additional tools and knowledge. Remote MCP servers can be used to connect models over the Internet to new data sources and capabilities. In this guide, we'll cover how to build a remote MCP server that reads data from a private data source (a [vector store](https://developers.openai.com/api/docs/guides/retrieval)) and makes it available through a plugin in ChatGPT and Codex, through ChatGPT deep research and company knowledge, and [through the API](https://developers.openai.com/api/docs/guides/deep-research). **Note**: To build a plugin with an MCP server, start with the plugin docs: [Quickstart](https://developers.openai.com/plugins/quickstart), [Build your MCP server](https://developers.openai.com/plugins/build/mcp-server), [Connect and test your plugin](https://developers.openai.com/plugins/deploy/connect-chatgpt), and [Authentication](https://developers.openai.com/plugins/build/auth). If your MCP server doesn't need UI, you can expose tools without UI resources. ## Configure a data source You can use data from any source to power a remote MCP server, but for simplicity, we will use [vector stores](https://developers.openai.com/api/docs/guides/retrieval) in the OpenAI API. Begin by uploading a PDF document to a new vector store - [you can use this public domain 19th century book about cats](https://cdn.openai.com/API/docs/cats.pdf) for an example. You can upload files and create a vector store [in the dashboard here](https://platform.openai.com/storage/vector_stores), or you can create vector stores and upload files via API. [Follow the vector store guide](https://developers.openai.com/api/docs/guides/retrieval) to set up a vector store and upload a file to it. Make a note of the vector store's unique ID to use in the example to follow. ![vector store configuration](https://cdn.openai.com/API/docs/images/vector_store.png) ## Create an MCP server Next, let's create a remote MCP server that will do search queries against our vector store, and be able to return document content for files with a given ID. In this example, we are going to build our MCP server using Python and [FastMCP](https://github.com/jlowin/fastmcp). A full implementation of the server appears at the end of this section, along with instructions for running it in a [browser-based development environment](https://replit.com/). Note that there are a number of other MCP server frameworks you can use in a variety of programming languages. Whichever framework you use though, the tool definitions in your server will need to conform to the shape described here. To work with ChatGPT deep research and company knowledge, your MCP server should implement two read-only tools: `search` and `fetch`, using the compatibility schema in [Company knowledge compatibility](https://developers.openai.com/plugins/build/mcp-server#company-knowledge-compatibility). The same interface is useful for research workflows via API. Declare an output schema for each tool so clients can validate the result shape. In FastMCP, typed return models can generate this schema automatically; the example below passes `output_schema` explicitly from the same models. ### `search` tool The `search` tool is responsible for returning a list of relevant search results from your MCP server's data source, given a user's query. _Arguments:_ A single query string. _Returns:_ An object with a single key, `results`, whose value is an array of result objects. Each result object should include: - `id` - a unique ID for the document or search result item - `title` - human-readable title. - `url` - canonical URL for citation. In MCP, return this object as `structuredContent` and include the same value as a JSON-encoded string in the [content array](https://modelcontextprotocol.io/docs/learn/architecture#understanding-the-tool-execution-response) for compatibility. The final tool response should look like: ```json { "structuredContent": { "results": [{ "id": "doc-1", "title": "...", "url": "..." }] }, "content": [ { "type": "text", "text": "{\"results\":[{\"id\":\"doc-1\",\"title\":\"...\",\"url\":\"...\"}]}" } ] } ``` ### `fetch` tool The fetch tool is used to retrieve the full contents of a search result document or item. _Arguments:_ A string which is a unique identifier for the search document. _Returns:_ A single object with the following properties: - `id` - a unique ID for the document or search result item - `title` - a string title for the search result item - `text` - The full text of the document or item - `url` - a URL to the document or search result item. Useful for citing specific resources in research. - `metadata` - an optional key/value pairing of data about the result In MCP, return this object as `structuredContent` and include the same value as a JSON-encoded string in the content array for compatibility. The final tool response should look like: ```json { "structuredContent": { "id": "doc-1", "title": "...", "text": "full text...", "url": "https://example.com/doc", "metadata": { "source": "vector_store" } }, "content": [ { "type": "text", "text": "{\"id\":\"doc-1\",\"title\":\"...\",\"text\":\"full text...\",\"url\":\"https://example.com/doc\",\"metadata\":{\"source\":\"vector_store\"}}" } ] } ``` ### Citation behavior For both `search` results and `fetch` responses, ChatGPT creates citation metadata only when `url` is a non-empty string. A result with a `title` but no usable `url` remains ordinary tool output instead of becoming an empty citation. To make a result citable, return its canonical `url`. For example, ChatGPT might call `search` with: ```json { "query": "What is the quarterly plan?" } ``` The MCP server can respond with a URL-backed result: ```json { "structuredContent": { "results": [ { "id": "quarterly-plan", "title": "Quarterly plan", "url": "https://example.com/quarterly-plan" } ] }, "content": [ { "type": "text", "text": "{\"results\":[{\"id\":\"quarterly-plan\",\"title\":\"Quarterly plan\",\"url\":\"https://example.com/quarterly-plan\"}]}" } ] } ``` In this response, the `url` field has a value, which makes the result eligible for citation metadata. The query itself does not trigger citation handling. If the result omits `url`, or provides an empty or non-string value, ChatGPT preserves the result as ordinary tool output. ### Server example You can try this example MCP server in a [browser-based development environment](https://replit.com/). Configure the sample with your own API credentials and vector store information. [Example MCP server on Replit Remix the server example on Replit to test live.](https://replit.com/@kwhinnery-oai/DeepResearchServer?v=1#README.md) A full implementation of both the `search` and `fetch` tools in FastMCP is below also for convenience. #### Full implementation - FastMCP server ```python # Replace the illustrative IDs and URLs below with your own resource values. """ Sample MCP Server for ChatGPT Integration This server implements the Model Context Protocol (MCP) with search and fetch capabilities designed to work with ChatGPT's chat and deep research features. """ import logging import os from typing import Any from fastmcp import FastMCP from openai import OpenAI from pydantic import BaseModel class SearchResult(BaseModel): id: str title: str url: str class SearchOutput(BaseModel): results: list[SearchResult] class FetchOutput(BaseModel): id: str title: str text: str url: str metadata: dict[str, Any] | None = None # Configure logging logging.basicConfig(level=logging.INFO) logger = logging.getLogger(__name__) # OpenAI configuration OPENAI_API_KEY = os.environ["OPENAI_API_KEY"] VECTOR_STORE_ID = "vs_123" # Initialize OpenAI client openai_client = OpenAI(api_key=OPENAI_API_KEY) server_instructions = """ This MCP server provides search and document retrieval capabilities for ChatGPT Apps and deep research. Use the search tool to find relevant documents based on keywords, then use the fetch tool to retrieve complete document content with citations. """ def create_server(): """Create and configure the MCP server with search and fetch tools.""" # Initialize the FastMCP server mcp = FastMCP(name="Sample MCP Server", instructions=server_instructions) @mcp.tool(output_schema=SearchOutput.model_json_schema()) async def search(query: str) -> SearchOutput: """ Search for documents using OpenAI Vector Store search. This tool searches through the vector store to find semantically relevant matches. Returns a list of search results with basic information. Use the fetch tool to get complete document content. Args: query: Search query string. Natural language queries work best for semantic search. Returns: Dictionary with 'results' key containing list of matching documents. Each result includes id, title, and URL. """ if not query or not query.strip(): return SearchOutput(results=[]) if not openai_client: logger.error("OpenAI client not initialized - API key missing") raise ValueError("OpenAI API key is required for vector store search") # Search the vector store using OpenAI API logger.info(f"Searching {VECTOR_STORE_ID} for query: '{query}'") response = openai_client.vector_stores.search( vector_store_id=VECTOR_STORE_ID, query=query ) results = [] # Process the vector store search results if hasattr(response, "data") and response.data: for i, item in enumerate(response.data): # Extract file_id, filename, and content item_id = getattr(item, "file_id", f"vs_{i}") item_filename = getattr(item, "filename", f"Document {i + 1}") result = SearchResult( id=item_id, title=item_filename, url=f"https://platform.openai.com/storage/files/{item_id}", ) results.append(result) logger.info(f"Vector store search returned {len(results)} results") return SearchOutput(results=results) @mcp.tool(output_schema=FetchOutput.model_json_schema()) async def fetch(id: str) -> FetchOutput: """ Retrieve complete document content by ID for detailed analysis and citation. This tool fetches the full document content from OpenAI Vector Store. Use this after finding relevant documents with the search tool to get complete information for analysis and proper citation. Args: id: File ID from vector store (file-xxx) or local document ID Returns: Complete document with id, title, full text content, optional URL, and metadata Raises: ValueError: If the specified ID is not found """ if not id: raise ValueError("Document ID is required") if not openai_client: logger.error("OpenAI client not initialized - API key missing") raise ValueError( "OpenAI API key is required for vector store file retrieval" ) logger.info(f"Fetching content from vector store for file ID: {id}") # Fetch file content from vector store content_response = openai_client.vector_stores.files.content( vector_store_id=VECTOR_STORE_ID, file_id=id ) # Get file metadata file_info = openai_client.vector_stores.files.retrieve( vector_store_id=VECTOR_STORE_ID, file_id=id ) # Extract content from paginated response file_content = "" if hasattr(content_response, "data") and content_response.data: # Combine all content chunks from FileContentResponse objects content_parts = [] for content_item in content_response.data: if hasattr(content_item, "text"): content_parts.append(content_item.text) file_content = "\n".join(content_parts) else: file_content = "No content available" # Use filename as title and create proper URL for citations filename = getattr(file_info, "filename", f"Document {id}") result = FetchOutput( id=id, title=filename, text=file_content, url=f"https://platform.openai.com/storage/files/{id}", ) # Add metadata if available from file info if hasattr(file_info, "attributes") and file_info.attributes: result.metadata = dict(file_info.attributes) logger.info(f"Fetched vector store file: {id}") return result return mcp def main(): """Main function to start the MCP server.""" logger.info(f"Using vector store: {VECTOR_STORE_ID}") # Create the MCP server server = create_server() # Configure and start the server logger.info("Starting MCP server on 0.0.0.0:8000") logger.info("Server will be accessible via SSE transport") try: # Use FastMCP's built-in run method with SSE transport port = int(os.environ.get("OPENAI_EXAMPLE_PORT", "8000")) server.run( transport="sse", host="0.0.0.0", port=port, uvicorn_config={"loop": "asyncio"}, ) except KeyboardInterrupt: logger.info("Server stopped by user") except Exception as e: logger.error(f"Server error: {e}") raise if __name__ == "__main__": main() ``` #### Replit setup On Replit, configure `OPENAI_API_KEY` with your OpenAI API key in the "Secrets" UI. In the sample, replace `vs_123` with the ID of the vector store you created earlier for search. On free Replit accounts, server URLs are active for as long as the editor is active, so while you are testing, you'll need to keep the browser tab open. You can get a URL for your MCP server by clicking on the chainlink icon: ![replit configuration](https://cdn.openai.com/API/docs/images/replit.png) In the long dev URL, ensure it ends with `/sse/`, which is the server-sent events (streaming) interface to the MCP server. This is the URL you will use to connect your app in ChatGPT and call it via API. An example Replit URL looks like: ``` https://777xxx.janeway.replit.dev/sse/ ``` ## Test and connect your MCP server You can test your MCP server with a deep research model [in the prompts dashboard](https://platform.openai.com/chat). Create a new prompt, or edit an existing one, and add a new MCP tool to the prompt configuration. This compatibility example exposes only read-only `search` and `fetch` tools, so its API request skips approval for those tools. Keep approval enabled for tools that can modify data or take other consequential actions. If you are testing this server as part of a plugin, follow [Connect and test your plugin](https://developers.openai.com/plugins/deploy/connect-chatgpt). ![prompts configuration](https://cdn.openai.com/API/docs/images/prompts_mcp.png) Once you have configured your MCP server, you can chat with a model using it via the Prompts UI. ![prompts chat](https://cdn.openai.com/API/docs/images/chat_prompts_mcp.png) You can test the MCP server using the Responses API directly with a request like this one: ```bash curl https://api.openai.com/v1/responses \ -H "Content-Type: application/json" \ -H "Authorization: Bearer $OPENAI_API_KEY" \ -d '{ "model": "gpt-5.6-sol", "input": [ { "role": "developer", "content": [ { "type": "input_text", "text": "You are a research assistant that searches MCP servers to find answers to your questions." } ] }, { "role": "user", "content": [ { "type": "input_text", "text": "Are cats attached to their homes? Give a succinct one page overview." } ] } ], "reasoning": { "summary": "auto" }, "tools": [ { "type": "mcp", "server_label": "cats", "server_url": "https://777ff573-9947-4b9c-8982-658fa40c7d09-00-3le96u7wsymx.janeway.replit.dev/sse/", "allowed_tools": [ "search", "fetch" ], "require_approval": "never" } ] }' ``` ### Handle authentication As someone building a custom remote MCP server, authorization and authentication help you protect your data. We recommend using OAuth with [Client ID Metadata Documents](https://modelcontextprotocol.io/specification/2025-11-25/basic/authorization#client-id-metadata-documents) for client registration when your authorization server supports CIMD and the plugin creator chooses it. ChatGPT supports CIMD with public-client token exchange (`none`) or signed client assertion token exchange (`private_key_jwt`). Dynamic client registration remains supported when configured. For plugin authentication requirements, see [Authentication](https://developers.openai.com/plugins/build/auth). For protocol details, read the [MCP user guide](https://modelcontextprotocol.io/docs/concepts/transports#authentication-and-authorization) or the [authorization specification](https://modelcontextprotocol.io/specification/2025-11-25/basic/authorization). If you connect your custom remote MCP server through a plugin, users in your workspace will get an OAuth flow to your service. ### Connect in ChatGPT 1. In [ChatGPT](https://chatgpt.com), open **Settings → Security and login** and turn on **Developer mode**. 1. Go to [ChatGPT Plugins](https://chatgpt.com/plugins), select the plus button, and connect your server URL in developer mode. 1. Test your plugin by running prompts in chat and deep research. For detailed setup steps, see [Connect and test your plugin](https://developers.openai.com/plugins/deploy/connect-chatgpt). ## Risks and safety Custom MCP servers enable you to connect your ChatGPT workspace to external applications, which allows ChatGPT to access, send and receive data in these applications. Please note that custom MCP servers are not developed or verified by OpenAI, and are third-party services that are subject to their own terms and conditions. If you come across a malicious MCP server, please report it to security@openai.com. ### Prompt injection-related risks Prompt injections are a form of attack where an attacker embeds malicious instructions in content that one of our models is likely to encounter–such as a webpage–with the intention that the instructions override ChatGPT’s intended behavior. If the model obeys the injected instructions it may take actions the user and developer never intended—including sending private data to an external destination. For example, you might ask ChatGPT to find a restaurant for a group dinner by checking your calendar and recent emails. While researching, it might encounter a malicious comment—essentially a harmful piece of content designed to trick the agent into performing unintended actions—directing it to retrieve a password reset code from Gmail and send it to a malicious website. Below is a table of specific scenarios to consider. We recommend reviewing this table carefully to inform your decision about whether to use custom MCPs. | Scenario / Risk | Is it safe if I trust the MCP’s developer? | What can I do to reduce risk? | | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | An attacker may somehow insert a prompt injection attack into data accessible via the MCP.

_Examples:_
• For a customer support MCP, an attacker could send you a customer support request with a prompt injection attack. | Trusting a MCP’s developer does not make this safe.

For this to be safe you need to trust _all content that can be accessed within the MCP_. | • Do not use a MCP if it could contain malicious or untrusted user input, even if you trust the developer of the MCP.
• Configure access to minimize how many people have access to the MCP. | | A malicious MCP may request excessive parameters to a read or write action.

_Example:_
• An employee flight booking MCP could expose a read action to get a flight schedule, but request parameters including `summaryOfConversation`, `userAnnualIncome`, `userHomeAddress`. | Trusting a MCP’s developer does not necessarily make this safe.

A MCP’s developer may consider it reasonable to be requesting certain data that you do not consider acceptable to share. | • When installing MCP servers manually, review the parameters requested for each action and ensure there is no privacy overreach. | | An attacker may use a prompt injection attack to trick ChatGPT into fetching sensitive data from a custom MCP, to then be sent to the attacker.

_Example:_
• An attacker may deliver a prompt injection attack to one of the enterprise users via a different MCP (for example, email), where the attack attempts to trick ChatGPT into reading sensitive data from an internal tool and sending it to the attacker. | Trusting a MCP’s developer does not make this safe.

Everything within the new MCP could be safe and trusted since the risk is this data being stolen by attacks coming from a different malicious source. | • _ChatGPT is designed to protect users_, but attackers may attempt to steal your data, so be aware of the risk and consider whether taking it makes sense.
• Configure access to minimize how many people have access to MCPs with particularly sensitive data. | | An attacker may use a prompt injection attack to leak sensitive information through a write action to a custom MCP.

_Example:_
• An attacker uses a prompt injection attack through a different MCP to trick ChatGPT into fetching sensitive data, then using an MCP for a customer support system to send it to the attacker. | Trusting a MCP’s developer does not make this safe.

Even if you fully trust the MCP, if write actions have any consequences that can be observed by an attacker, they could attempt to take advantage of it. | • Users should review write actions carefully when they happen (to ensure they were intended and do not contain any data that shouldn’t be shared). | | An attacker may use a prompt injection attack to leak sensitive information through a read action to a malicious custom MCP because the MCP can log these actions. | This attack only works if the MCP is malicious, or if the MCP incorrectly marks write actions as read actions.

If you trust a MCP’s developer to correctly only mark read actions as _read_, and trust that developer to not attempt to steal data, then this risk is likely minimal. | • Only use MCPs from developers that you trust (though note this isn’t sufficient to make it safe). | | An attacker may use a prompt injection attack to trick ChatGPT into taking a harmful or destructive write action via a custom MCP that users did not intend. | Trusting a MCP’s developer does not make this safe.

Everything within the new MCP could be safe and trusted, and this risk still exists since the attack comes from a different malicious source. | • Users should carefully review write actions to ensure they are intended and correct.
• ChatGPT is designed to protect users, but attackers may attempt to trick ChatGPT into taking unintended write actions.
• Configure access to minimize how many people have access to MCPs with particularly sensitive data. | ### Non-prompt injection related risks Custom MCPs introduce other risks unrelated to prompt injection attacks: - **Write actions can increase both the usefulness and the risks of MCP servers**, because they make it possible for the server to take potentially destructive actions rather than only providing information back to ChatGPT. ChatGPT currently requires manual confirmation in any conversation before write actions can be taken. The confirmation will flag potentially sensitive data but you should only use write actions in situations where you have carefully considered, and are comfortable with, the possibility that ChatGPT might make a mistake involving such an action. It is possible for write actions to occur even if the MCP server has tagged the action as read only, making it even more important that you trust the custom MCP server before deploying to ChatGPT. - **Any MCP server may receive sensitive data as part of querying**. Even when the server is not malicious, it will have access to whatever data ChatGPT supplies during the interaction, potentially including sensitive data the user may earlier have provided to ChatGPT. For instance, such data could be included in queries ChatGPT sends to the MCP server when using deep research or chat app tools. ### Connecting to trusted servers We recommend that you do not connect to a custom MCP server unless you know and trust the underlying application. For example, choose official servers hosted by the service providers themselves. Connect to the Stripe server hosted by Stripe at `mcp.stripe.com` instead of an unofficial Stripe MCP server hosted by a third party. Because few official MCP servers are available today, you may consider a server hosted by an organization that proxies requests to another service through an API. Only connect after you have reviewed how the organization uses your data and verified that you can trust the server. When building and connecting to your own MCP server, double-check that it's the correct server. Be careful about the data you provide in response to requests and how you treat data sent to you when OpenAI calls your MCP server. Your remote MCP server permits others to connect OpenAI to your services and allows OpenAI to access, send and receive data, and take action in these services. Avoid putting any sensitive information in the JSON for your tools, and avoid storing any sensitive information from ChatGPT users accessing your remote MCP server. As someone building an MCP server, don't put anything malicious in your tool definitions. --- # ChatGPT Developer mode [