Skip to content
For the complete documentation index, see llms.txt. Markdown versions of documentation pages are available by appending .md to the page URL.
Primary navigation
Theme

Suggested
Overview Models Agents Tools Audio & voice Production API reference

Realtime

Connect
WebSocket
/realtime
Models
AudioTranscription object { delay, keywords, language, 3 more }
ConversationCreatedEvent object { conversation, event_id, type }

Returned when a conversation is created. Emitted right after session creation.

ConversationItem = RealtimeConversationItemSystemMessage { content, role, type, 3 more } or RealtimeConversationItemUserMessage { content, role, type, 3 more } or RealtimeConversationItemAssistantMessage { content, role, type, 3 more } or 6 more

A single item within a Realtime conversation.

ConversationItemAdded object { event_id, item, type, previous_item_id }

Sent by the server when an Item is added to the default Conversation. This can happen in several cases:

  • When the client sends a conversation.item.create event.
  • When the input audio buffer is committed. In this case the item will be a user message containing the audio from the buffer.
  • When the model is generating a Response. In this case the conversation.item.added event will be sent when the model starts generating a specific Item, and thus it will not yet have any content (and status will be in_progress).

The event will include the full content of the Item (except when model is generating a Response) except for audio data, which can be retrieved separately with a conversation.item.retrieve event if necessary.

ConversationItemCreateEvent object { item, type, event_id, previous_item_id }

Add a new Item to the Conversation’s context, including messages, function calls, and function call responses. This event can be used both to populate a “history” of the conversation and to add new items mid-stream, but has the current limitation that it cannot populate assistant audio messages.

If successful, the server will emit a conversation.item.added event and, when the item is finalized, a conversation.item.done event. Otherwise, an error event will be sent.

ConversationItemCreatedEvent object { event_id, item, type, previous_item_id }

Returned when a conversation item is created. There are several scenarios that produce this event:

  • The server is generating a Response, which if successful will produce either one or two Items, which will be of type message (role assistant) or type function_call.
  • The input audio buffer has been committed, either by the client or the server (in server_vad mode). The server will take the content of the input audio buffer and add it to a new user message Item.
  • The client has sent a conversation.item.create event to add a new Item to the Conversation.
ConversationItemDeleteEvent object { item_id, type, event_id }

Send this event when you want to remove any item from the conversation history. The server will respond with a conversation.item.deleted event, unless the item does not exist in the conversation history, in which case the server will respond with an error.

item_id: string

The ID of the item to delete.

type: "conversation.item.delete"

The event type, must be conversation.item.delete.

event_id: optional string

Optional client-generated ID used to identify this event.

maxLength512
ConversationItemDeletedEvent object { event_id, item_id, type }

Returned when an item in the conversation is deleted by the client with a conversation.item.delete event. This event is used to synchronize the server’s understanding of the conversation history with the client’s view.

event_id: string

The unique ID of the server event.

item_id: string

The ID of the item that was deleted.

type: "conversation.item.deleted"

The event type, must be conversation.item.deleted.

ConversationItemDone object { event_id, item, type, previous_item_id }

Returned when a conversation item is finalized.

The event will include the full content of the Item except for audio data, which can be retrieved separately with a conversation.item.retrieve event if needed.

ConversationItemInputAudioTranscriptionCompletedEvent object { content_index, event_id, item_id, 5 more }

This event is the output of audio transcription for user audio written to the user audio buffer. Transcription begins when the input audio buffer is committed by the client or server (when VAD is enabled). Transcription runs asynchronously with Response creation, so this event may come before or after the Response events.

Realtime API models accept audio natively, and thus input transcription is a separate process run on a separate ASR (Automatic Speech Recognition) model. The transcript may diverge somewhat from the model’s interpretation, and should be treated as a rough guide.

ConversationItemInputAudioTranscriptionDeltaEvent object { event_id, item_id, type, 3 more }

Returned when the text value of an input audio transcription content part is updated with incremental transcription results.

ConversationItemInputAudioTranscriptionFailedEvent object { content_index, error, event_id, 2 more }

Returned when input audio transcription is configured, and a transcription request for a user message failed. These events are separate from other error events so that the client can identify the related Item.

ConversationItemInputAudioTranscriptionSegment object { id, content_index, end, 6 more }

Returned when an input audio transcription segment is identified for an item.

ConversationItemRetrieveEvent object { item_id, type, event_id }

Send this event when you want to retrieve the server’s representation of a specific item in the conversation history. This is useful, for example, to inspect user audio after noise cancellation and VAD. The server will respond with a conversation.item.retrieved event, unless the item does not exist in the conversation history, in which case the server will respond with an error.

item_id: string

The ID of the item to retrieve.

type: "conversation.item.retrieve"

The event type, must be conversation.item.retrieve.

event_id: optional string

Optional client-generated ID used to identify this event.

maxLength512
ConversationItemTruncateEvent object { audio_end_ms, content_index, item_id, 2 more }

Send this event to truncate a previous assistant message’s audio. The server will produce audio faster than realtime, so this event is useful when the user interrupts to truncate audio that has already been sent to the client but not yet played. This will synchronize the server’s understanding of the audio with the client’s playback.

Truncating audio will delete the server-side text transcript to ensure there is not text in the context that hasn’t been heard by the user.

If successful, the server will respond with a conversation.item.truncated event.

ConversationItemTruncatedEvent object { audio_end_ms, content_index, event_id, 2 more }

Returned when an earlier assistant audio message item is truncated by the client with a conversation.item.truncate event. This event is used to synchronize the server’s understanding of the audio with the client’s playback.

This action will truncate the audio and remove the server-side text transcript to ensure there is no text in the context that hasn’t been heard by the user.

ConversationItemWithReference object { id, arguments, call_id, 7 more }

The item to add to the conversation.

InputAudioBufferAppendEvent object { audio, type, event_id }

Send this event to append audio bytes to the input audio buffer. The audio buffer is temporary storage you can write to and later commit. A “commit” will create a new user message item in the conversation history from the buffer content and clear the buffer. Input audio transcription (if enabled) will be generated when the buffer is committed.

If VAD is enabled the audio buffer is used to detect speech and the server will decide when to commit. When Server VAD is disabled, you must commit the audio buffer manually. Input audio noise reduction operates on writes to the audio buffer.

The client may choose how much audio to place in each event up to a maximum of 15 MiB, for example streaming smaller chunks from the client may allow the VAD to be more responsive. Unlike most other client events, the server will not send a confirmation response to this event.

InputAudioBufferClearEvent object { type, event_id }

Send this event to clear the audio bytes in the buffer. The server will respond with an input_audio_buffer.cleared event.

type: "input_audio_buffer.clear"

The event type, must be input_audio_buffer.clear.

event_id: optional string

Optional client-generated ID used to identify this event.

maxLength512
InputAudioBufferClearedEvent object { event_id, type }

Returned when the input audio buffer is cleared by the client with a input_audio_buffer.clear event.

event_id: string

The unique ID of the server event.

type: "input_audio_buffer.cleared"

The event type, must be input_audio_buffer.cleared.

InputAudioBufferCommitEvent object { type, event_id }

Send this event to commit the user input audio buffer, which will create a new user message item in the conversation. This event will produce an error if the input audio buffer is empty. When in Server VAD mode, the client does not need to send this event, the server will commit the audio buffer automatically.

Committing the input audio buffer will trigger input audio transcription (if enabled in session configuration), but it will not create a response from the model. The server will respond with an input_audio_buffer.committed event.

type: "input_audio_buffer.commit"

The event type, must be input_audio_buffer.commit.

event_id: optional string

Optional client-generated ID used to identify this event.

maxLength512
InputAudioBufferCommittedEvent object { event_id, item_id, type, previous_item_id }

Returned when an input audio buffer is committed, either by the client or automatically in server VAD mode. The item_id property is the ID of the user message item that will be created, thus a conversation.item.created event will also be sent to the client.

InputAudioBufferDtmfEventReceivedEvent object { event, received_at, type }

SIP Only: Returned when an DTMF event is received. A DTMF event is a message that represents a telephone keypad press (0–9, *, #, A–D). The event property is the keypad that the user press. The received_at is the UTC Unix Timestamp that the server received the event.

event: string

The telephone keypad that was pressed by the user.

received_at: integer

UTC Unix Timestamp when DTMF Event was received by server.

type: "input_audio_buffer.dtmf_event_received"

The event type, must be input_audio_buffer.dtmf_event_received.

InputAudioBufferSpeechStartedEvent object { audio_start_ms, event_id, item_id, type }

Sent by the server when in server_vad mode to indicate that speech has been detected in the audio buffer. This can happen any time audio is added to the buffer (unless speech is already detected). The client may want to use this event to interrupt audio playback or provide visual feedback to the user.

The client should expect to receive a input_audio_buffer.speech_stopped event when speech stops. The item_id property is the ID of the user message item that will be created when speech stops and will also be included in the input_audio_buffer.speech_stopped event (unless the client manually commits the audio buffer during VAD activation).

InputAudioBufferSpeechStoppedEvent object { audio_end_ms, event_id, item_id, type }

Returned in server_vad mode when the server detects the end of speech in the audio buffer. The server will also send an conversation.item.created event with the user message item that is created from the audio buffer.

InputAudioBufferTimeoutTriggered object { audio_end_ms, audio_start_ms, event_id, 2 more }

Returned when the Server VAD timeout is triggered for the input audio buffer. This is configured with idle_timeout_ms in the turn_detection settings of the session, and it indicates that there hasn’t been any speech detected for the configured duration.

The audio_start_ms and audio_end_ms fields indicate the segment of audio after the last model response up to the triggering time, as an offset from the beginning of audio written to the input audio buffer. This means it demarcates the segment of audio that was silent and the difference between the start and end values will roughly match the configured timeout.

The empty audio will be committed to the conversation as an input_audio item (there will be a input_audio_buffer.committed event) and a model response will be generated. There may be speech that didn’t trigger VAD but is still detected by the model, so the model may respond with something relevant to the conversation or a prompt to continue speaking.

LogProbProperties object { token, bytes, logprob }

A log probability object.

token: string

The token that was used to generate the log probability.

bytes: array of number

The bytes that were used to generate the log probability.

logprob: number

The log probability of the token.

McpListToolsCompleted object { event_id, item_id, type }

Returned when listing MCP tools has completed for an item.

event_id: string

The unique ID of the server event.

item_id: string

The ID of the MCP list tools item.

type: "mcp_list_tools.completed"

The event type, must be mcp_list_tools.completed.

McpListToolsFailed object { event_id, item_id, type }

Returned when listing MCP tools has failed for an item.

event_id: string

The unique ID of the server event.

item_id: string

The ID of the MCP list tools item.

type: "mcp_list_tools.failed"

The event type, must be mcp_list_tools.failed.

McpListToolsInProgress object { event_id, item_id, type }

Returned when listing MCP tools is in progress for an item.

event_id: string

The unique ID of the server event.

item_id: string

The ID of the MCP list tools item.

type: "mcp_list_tools.in_progress"

The event type, must be mcp_list_tools.in_progress.

NoiseReductionType = "near_field" or "far_field"

Type of noise reduction. near_field is for close-talking microphones such as headphones, far_field is for far-field microphones such as laptop or conference room microphones.

One of the following:
"near_field"
"far_field"
OutputAudioBufferClearEvent object { type, event_id }

WebRTC/SIP Only: Emit to cut off the current audio response. This will trigger the server to stop generating audio and emit a output_audio_buffer.cleared event. This event should be preceded by a response.cancel client event to stop the generation of the current response. Learn more.

type: "output_audio_buffer.clear"

The event type, must be output_audio_buffer.clear.

event_id: optional string

The unique ID of the client event used for error handling.

RateLimitsUpdatedEvent object { event_id, rate_limits, type }

Emitted at the beginning of a Response to indicate the updated rate limits. When a Response is created some tokens will be “reserved” for the output tokens, the rate limits shown here reflect that reservation, which is then adjusted accordingly once the Response is completed.

RealtimeAudioConfig object { input, output }

Configuration for input and output audio.

input: optional RealtimeAudioConfigInput { format, noise_reduction, transcription, turn_detection }
output: optional RealtimeAudioConfigOutput { format, speed, voice }
RealtimeAudioConfigInput object { format, noise_reduction, transcription, turn_detection }
RealtimeAudioConfigOutput object { format, speed, voice }
RealtimeAudioFormats = PCMAudio { rate, type } or PCMUAudio { type } or PCMAAudio { type }

The PCM audio format. Only a 24kHz sample rate is supported.

RealtimeAudioInputTurnDetection = ServerVad { type, create_response, idle_timeout_ms, 4 more } or SemanticVad { type, create_response, eagerness, interrupt_response }

Configuration for turn detection, ether Server VAD or Semantic VAD. This can be set to null to turn off, in which case the client must manually trigger model response.

Server VAD means that the model will detect the start and end of speech based on audio volume and respond at the end of user speech.

Semantic VAD is more advanced and uses a turn detection model (in conjunction with VAD) to semantically estimate whether the user has finished speaking, then dynamically sets a timeout based on this probability. For example, if user audio trails off with “uhhm”, the model will score a low probability of turn end and wait longer for the user to continue speaking. This can be useful for more natural conversations, but may have a higher latency.

For gpt-realtime-whisper transcription sessions, turn detection must be set to null; VAD is not supported.

RealtimeClientEvent = ConversationItemCreateEvent { item, type, event_id, previous_item_id } or ConversationItemDeleteEvent { item_id, type, event_id } or ConversationItemRetrieveEvent { item_id, type, event_id } or 8 more

A realtime client event.

RealtimeConversationItemAssistantMessage object { content, role, type, 3 more }

An assistant message item in a Realtime conversation.

RealtimeConversationItemFunctionCall object { arguments, name, type, 4 more }

A function call item in a Realtime conversation.

RealtimeConversationItemFunctionCallOutput object { call_id, output, type, 3 more }

A function call output item in a Realtime conversation.

RealtimeConversationItemSystemMessage object { content, role, type, 3 more }

A system message in a Realtime conversation can be used to provide additional context or instructions to the model. This is similar but distinct from the instruction prompt provided at the start of a conversation, as system messages can be added at any point in the conversation. For major changes to the conversation’s behavior, use instructions, but for smaller updates (e.g. “the user is now asking about a different topic”), use system messages.

RealtimeConversationItemUserMessage object { content, role, type, 3 more }

A user message item in a Realtime conversation.

RealtimeError object { message, type, code, 2 more }

Details of the error.

RealtimeErrorEvent object { error, event_id, type }

Returned when an error occurs, which could be a client problem or a server problem. Most errors are recoverable and the session will stay open, we recommend to implementers to monitor and log error messages by default.

RealtimeFunctionTool object { description, name, parameters, type }
RealtimeMcpApprovalRequest object { id, arguments, name, 2 more }

A Realtime item requesting human approval of a tool invocation.

RealtimeMcpApprovalResponse object { id, approval_request_id, approve, 2 more }

A Realtime item responding to an MCP approval request.

RealtimeMcpListTools object { server_label, tools, type, id }

A Realtime item listing tools available on an MCP server.

RealtimeMcpProtocolError object { code, message, type }
code: integer
message: string
type: "protocol_error"
RealtimeMcpToolCall object { id, arguments, name, 5 more }

A Realtime item representing an invocation of a tool on an MCP server.

RealtimeMcpToolExecutionError object { message, type }
message: string
type: "tool_execution_error"
RealtimeMcphttpError object { code, message, type }
code: integer
message: string
type: "http_error"
RealtimeReasoning object { effort }

Configuration for reasoning-capable Realtime models such as gpt-realtime-2.

effort: optional RealtimeReasoningEffort (default: "low")

Constrains effort on reasoning for reasoning-capable Realtime models such as gpt-realtime-2.

RealtimeReasoningEffort = "minimal" or "low" or "medium" or 2 more

Constrains effort on reasoning for reasoning-capable Realtime models such as gpt-realtime-2.

RealtimeResponse object { id, audio, conversation_id, 8 more }

The response resource.

RealtimeResponseCreateAudioOutput object { output }

Configuration for audio input and output.

RealtimeResponseCreateParams object { audio, conversation, input, 9 more }

Create a new Realtime response with these parameters

RealtimeResponseStatus object { error, reason, type }

Additional details about the status.

RealtimeResponseUsage object { input_token_details, input_tokens, output_token_details, 2 more }

Usage statistics for the Response, this will correspond to billing. A Realtime API session will maintain a conversation context and append new Items to the Conversation, thus output from previous turns (text and audio tokens) will become the input for later turns.

RealtimeResponseUsageInputTokenDetails object { audio_tokens, cached_tokens, cached_tokens_details, 2 more }

Details about the input tokens used in the Response. Cached tokens are tokens from previous turns in the conversation that are included as context for the current response. Cached tokens here are counted as a subset of input tokens, meaning input tokens will include cached and uncached tokens.

RealtimeResponseUsageOutputTokenDetails object { audio_tokens, text_tokens }

Details about the output tokens used in the Response.

audio_tokens: optional integer

The number of audio tokens used in the Response.

text_tokens: optional integer

The number of text tokens used in the Response.

RealtimeServerEvent = ConversationCreatedEvent { conversation, event_id, type } or ConversationItemCreatedEvent { event_id, item, type, previous_item_id } or ConversationItemDeletedEvent { event_id, item_id, type } or 43 more

A realtime server event.

RealtimeSession object { id, expires_at, include, 17 more }

Realtime session object for the beta interface.

RealtimeSessionCreateRequest object { type, audio, include, 11 more }

Realtime session object configuration.

RealtimeToolChoiceConfig = ToolChoiceOptions or ToolChoiceFunction { name, type } or ToolChoiceMcp { server_label, type, name }

How the model chooses tools. Provide one of the string modes or force a specific function/MCP tool.

RealtimeToolsConfig = array of RealtimeToolsConfigUnion

Tools available to the model.

RealtimeToolsConfigUnion = RealtimeFunctionTool { description, name, parameters, type } or McpTool { server_label, type, allowed_callers, 9 more }

Give the model access to additional tools via remote Model Context Protocol (MCP) servers. Learn more about MCP.

RealtimeTracingConfig = "auto" or TracingConfiguration { group_id, metadata, workflow_name }

Realtime API can write session traces to the Traces Dashboard. Set to null to disable tracing. Once tracing is enabled for a session, the configuration cannot be modified.

auto will create a trace for the session with default values for the workflow name, group id, and metadata.

RealtimeTranscriptionSessionAudio object { input }

Configuration for input and output audio.

input: optional RealtimeTranscriptionSessionAudioInput { format, noise_reduction, transcription, turn_detection }
RealtimeTranscriptionSessionAudioInput object { format, noise_reduction, transcription, turn_detection }
RealtimeTranscriptionSessionAudioInputTurnDetection = ServerVad { type, create_response, idle_timeout_ms, 4 more } or SemanticVad { type, create_response, eagerness, interrupt_response }

Configuration for turn detection, ether Server VAD or Semantic VAD. This can be set to null to turn off, in which case the client must manually trigger model response.

Server VAD means that the model will detect the start and end of speech based on audio volume and respond at the end of user speech.

Semantic VAD is more advanced and uses a turn detection model (in conjunction with VAD) to semantically estimate whether the user has finished speaking, then dynamically sets a timeout based on this probability. For example, if user audio trails off with “uhhm”, the model will score a low probability of turn end and wait longer for the user to continue speaking. This can be useful for more natural conversations, but may have a higher latency.

For gpt-realtime-whisper transcription sessions, turn detection must be set to null; VAD is not supported.

RealtimeTranscriptionSessionCreateRequest object { type, audio, include }

Realtime transcription session object configuration.

RealtimeTranslationClientEvent = RealtimeTranslationSessionUpdateEvent { session, type, event_id } or RealtimeTranslationInputAudioBufferAppendEvent { audio, type, event_id } or RealtimeTranslationSessionCloseEvent { type, event_id }

A Realtime translation client event.

RealtimeTranslationClientSecretCreateRequest object { session, expires_after }

Create a translation session and client secret for the Realtime API.

RealtimeTranslationClientSecretCreateResponse object { expires_at, session, value }

Response from creating a translation session and client secret for the Realtime API.

RealtimeTranslationInputAudioBufferAppendEvent object { audio, type, event_id }

Send this event to append audio bytes to the translation session input audio buffer.

WebSocket translation sessions accept base64-encoded 24 kHz PCM16 mono little-endian raw audio bytes. Unsupported websocket audio formats return a validation error because lower-quality audio materially degrades translation quality.

Translation consumes 200 ms engine frames. For best realtime behavior, append audio in 200 ms chunks. If a chunk is shorter, the server buffers it until it has enough audio for one frame. If a chunk is longer, the server splits it into 200 ms frames and enqueues them back-to-back.

Keep appending silence while the session is active. If a client stops sending audio and later resumes, model time treats the resumed audio as contiguous with the previous audio rather than as a real-world pause.

RealtimeTranslationInputTranscriptDeltaEvent object { delta, event_id, type, elapsed_ms }

Returned when optional source-language transcript text is available. This event is emitted only when audio.input.transcription is configured.

Transcript deltas are append-only text fragments. Clients should not insert unconditional spaces between deltas.

RealtimeTranslationOutputAudioDeltaEvent object { delta, event_id, type, 4 more }

Returned when translated output audio is available. The delta contains a PCM16 audio chunk whose length can vary. Clients should decode and queue the complete delta instead of assuming a fixed byte or sample count.

RealtimeTranslationOutputTranscriptDeltaEvent object { delta, event_id, type, elapsed_ms }

Returned when translated transcript text is available.

Transcript deltas are append-only text fragments. Clients should not insert unconditional spaces between deltas.

RealtimeTranslationServerEvent = RealtimeErrorEvent { error, event_id, type } or RealtimeTranslationSessionCreatedEvent { event_id, session, type } or RealtimeTranslationSessionUpdatedEvent { event_id, session, type } or 4 more

A Realtime translation server event.

RealtimeTranslationSession object { id, audio, expires_at, 2 more }

A Realtime translation session. Translation sessions continuously translate input audio into the configured output language.

RealtimeTranslationSessionCloseEvent object { type, event_id }

Gracefully close the realtime translation session. The server flushes pending input audio and emits any remaining translated output before closing the session.

type: "session.close"

The event type, must be session.close.

event_id: optional string

Optional client-generated ID used to identify this event.

maxLength512
RealtimeTranslationSessionClosedEvent object { event_id, type }

Returned when a realtime translation session is closed.

event_id: string

The unique ID of the server event.

type: "session.closed"

The event type, must be session.closed.

RealtimeTranslationSessionCreateRequest object { model, audio }

Realtime translation session configuration. Translation sessions stream source audio in and translated audio plus transcript deltas out continuously.

RealtimeTranslationSessionCreatedEvent object { event_id, session, type }

Returned when a translation session is created. Emitted automatically when a new connection is established as the first server event. This event contains the default translation session configuration.

RealtimeTranslationSessionUpdateEvent object { session, type, event_id }

Send this event to update the translation session configuration. Translation sessions support updates to audio.output.language, audio.input.transcription, and audio.input.noise_reduction.

RealtimeTranslationSessionUpdateRequest object { audio }

Realtime translation session fields that can be updated with session.update.

RealtimeTranslationSessionUpdatedEvent object { event_id, session, type }

Returned when a translation session is updated with a session.update event, unless there is an error.

RealtimeTruncation = "auto" or "disabled" or RetentionRatioTruncation { retention_ratio, type, token_limits }

When the number of tokens in a conversation exceeds the model’s input token limit, the conversation be truncated, meaning messages (starting from the oldest) will not be included in the model’s context. A 32k context model with 4,096 max output tokens can only include 28,224 tokens in the context before truncation occurs.

Clients can configure truncation behavior to truncate with a lower max token limit, which is an effective way to control token usage and cost.

Truncation will reduce the number of cached tokens on the next turn (busting the cache), since messages are dropped from the beginning of the context. However, clients can also configure truncation to retain messages up to a fraction of the maximum context size, which will reduce the need for future truncations and thus improve the cache rate.

Truncation can be disabled entirely, which means the server will never truncate but would instead return an error if the conversation exceeds the model’s input token limit.

ResponseAudioDeltaEvent object { content_index, delta, event_id, 4 more }

Returned when the model-generated audio is updated.

ResponseAudioDoneEvent object { content_index, event_id, item_id, 3 more }

Returned when the model-generated audio is done. Also emitted when a Response is interrupted, incomplete, or cancelled.

ResponseAudioTranscriptDeltaEvent object { content_index, delta, event_id, 4 more }

Returned when the model-generated transcription of audio output is updated.

ResponseAudioTranscriptDoneEvent object { content_index, event_id, item_id, 4 more }

Returned when the model-generated transcription of audio output is done streaming. Also emitted when a Response is interrupted, incomplete, or cancelled.

ResponseCancelEvent object { type, event_id, response_id }

Send this event to cancel an in-progress response. The server will respond with a response.done event with a status of response.status=cancelled. If there is no response to cancel, the server will respond with an error. It’s safe to call response.cancel even if no response is in progress, an error will be returned the session will remain unaffected.

ResponseContentPartAddedEvent object { content_index, event_id, item_id, 4 more }

Returned when a new content part is added to an assistant message item during response generation.

ResponseContentPartDoneEvent object { content_index, event_id, item_id, 4 more }

Returned when a content part is done streaming in an assistant message item. Also emitted when a Response is interrupted, incomplete, or cancelled.

ResponseCreateEvent object { type, event_id, response }

This event instructs the server to create a Response, which means triggering model inference. When in Server VAD mode, the server will create Responses automatically.

A Response will include at least one Item, and may have two, in which case the second will be a function call. These Items will be appended to the conversation history by default.

The server will respond with a response.created event, events for Items and content created, and finally a response.done event to indicate the Response is complete.

The response.create event includes inference configuration like instructions and tools. If these are set, they will override the Session’s configuration for this Response only.

Responses can be created out-of-band of the default Conversation, meaning that they can have arbitrary input, and it’s possible to disable writing the output to the Conversation. Only one Response can write to the default Conversation at a time, but otherwise multiple Responses can be created in parallel. The metadata field is a good way to disambiguate multiple simultaneous Responses.

Clients can set conversation to none to create a Response that does not write to the default Conversation. Arbitrary input can be provided with the input field, which is an array accepting raw Items and references to existing Items.

ResponseCreatedEvent object { event_id, response, type }

Returned when a new Response is created. The first event of response creation, where the response is in an initial state of in_progress.

ResponseDoneEvent object { event_id, response, type }

Returned when a Response is done streaming. Always emitted, no matter the final state. The Response object included in the response.done event will include all output Items in the Response but will omit the raw audio data.

Clients should check the status field of the Response to determine if it was successful (completed) or if there was another outcome: cancelled, failed, or incomplete.

A response will contain all output items that were generated during the response, excluding any audio content.

ResponseFunctionCallArgumentsDeltaEvent object { call_id, delta, event_id, 4 more }

Returned when the model-generated function call arguments are updated.

ResponseFunctionCallArgumentsDoneEvent object { arguments, call_id, event_id, 5 more }

Returned when the model-generated function call arguments are done streaming. Also emitted when a Response is interrupted, incomplete, or cancelled.

ResponseItemReference object { id, type }

Reference an existing item in this Realtime session as response.create input. Item references are only accepted in the response input array, not as new conversation items.

id: string

The ID of an existing item in the same Realtime session.

pattern^[a-zA-Z0-9_-]+$
minLength1
maxLength32
type: "item_reference"

The type of this item. Always item_reference.

ResponseMcpCallArgumentsDelta object { delta, event_id, item_id, 4 more }

Returned when MCP tool call arguments are updated during response generation.

ResponseMcpCallArgumentsDone object { arguments, event_id, item_id, 3 more }

Returned when MCP tool call arguments are finalized during response generation.

ResponseMcpCallCompleted object { event_id, item_id, output_index, type }

Returned when an MCP tool call has completed successfully.

ResponseMcpCallFailed object { event_id, item_id, output_index, type }

Returned when an MCP tool call has failed.

ResponseMcpCallInProgress object { event_id, item_id, output_index, type }

Returned when an MCP tool call has started and is in progress.

ResponseOutputItemAddedEvent object { event_id, item, output_index, 2 more }

Returned when a new Item is created during Response generation.

ResponseOutputItemDoneEvent object { event_id, item, output_index, 2 more }

Returned when an Item is done streaming. Also emitted when a Response is interrupted, incomplete, or cancelled.

ResponseTextDeltaEvent object { content_index, delta, event_id, 4 more }

Returned when the text value of an “output_text” content part is updated.

ResponseTextDoneEvent object { content_index, event_id, item_id, 4 more }

Returned when the text value of an “output_text” content part is done streaming. Also emitted when a Response is interrupted, incomplete, or cancelled.

SessionCreatedEvent object { event_id, session, type }

Returned when a Session is created. Emitted automatically when a new connection is established as the first server event. This event will contain the default Session configuration.

SessionUpdateEvent object { session, type, event_id }

Send this event to update the session’s configuration. The client may send this event at any time to update any field except for voice and model. voice can be updated only if there have been no other audio outputs yet.

When the server receives a session.update, it will respond with a session.updated event showing the full, effective configuration. Only the fields that are present in the session.update are updated. To clear a field like instructions, pass an empty string. To clear a field like tools, pass an empty array. To clear a field like turn_detection, pass null.

To turn off input audio noise reduction, send this Realtime event:

{"type":"session.update","session":{"type":"realtime","audio":{"input":{"noise_reduction":null}}}}

For a transcription session, use "type":"transcription" inside session. Omitting audio.input.noise_reduction from an update leaves its current setting unchanged.

SessionUpdatedEvent object { event_id, session, type }

Returned when a session is updated with a session.update event, unless there is an error.

TranscriptionSessionUpdate object { session, type, event_id }

Send this event to update a transcription session.

TranscriptionSessionUpdatedEvent object { event_id, session, type }

Returned when a transcription session is updated with a transcription_session.update event, unless there is an error.

RealtimeCalls

Accept call
POST
/realtime/calls/{call_id}/accept
Create call
POST
/realtime/calls
Hang up call
POST
/realtime/calls/{call_id}/hangup
Refer call
POST
/realtime/calls/{call_id}/refer
Reject call
POST
/realtime/calls/{call_id}/reject

RealtimeClient Secrets

Create client secret
POST
/realtime/client_secrets
Models
ClientSecretCreateResponse object { expires_at, session, value }

Response from creating a session and client secret for the Realtime API.

RealtimeSessionCreateResponse object { id, object, type, 13 more }

A Realtime session configuration object.

RealtimeTranscriptionSessionCreateResponse object { id, object, type, 3 more }

A Realtime transcription session configuration object.

RealtimeTranscriptionSessionTurnDetection object { prefix_padding_ms, silence_duration_ms, threshold, type }

Configuration for turn detection. Can be set to null to turn off. Server VAD means that the model will detect the start and end of speech based on audio volume and respond at the end of user speech. For gpt-realtime-whisper, this must be null; VAD is not supported.

RealtimeSessions

Create session
Deprecated
POST
/realtime/sessions
Models
SessionCreateResponse object { id, audio, expires_at, 10 more }

A Realtime session configuration object.

RealtimeTranscription Sessions

Create transcription session
Deprecated
POST
/realtime/transcription_sessions
Models
TranscriptionSessionCreateResponse object { client_secret, input_audio_format, input_audio_transcription, 2 more }

A new Realtime transcription session configuration.

When a session is created on the server via REST API, the session object also contains an ephemeral key. Default TTL for keys is 10 minutes. This property is not present when a session is updated via the WebSocket API.

RealtimeTranslations

Connect
WebSocket
/realtime/translations

RealtimeTranslationsClient Secrets

Create translation client secret
POST
/realtime/translations/client_secrets