Skip to content
For the complete documentation index, see llms.txt. Markdown versions of documentation pages are available by appending .md to the page URL.
Primary navigation

Suggested
Overview Models Agents Tools Audio & voice Production API reference

Live

Connect
WebSocket
/live/sessions
Create session
client.live.create(body, options?): LiveCreateResponse { session, transport }
POST
/live/sessions
Models
AudioFormat = AudioPCM { rate, type } | AudioPCMU { rate, type } | AudioPCMA { rate, type }

Audio encoding and sample rate for audio sent and received over a Live WebSocket connection. WebRTC and SIP negotiate their media format separately.

BuiltInVoice = "alloy" | "ash" | "ballad" | 20 more

A built-in voice available for Live speech.

ClientConfig { data_channel }

Startup-only capabilities for an untrusted frontend attached to a unified WebRTC session. Trusted sideband connections are unaffected.

ClientDelegation { type, acknowledgement_delay_ms }

Delegate tasks to your application. The Live session emits delegation events that your backend handles.

ClientEvent = SessionStartEvent { session, type, event_id } | SessionUpdateEvent { session, type, event_id } | InputAudioAppendEvent { audio, type, event_id } | 12 more

Client events for Live. Initialize a primary WebSocket with session.start and wait for session.started. WebRTC creation already starts the session. Audio append is primary WebSocket-only. See the Live prompting guide before writing frontend instructions and delegation policies.

CommentaryAppendEvent { content, delegation_id, type, event_id }

Provide context the Live model can communicate to the user, optionally for an existing client delegation. Rejected while an audio fixture is active.

CommentaryAppendedEvent { end_ms, event_id, start_ms, 2 more }

Returned when a session.commentary.append command is accepted into the Live session timeline. Acknowledges the added commentary without guaranteeing exact wording or completed audio playback.

CustomVoice { id }
DataChannelConfig { allowed_client_events, allowed_server_events }

Control which Live events an untrusted WebRTC frontend can send and receive over its data channel. These restrictions do not apply to trusted sideband connections.

DecisionAssistantMessage { content, role, id, 2 more }
DecisionChoiceOption { value, description }
DecisionCreateEvent { event_id, type }

Evaluate the configured questions using the current conversation and Decision inputs. Reusing an event_id is rejected for the lifetime of the session.

DecisionDeveloperMessage { content, role, id, type }
DecisionEvent { delegation_id, event, event_id, 4 more }

A Decision API event for a Live delegation or transcript trigger. For a trigger, source_event_id identifies the transcript delta that caused this evaluation.

DecisionInputText { text, type }
DecisionItemCreateEvent { item, type, event_id }

Append an input item for future decisions without starting an evaluation or changing Live voice context.

DecisionItemCreatedEvent { event_id, item, type, client_event_id }

An input item was appended for future decisions.

DecisionItemDeleteEvent { item_id, type, event_id }

Delete an input item from future decisions without starting an evaluation or changing Live voice context.

DecisionItemDeletedEvent { event_id, item_id, type, client_event_id }

An input item was removed from future decisions.

DecisionJSONObject = Record<string, unknown>
DecisionQuestion = Choice { choices, instructions, type, name } | Predicate { instructions, type, name } | Score { instructions, levels, type, name }
DecisionStoredAssistantMessage { id, content, role, 2 more }
DecisionStoredDeveloperMessage { id, content, role, type }
DecisionStoredSystemMessage { id, content, role, type }
DecisionStoredUserMessage { id, content, role, type }
DecisionSystemMessage { content, role, id, type }
DecisionTrigger { decisions, event_type, type }
DecisionUserMessage { content, role, id, type }
DecisionsDelegationConfig { model, questions }
DecisionsDelegationUpdateConfig { model, questions }
DelegationCreatedEvent { delegation, event_id, offset_ms, 2 more }

Returned when the Live model delegates work to your application, Responses, or Decisions. Contains delegation metadata and the position on the session timeline where the work was delegated.

Error { code, message, type, 2 more }

Details of an error encountered by the Live session, including the affected parameter or client command when available.

ErrorEvent { error, event_id, type, client_event_id }

Reports an error in the Live session, such as an invalid client command. Use error.client_event_id, when present, to identify the command that caused the error.

ForkSessionConfig { audio, client, delegation, store }

Overrides for a stored session after connecting to the fork WebSocket. An empty object inherits the stored configuration; do not supply a new model. audio.format applies only to the new WebSocket connection. client overrides are only supported for WebRTC forks.

ForkSessionStartEvent { session, type, event_id }

Start a Live session after connecting to a stored session’s fork WebSocket. Send an empty session object to use the stored configuration.

FunctionTool { name, type, description, 2 more }

A function tool available to the Responses backend when the Live model delegates a task.

InfoEvent { code, event_id, message, 2 more }

An informational notice about the Live session, such as the event permissions applied to a frontend data channel.

InitialItem = Developer { content, role, id, 2 more } | User { content, role, id, 2 more } | Assistant { content, role, id, 2 more }

A developer, user, or assistant message supplied as text history before the Live session starts.

InputAudioAppendEvent { audio, type, event_id }

Send audio to a Live session over its primary WebSocket. WebRTC and SIP sessions send audio over their media transport.

InputAudioMuteEvent { type, event_id }

Mute audio input to the Live model without closing the session. The server acknowledges with session.input_audio.muted.

InputAudioMutedEvent { event_id, type, client_event_id }

Acknowledges a session.input_audio.mute command. An independent input mute sends silence to the model and preserves sideband audio reflection. Fixture playback does not emit this event.

InputAudioUnmuteEvent { type, event_id }

Resume audio input to a Live model after muting it. The server acknowledges with session.input_audio.unmuted. An active audio fixture still pauses inference until playback finishes.

InputAudioUnmutedEvent { event_id, type, client_event_id }

Acknowledges a session.input_audio.unmute command, removing the independent input mute. An active audio fixture still pauses inference until playback finishes. Fixture playback does not emit this event.

InputTranscriptDeltaEvent { delta, end_ms, event_id, 3 more }

A transcript fragment for user input audio in the Live session. Accumulate fragments in delivery order; these events do not define complete turns or include a transcript-done event.

InstructionsAppendEvent { content, delegation_id, type, event_id }

Append developer instructions to the Live conversation while it is running, optionally associating them with an existing client delegation. Rejected while an audio fixture is active. These instructions remain in conversation history through compaction and are included in future backend delegations. For external audio, describe content already delivered so the assistant does not repeat it.

InstructionsAppendedEvent { end_ms, event_id, start_ms, 2 more }

Returned when a session.instructions.append command is accepted into the Live session timeline. Acknowledges the appended instructions without guaranteeing that the model has acted on them.

LiveCreateResponse { session, transport }

The created Live session identifier and WebRTC answer. Apply transport.sdp as the peer’s remote answer and wait for session.started on the data channel before sending commands.

MediaSessionConfig { model, audio, client, 5 more }

Startup configuration for a Live media session. Follow the Live prompting guide when writing frontend instructions and the backend prompt under delegation.responses.instructions.

MediaSessionForkConfig { client, delegation, store }

Optional overrides for a stored Live session. Omitted settings are inherited. The model, voice, frontend instructions, and prior conversation come from the stored session. WebRTC negotiates its audio format; audio.format is only supported on WebSocket forks.

OutputAudioDeltaEvent { delta, type, end_ms, start_ms }

An audio chunk generated by the Live model. Decode and play primary WebSocket chunks in delivery order using the configured session audio format. Sideband connections receive reflected output audio with timestamps.

OutputAudioFixturePlayEvent { audio, type, event_id }

Play one complete audio fixture through the session output. Send this event over the primary WebSocket or a server sideband. Audio must be raw mono PCM16 little-endian at 24 kHz, regardless of the session audio format, and at most 30 seconds. The entire payload is validated before model speech is interrupted and input is muted. Only one fixture can be active. fixture.playing acknowledges acceptance and fixture.played reports server mixer drain. These notifications are not exact transport-media boundaries. Input arriving during playback is dropped and not replayed.

OutputAudioFixturePlayedEvent { event_id, type, client_event_id }

The complete audio fixture has drained from the server mixer and normal inference has resumed. Transport buffers may still contain audio; this event does not prove listener-side completion. The optional client_event_id echoes the play command. This event is not emitted for canceled or failed playback.

OutputAudioFixturePlayingEvent { event_id, type, client_event_id }

The existing playback path has accepted the validated audio fixture. This acknowledgment does not identify when the listener receives the first sample. The optional client_event_id echoes the play command.

OutputTranscriptDeltaEvent { delta, end_ms, event_id, 3 more }

A transcript fragment for assistant output audio in the Live session. Accumulate fragments in delivery order; these events do not define complete turns or include a transcript-done event.

ResponseCreateEvent { type, event_id }

Request a response from the Live session’s Responses backend, or continue a delegated response waiting for tool results. Requires Responses delegation.

ResponseEvent { event, event_id, type, 2 more }

A streaming Responses API event from a backend delegated to by the Live session. Use the outer delegation_id to associate the nested stream with its Live delegation.

ResponseItemCreateEvent { item, type, event_id }

Add an input item to the Live session’s Responses backend. Requires Responses delegation; use response.create to request a response.

ResponsesDelegationConfig { model, instructions, max_output_tokens, 6 more }

Model, prompt, and tool settings for tasks delegated by the Live session to a Responses backend.

ResponsesDelegationUpdateConfig { instructions, max_output_tokens, model, 6 more }

Updates to the Responses backend of an existing Live session. Omitted settings retain their current values.

ServerEvent = SessionStartedEvent { event_id, session, type, client_event_id } | SessionUpdatedEvent { event_id, session, type, client_event_id } | InputAudioMutedEvent { event_id, type, client_event_id } | 25 more

Server events for Live. Response lifecycle events are wrapped inside response.event; dispatch the nested event by its full type and tolerate new response event types. Follow the Live prompting guide when designing the conversation and delegation policy.

ServerEventSelector { type, response_event }

A Live server event selector for the WebRTC frontend data channel.

SessionCloseEvent { type, event_id }

Request that the Live session close. The terminal session.closed event contains the close reason and final usage.

SessionClosedEvent { event_id, reason, session, 3 more }

Returned after the Live session finishes finalizing, with the close reason, final session snapshot, and cumulative audio usage. A connection closing without this event does not confirm successful finalization.

SessionConfig { model, audio, client, 5 more }

Initial configuration for a Live session, including its model, conversation instructions, audio, and delegated task handling.

SessionResource { id, expires_at, model, 8 more }

The resolved Live session configuration and server-assigned session metadata.

SessionStartEvent { session, type, event_id }

Start a Live session on a primary WebSocket. Send this event before other commands and wait for session.started.

SessionStartedEvent { event_id, session, type, client_event_id }

Returned when a Live session has started. Contains the resolved session configuration, including server defaults.

SessionUpdateConfig { delegation, triggers }

Changes to an active Live session’s delegation settings and transcript triggers.

SessionUpdateEvent { session, type, event_id }

Update the delegation settings or transcript triggers of an active Live session. The server acknowledges accepted changes with session.updated.

SessionUpdatedEvent { event_id, session, type, client_event_id }

Returned when a Live session update is accepted. Contains the resolved session configuration after the update.

SessionUsage { seconds }

Cumulative audio duration for a Live session. Values are totals for the session, not increments to sum across usage events.

SessionUsageUpdatedEvent { event_id, type, usage, 2 more }

Reports cumulative Live audio usage and, when available, the most recent context-window usage. Delegated Responses token usage is reported separately in response.event events.

ThinkingAppendEvent { content, delegation_id, type, event_id }

Provide silent reasoning or progress context to the Live model, optionally for an existing client delegation. Rejected while an audio fixture is active.

ThinkingAppendedEvent { end_ms, event_id, start_ms, 2 more }

Returned when a session.thinking.append command is accepted into the Live session timeline. Acknowledges the added reasoning context without guaranteeing any spoken output.

LiveForks

Connect
WebSocket
/live/sessions/{session_id}/fork
Models
ForkClientEvent = ForkSessionStartEvent { session, type, event_id } | SessionUpdateEvent { session, type, event_id } | InputAudioAppendEvent { audio, type, event_id } | 12 more

Client events for a Live fork WebSocket. First send session.start with an overrides object (which may be empty), then wait for session.started before sending other commands. The model and conversation are inherited from the stored session.

ForkServerEvent = SessionStartedEvent { event_id, session, type, client_event_id } | SessionUpdatedEvent { event_id, session, type, client_event_id } | InputAudioMutedEvent { event_id, type, client_event_id } | 25 more

Server events for Live. Response lifecycle events are wrapped inside response.event; dispatch the nested event by its full type and tolerate new response event types. Follow the Live prompting guide when designing the conversation and delegation policy.

LiveSessions

Accept call
client.live.sessions.accept(sessionID, body, options?): void
POST
/live/sessions/{session_id}/accept
Download recording
client.live.sessions.downloadRecording(sessionID, options?): Response
GET
/live/sessions/{session_id}/content
Fork session
client.live.sessions.fork(sessionID, body, options?): SessionForkResponse { session, transport }
POST
/live/sessions/{session_id}/fork
Hang up session
client.live.sessions.hangup(sessionID, options?): void
POST
/live/sessions/{session_id}/hangup
Transfer call
client.live.sessions.refer(sessionID, body, options?): void
POST
/live/sessions/{session_id}/refer
Reject call
client.live.sessions.reject(sessionID, body, options?): void
POST
/live/sessions/{session_id}/reject
Models
SessionForkResponse { session, transport }

The created Live session identifier and WebRTC answer. Apply transport.sdp as the peer’s remote answer and wait for session.started on the data channel before sending commands.

LiveSideband

Connect
WebSocket
/live/sessions/{session_id}/attach
Models
ConnectClientEvent = SessionUpdateEvent { session, type, event_id } | InputAudioMuteEvent { type, event_id } | InputAudioUnmuteEvent { type, event_id } | 11 more

Client events accepted by an attached Live sideband WebSocket. The session has already started. Send input audio over the primary connection.

ConnectServerEvent = SessionStartedEvent { event_id, session, type, client_event_id } | SessionUpdatedEvent { event_id, session, type, client_event_id } | InputAudioMutedEvent { event_id, type, client_event_id } | 25 more

Server events received by an attached Live sideband WebSocket, including reflected input and output audio and transport notifications.