For the complete documentation index, see llms.txt. Markdown versions of documentation pages are available by appending .md to the page URL.
Primary navigation

Suggested

GPT-Live

Realtime API

Build with voice

Connections

Audio processing

API Dashboard
Try ChatGPT

WebRTC

Connect browser voice applications with WebRTC.

Choose the API your application uses. Each API has its own authentication, session creation, and event contract.

Connect a browser to GPT-Live

Use WebRTC for browser voice applications. Microphone input and generated speech travel on negotiated media tracks. A data channel carries JSON events for transcripts, session updates, and delegated work.

Your browser creates a Session Description Protocol (SDP) offer. Your application server exchanges it for an answer with POST /v1/live/sessions, using the project API key. Keep the key and session configuration on your trusted server.

Before you start

You need:

  • A project API key with access to GPT-Live.
  • A server runtime for your chosen SDK example. The Node.js example requires Node.js 22.6 or later.
  • A browser with microphone permission, running on HTTPS or localhost.

The example uses Responses delegation with gpt-5.6-terra and hosted web search. For backend instructions and application tools, see Delegation and tools. For voice and backend usage, see Cost optimization.

Understand the connection sequence

  1. Request microphone access from a user action and add its tracks to a peer connection.
  2. Create the data channel and register event listeners before creating the SDP offer.
  3. Set the local description, wait for ICE candidate gathering, and send the offer to your server.
  4. Have your server post JSON containing session and transport: { type: "webrtc", sdp: ... } to OpenAI.
  5. Apply the returned SDP answer as the remote description. Wait for session.started on the data channel before sending application commands.

The HTTP request starts the session. Do not send session.start on the data channel. The oai-events string in the example is the data-channel label.

Creating a WebRTC session with POST /v1/live/sessions bills 15 seconds of voice duration during initialization. That amount is credited against duration charges once the session starts running; it is not an extra 15 seconds added to the running session. See WebRTC initialization charges for cost accounting.

Create the application server

Save the server example in a new directory and set OPENAI_API_KEY in its environment. For Node.js, use server.mjs and install openai and express with npm install openai express. Install the corresponding OpenAI SDK for the other language variants; the Ruby example also uses webrick. This example binds to 127.0.0.1, accepts session requests from http://localhost:3000, and serves index.html from the directory where you run it.

Choose a server language below; each variant serves index.html and the same /api/session endpoint on port 3000. Use an SDK version with Live support. Run only one variant at a time.

import express from "express";
import OpenAI from "openai";
import { readFile } from "node:fs/promises";
import { resolve } from "node:path";

const app = express();
const client = new OpenAI({ maxRetries: 0 });
const port = 3000;
const origin = `http://localhost:${port}`;
const indexPath = resolve("index.html");

app.use(express.json({ limit: "64kb" }));
app.get("/", async (_request, response) => {
  response.type("html").send(await readFile(indexPath, "utf8"));
});

// Local-only demo. Add your application's authentication and authorization
// before exposing session creation to other users.
app.post("/api/session", async (request, response) => {
  if (request.headers.origin !== origin) {
    response.status(403).json({ error: "Unexpected request origin" });
    return;
  }
  if (typeof request.body?.sdp !== "string" || !request.body.sdp.trim()) {
    response.status(400).json({ error: "An SDP offer is required" });
    return;
  }
  if (!process.env.OPENAI_API_KEY) {
    response.status(503).json({ error: "Set OPENAI_API_KEY on the server" });
    return;
  }

  try {
    const result = await client.live.create({
      session: {
        model: "gpt-live-1",
        instructions:
          "Be concise. Delegate requests needing current information to the backend, which can search the web.",
        delegation: {
          type: "responses",
          responses: {
            model: "gpt-5.6-terra",
            instructions:
              "Use web search when current facts are needed. Return concise, grounded results for a spoken conversation.",
            tools: [{ type: "web_search" }],
            tool_choice: "auto",
          },
        },
      },
      transport: {
        type: "webrtc",
        sdp: request.body.sdp,
      },
    });
    // Preserve the SDK's typed session ID and SDP answer.
    response.status(201).json(result);
  } catch (error) {
    if (!(error instanceof OpenAI.APIError)) throw error;
    console.error("Live session creation failed", error.status);
    response
      .status(error.status ?? 502)
      .json({ error: "Live session creation failed" });
  }
});

app.listen(port, "127.0.0.1", () => console.log(`Open ${origin}`));

Before making the server accessible to other users, protect /api/session with your application’s authentication, authorization, request limits, and HTTPS. The origin check in this local example does not authenticate users.

Create the browser client

Create index.html in the directory where you run the server:

<!doctype html>
<html lang="en">
  <head>
    <meta charset="utf-8" />
    <meta name="viewport" content="width=device-width, initial-scale=1" />
    <title>GPT-Live connection</title>
  </head>
  <body>
    <script type="module">
      // Paste the browser code below here.
    </script>
  </body>
</html>

Paste the following code inside the module script. It adds start and end controls, connects the microphone and output audio, and handles session events. /api/session is a route on your application server.

const start = document.createElement("button");
start.textContent = "Start conversation";
const stop = document.createElement("button");
stop.textContent = "End conversation";
stop.disabled = true;
const status = document.createElement("p");
const audio = new Audio();
audio.autoplay = true;
audio.controls = true;
document.body.append(start, stop, status, audio);

/** @type {RTCPeerConnection | undefined} */
let peer;
/** @type {RTCDataChannel | undefined} */
let events;
/** @type {MediaStream | undefined} */
let microphone;
/** @type {ReturnType<typeof setTimeout> | undefined} */
let closeTimeout;
let ready = false;
let finalized = false;

function cleanup() {
  clearTimeout(closeTimeout);
  microphone?.getTracks().forEach((track) => track.stop());
  events?.close();
  peer?.close();
  audio.srcObject = null;
  ready = false;
  start.disabled = false;
  stop.disabled = true;
}

start.addEventListener("click", async () => {
  start.disabled = true;
  finalized = false;
  status.textContent = "Connecting…";
  try {
    const connection = new RTCPeerConnection();
    peer = connection;
    connection.addEventListener("track", (event) => {
      audio.srcObject = new MediaStream([event.track]);
      audio.play().catch(() => {
        status.textContent =
          "Select play on the audio controls to hear the assistant.";
      });
    });
    microphone = await navigator.mediaDevices.getUserMedia({ audio: true });
    for (const track of microphone.getAudioTracks()) {
      connection.addTrack(track, microphone);
    }

    // Create the event channel before creating the SDP offer.
    events = connection.createDataChannel("oai-events");
    events.addEventListener("message", ({ data }) => {
      /** @type {import("openai/resources/live/live").ServerEvent} */
      const event = JSON.parse(data);
      if (event.type === "session.started") {
        ready = true;
        stop.disabled = false;
        status.textContent = "Connected: " + event.session.id;
      } else if (event.type === "session.closed") {
        finalized = true;
        console.log("Final session usage", event.usage);
        status.textContent = "Conversation ended.";
        cleanup();
      } else {
        // Save transcript and nested Responses events as needed by your app.
        console.log(event);
      }
    });
    events.addEventListener("close", (event) => {
      if (event.target !== events) return;
      if (!finalized) {
        status.textContent = "Disconnected without final session usage.";
        cleanup();
      }
    });

    const offer = await connection.createOffer();
    await connection.setLocalDescription(offer);
    if (connection.iceGatheringState !== "complete") {
      await new Promise((resolve, reject) => {
        const timeout = setTimeout(() => {
          connection.removeEventListener("icegatheringstatechange", onState);
          reject(new Error("Timed out while gathering ICE candidates"));
        }, 10_000);
        function onState() {
          if (connection.iceGatheringState !== "complete") return;
          clearTimeout(timeout);
          connection.removeEventListener("icegatheringstatechange", onState);
          resolve(undefined);
        }
        connection.addEventListener("icegatheringstatechange", onState);
        onState();
      });
    }

    const sdp = connection.localDescription?.sdp;
    if (!sdp) throw new Error("Missing local SDP offer");
    const response = await fetch("/api/session", {
      method: "POST",
      headers: { "Content-Type": "application/json" },
      body: JSON.stringify({ sdp }),
    });
    if (!response.ok) throw new Error(await response.text());
    /** @type {import("openai/resources/live/live").LiveCreateResponse} */
    const result = await response.json();
    console.log("Created session", result.session.id);
    await connection.setRemoteDescription({
      type: "answer",
      sdp: result.transport.sdp,
    });
    // The HTTP request started this session. Do not send session.start here.
  } catch (error) {
    status.textContent =
      error instanceof Error ? error.message : String(error);
    cleanup();
  }
});

stop.addEventListener("click", () => {
  if (!ready || !events || events.readyState !== "open") return;
  stop.disabled = true;
  status.textContent = "Finishing the conversation…";
  // The session.closed handler is already registered. Keep media and events
  // alive while pending work drains; only clean up after the final event.
  events.send(JSON.stringify({ type: "session.close" }));
  closeTimeout = setTimeout(() => {
    status.textContent = "Incomplete finalization: no session.closed event.";
    cleanup();
  }, 15_000);
});

Run your chosen server (node server.mjs, python server.py, go run main.go, ruby server.rb, or the Java LiveConnectionWebrtcExample class), open http://localhost:3000, and select Start conversation. After the status changes to Connected, ask a question that needs current information to exercise hosted search. Use the audio controls if your browser blocks autoplay.

Read the session response

A successful request returns HTTP 201 with JSON containing the session ID and SDP answer:

{
  "session": { "id": "live_123" },
  "transport": { "type": "webrtc", "sdp": "<SDP answer>" }
}

Read result.session.id and pass result.transport.sdp to setRemoteDescription. Treat the session ID as opaque and preserve it unchanged, including its prefix.

Handle media and events

Send microphone audio and receive generated speech through the media tracks. WebRTC negotiates the audio format through SDP, so omit audio.format from session configuration. Do not send session.input_audio.append or expect session.output_audio.delta on the data channel.

Use the data channel for transcript deltas, session commands, and nested response.event messages. See Managing sessions for transcript handling and lifecycle events, and Server-side controls if your server needs its own event connection.

To end the conversation, send session.close and keep receiving until session.closed before closing the peer connection and microphone tracks. The example registers the final-event listener before sending the command. If the connection fails or times out first, final usage is unconfirmed. See Usage and graceful close for final usage handling.