Connect
The Realtime API enables you to build low-latency, multi-modal conversational experiences. It currently supports text and audio as both input and output, as well as function calling.
Some notable benefits of the API include:
- Native speech-to-speech: Skipping an intermediate text format means low latency and nuanced output.
- Natural, steerable voices: The models have natural inflection and can laugh, whisper, and adhere to tone direction.
- Simultaneous multimodal output: Text is useful for moderation; faster-than-realtime audio ensures stable playback.
The Realtime API is a stateful, event-based API that communicates over a WebSocket.
To start a Realtime conversation, set model in the connection URL. To join an existing call, set call_id instead. Transcription sessions use intent=transcription.
WebSocket wss://api.openai.com/v1/realtime
Connect
require "openai"
require "async"
client = OpenAI::Client.new
# Run the deadline in the same scheduler as the WebSocket and its cleanup.
Sync do |task|
task.with_timeout(60) do
# The connection block closes the socket on completion or error.
client.realtime.connect(model: "gpt-realtime") do |connection|
connection.session.update(type: :realtime, output_modalities: [:text])
connection.conversation.items.create(
type: :message,
role: :user,
content: [{type: :input_text, text: "Say hello in one sentence."}]
)
connection.response.create
# A partial answer or an early close must not count as success.
completed = false
connection.each do |event|
case event.type.to_s
when "response.output_text.delta"
print(event.delta)
$stdout.flush
when "response.done"
unless event.response.status == :completed
raise "Realtime response ended with #{event.response.status.inspect}"
end
completed = true
puts
break
when "error"
raise "Realtime request failed: #{event.error.message}"
end
end
raise "Connection closed before response.done" unless completed
end
end
end