Responses API 支援 WebSocket 模式,適合執行時間長且需要頻繁呼叫工具的工作流程。除了降低延遲,還可以透過 stream_id 在 WebSocket 連線上進行多工處理:只要持續開啟一條連至 /v1/responses 的連線,就能平行執行多個對話,也能從現有對話建立分支,在新的串流上繼續。接續每一輪時,只需傳送新增的輸入項目和 previous_response_id。
WebSocket 模式支援零資料保留(ZDR),也支援 store=false。
當工作流程需要模型與工具反覆往返時,WebSocket 模式最能發揮效益,例如智慧體式程式碼編寫,或反覆呼叫工具的編排迴圈。
由於連線會保持開啟,而且每一輪只傳送新增的輸入,WebSocket 模式能減少接續每一輪的額外開銷,降低整條長回應鏈的端對端延遲。在包含 20 次以上工具呼叫的執行過程中,我們觀察到端對端執行速度最多可提升約 40%。
依使用的語言安裝 WebSocket 相依套件。Python 請執行 pip install "openai[realtime]>=3.8.0",JavaScript 請執行 npm install openai@^7.10.0 ws,Ruby 請執行 gem install openai async-websocket。
若使用 Go,請執行 go get github.com/openai/openai-go/v3@v3.98.0。
若使用 Java,請加入 Maven 相依套件 com.openai:openai-java:4.101.0,並使用 Java 17 或更新版本執行這些範例。
上述 Go 和 Java SDK 版本皆原生支援 Responses WebSocket。
在 WebSocket 模式中,每一輪都由用戶端傳送 response.create 事件來啟動。事件承載的資料與一般的 Responses 建立請求主體 相同,但不使用 stream 和 background 等傳輸方式專用的欄位。
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43 import OpenAI from "openai" ;
import { ResponsesWS } from "openai/resources/responses/ws" ;
const client = new OpenAI ();
const ws = new ResponsesWS (client);
try {
ws. send ({
type: "response.create" ,
stream_id: "main" ,
model: "gpt-6-astra" ,
store: false ,
input: [
{
type: "message" ,
role: "user" ,
content: [{ type: "input_text" , text: "Find fizz_buzz()" }],
},
],
tools: [],
});
let completed = false ;
for await ( const event of ws) {
if (event.type === "error" ) throw event.error;
if (event.type !== "message" ) continue ;
const message = event.message;
if (message.type === "response.output_text.delta" ) {
process.stdout. write (message.delta);
} else if (message.type === "response.completed" ) {
completed = true ;
break ;
} else if (
message.type === "response.failed" ||
message.type === "response.incomplete"
) {
throw new Error ( JSON . stringify (message));
}
}
if ( ! completed)
throw new Error ( "Connection closed before the response finished." );
} finally {
ws. close ();
} 1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26 from openai import OpenAI
client = OpenAI()
with client.responses.connect() as connection:
connection.response.create(
stream_id="main",
model="gpt-6-astra",
store=False,
input=[
{
"type": "message",
"role": "user",
"content": [{"type": "input_text", "text": "Find fizz_buzz()"}],
}
],
tools=[],
)
for event in connection:
if event.type == "response.completed":
print(event.response.output_text)
break
if event.type in {"response.failed", "response.incomplete", "error"}:
raise RuntimeError(event.to_json())
else:
raise RuntimeError("Connection closed before the response finished.") 1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27 ctx, cancel := context.WithTimeout(context.Background(), 2*time.Minute)
defer cancel()
client := openai.NewClient()
conn, err := client.Responses.Connect(ctx, responses.ResponseConnectionOptions{})
if err != nil {
log.Fatal(err)
}
defer conn.Close()
if err := conn.Create(ctx, responses.ResponsesClientEventResponseCreateParam{
Model: "gpt-6-astra",
Store: openai.Bool(false),
Input: responses.ResponsesClientEventResponseCreateInputUnionParam{
OfString: openai.String("Find fizz_buzz()"),
},
StreamID: openai.String("main"),
Tools: []responses.ToolUnionParam{},
}); err != nil {
log.Fatal(err)
}
response, err := conn.FinalResponse(ctx)
if err != nil {
log.Fatal(err)
}
if response.Status != responses.ResponseStatusCompleted {
log.Fatalf("Response ended with status %s", response.Status)
}
fmt.Println(response.OutputText()) 1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28 import com.openai.client.okhttp.OpenAIOkHttpClient;
import com.openai.models.responses.*;
import java.util.*;
var client = OpenAIOkHttpClient.fromEnv();
try (var conn = client.responses().connect()) {
conn.send(
ResponsesClientEvent.ofResponseCreate(
ResponsesClientEvent.ResponseCreate.builder()
.model("gpt-6-astra")
.store(false)
.input("Find fizz_buzz()")
.streamId("main")
.tools(List.of())
.build()));
var response = conn.finalResponse();
if (response.status().filter(ResponseStatus.COMPLETED::equals).isEmpty())
throw new IllegalStateException(
"Response ended with status " + response.status().orElse(null));
response.output().stream()
.flatMap(item -> item.message().stream())
.flatMap(message -> message.content().stream())
.flatMap(content -> content.outputText().stream())
.forEach(text -> System.out.println(text.text()));
} finally {
client.close();
} 1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31 require "async"
require "openai"
def wait_for_response(connection)
while (event = connection.receive)
case event.type.to_s
when "response.completed" then return event.response
when "response.failed", "response.incomplete", "error"
raise "Response failed: #{event.to_json}"
end
end
raise "Connection closed before the response finished"
end
client = OpenAI::Client.new
Sync do |task|
task.with_timeout(120) do
client.responses.connect(request_options: { timeout: 10 }) do |connection|
connection.response.create(
stream_id: "main", model: "gpt-6-astra", store: false,
input: [
{
role: "user",
content: "Find fizz_buzz()"
}
], tools: []
)
puts(wait_for_response(connection).output_text)
end
end
end
用戶端也可以傳送 response.create 並指定 generate: false,預先準備請求狀態。如果你已經知道接下來某一輪要傳送哪些工具、指令或自訂訊息,就可以採用這個做法。指定 generate: false 時,服務不會傳回模型輸出,但會準備好請求狀態,讓下一輪更快開始產生回應。預先準備狀態的請求會傳回一個回應 ID。你可以將 previous_response_id 設為這個 ID,從該回應接續,也可以在回應鏈後續的回合這麼做。下一節說明如何使用 previous_response_id 和增量輸入接續工作階段。
若要在回應仍在產生時加入使用者指令,請使用回合中引導 。這項功能會保留已完成的工作,並在接續執行時納入新指令。若要在回合之間接續對話或傳送工具結果,請依照下列範例使用 response.create。
若要接續執行,請再傳送一個 response.create 事件,並設定下列欄位:
將 previous_response_id 設為前一個回應的 ID。
在 input 中只放入新增的項目,例如工具輸出和下一則使用者訊息。
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79 import OpenAI from "openai" ;
import { ResponsesWS } from "openai/resources/responses/ws" ;
const client = new OpenAI ();
const model = "gpt-6-astra" ;
const tools : OpenAI . Responses . Tool [] = [
{
type: "function" ,
name: "get_test_results" ,
description: "Return a local demo test result." ,
parameters: { type: "object" , properties: {}, additionalProperties: false },
strict: true ,
},
];
async function waitForResponse ( ws : ResponsesWS ) {
for await ( const event of ws) {
if (event.type === "error" ) throw event.error;
if (event.type !== "message" ) continue ;
const message = event.message;
if (message.type === "response.output_text.delta" ) {
process.stdout. write (message.delta);
} else if (message.type === "response.completed" ) {
return message.response;
} else if (
message.type === "response.failed" ||
message.type === "response.incomplete"
) {
throw new Error ( JSON . stringify (message));
}
}
throw new Error ( "Connection closed before the response finished." );
}
const ws = new ResponsesWS (client);
try {
ws. send ({
type: "response.create" ,
stream_id: "main" ,
model,
store: false ,
input: "Find the failing test and suggest a fix." ,
tools,
tool_choice: { type: "function" , name: "get_test_results" },
parallel_tool_calls: false ,
});
const first = await waitForResponse (ws);
const call = first.output. find (( item ) => item.type === "function_call" );
if ( ! call || call.name !== "get_test_results" ) {
throw new Error ( "Expected a get_test_results function call." );
}
const result = {
test: "test_fizz_buzz" ,
failure: 'Expected "FizzBuzz" for 15, got "Fizz".' ,
};
// Continue on the same socket with the actual response and tool-call IDs.
ws. send ({
type: "response.create" ,
stream_id: "main" ,
model,
store: false ,
previous_response_id: first.id,
input: [
{
type: "function_call_output" ,
call_id: call.call_id,
output: JSON . stringify (result),
},
{ role: "user" , content: "Now optimize it." },
],
tools,
tool_choice: "none" ,
});
await waitForResponse (ws);
} finally {
ws. close ();
} 1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74 import json
from openai import OpenAI
from openai.resources.responses.responses import ResponsesConnection
from openai.types.responses import FunctionToolParam, Response
client = OpenAI()
model = "gpt-6-astra"
tools: list[FunctionToolParam] = [
{
"type": "function",
"name": "get_test_results",
"description": "Read the demo test results.",
"parameters": {
"type": "object",
"properties": {},
"required": [],
"additionalProperties": False,
},
"strict": True,
}
]
def get_test_results():
# Demo data. Replace this function with your test runner.
return {
"test": "test_fizz_buzz",
"failure": 'Expected "FizzBuzz" for 15, got "Fizz".',
}
def wait_for_response(connection: ResponsesConnection) -> Response:
for event in connection:
if event.type == "response.completed":
return event.response
if event.type in {"response.failed", "response.incomplete", "error"}:
raise RuntimeError(event.to_json())
raise RuntimeError("Connection closed before the response finished.")
with client.responses.connect() as connection:
connection.response.create(
stream_id="main",
model=model,
store=False,
input="Find the failing test and suggest a fix.",
tools=tools,
tool_choice={"type": "function", "name": "get_test_results"},
parallel_tool_calls=False,
)
response = wait_for_response(connection)
call = next(item for item in response.output if item.type == "function_call")
if call.name != "get_test_results" or json.loads(call.arguments) != {}:
raise ValueError("Expected a get_test_results call with no arguments")
# Continue on the same connection using the actual response and tool-call IDs.
connection.response.create(
stream_id="main",
model=model,
store=False,
previous_response_id=response.id,
input=[
{
"type": "function_call_output",
"call_id": call.call_id,
"output": json.dumps(get_test_results()),
},
{"role": "user", "content": "Now optimize it."},
],
tools=tools,
tool_choice="none",
)
print(wait_for_response(connection).output_text) 1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91 ctx, cancel := context.WithTimeout(context.Background(), 2*time.Minute)
defer cancel()
client := openai.NewClient()
tools := []responses.ToolUnionParam{
{
OfFunction: &responses.FunctionToolParam{
Name: "get_test_results",
Description: openai.String("Return a local demo test result."),
Parameters: map[string]any{
"type": "object",
"properties": map[string]any{},
"additionalProperties": false,
},
Strict: openai.Bool(true),
},
},
}
conn, err := client.Responses.Connect(ctx, responses.ResponseConnectionOptions{})
if err != nil {
log.Fatal(err)
}
defer conn.Close()
if err := conn.Create(ctx, responses.ResponsesClientEventResponseCreateParam{
Model: "gpt-6-astra",
Store: openai.Bool(false),
Input: responses.ResponsesClientEventResponseCreateInputUnionParam{
OfString: openai.String("Find the failing test and suggest a fix."),
},
StreamID: openai.String("main"),
Tools: tools,
ParallelToolCalls: openai.Bool(false),
ToolChoice: responses.ResponsesClientEventResponseCreateToolChoiceUnionParam{
OfFunctionTool: &responses.ToolChoiceFunctionParam{
Name: "get_test_results",
},
},
}); err != nil {
log.Fatal(err)
}
first, err := conn.FinalResponse(ctx)
if err != nil {
log.Fatal(err)
}
if first.Status != responses.ResponseStatusCompleted {
log.Fatalf("Response ended with status %s", first.Status)
}
var callID string
for _, item := range first.Output {
if item.Type == "function_call" && item.Name == "get_test_results" {
callID = item.CallID
break
}
}
if callID == "" {
log.Fatal("Expected a get_test_results function call")
}
// The result is a local demo fixture; carry the real response and call IDs.
result := `{"test":"test_fizz_buzz","failure":"Expected FizzBuzz for 15, got Fizz."}`
if err := conn.Create(ctx, responses.ResponsesClientEventResponseCreateParam{
Model: "gpt-6-astra",
StreamID: openai.String("main"),
Store: openai.Bool(false),
PreviousResponseID: openai.String(first.ID),
Tools: tools,
ToolChoice: responses.ResponsesClientEventResponseCreateToolChoiceUnionParam{
OfToolChoiceMode: openai.Opt(responses.ToolChoiceOptionsNone),
},
Input: responses.ResponsesClientEventResponseCreateInputUnionParam{
OfResponse: &responses.ResponseInputParam{
{
OfFunctionCallOutput: &responses.ResponseInputItemFunctionCallOutputParam{
CallID: openai.String(callID),
Output: responses.ResponseInputItemFunctionCallOutputOutputUnionParam{
OfString: openai.String(result),
},
},
},
responses.ResponseInputItemParamOfMessage("Now optimize it.", responses.EasyInputMessageRoleUser),
},
},
}); err != nil {
log.Fatal(err)
}
response, err := conn.FinalResponse(ctx)
if err != nil {
log.Fatal(err)
}
if response.Status != responses.ResponseStatusCompleted {
log.Fatalf("Response ended with status %s", response.Status)
}
fmt.Println(response.OutputText()) 1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78 import com.openai.client.okhttp.OpenAIOkHttpClient;
import com.openai.core.JsonValue;
import com.openai.models.responses.*;
import java.util.*;
var client = OpenAIOkHttpClient.fromEnv();
var tool =
FunctionTool.builder()
.name("get_test_results")
.description("Return a local demo test result.")
.strict(true)
.parameters(
FunctionTool.Parameters.builder()
.putAdditionalProperty("type", JsonValue.from("object"))
.putAdditionalProperty("properties", JsonValue.from(Map.of()))
.putAdditionalProperty("additionalProperties", JsonValue.from(false))
.build())
.build();
try (var conn = client.responses().connect()) {
conn.send(
ResponsesClientEvent.ofResponseCreate(
ResponsesClientEvent.ResponseCreate.builder()
.model("gpt-6-astra")
.store(false)
.input("Find the failing test and suggest a fix.")
.streamId("main")
.addTool(tool)
.parallelToolCalls(false)
.toolChoice(ToolChoiceFunction.builder().name("get_test_results").build())
.build()));
var first = conn.finalResponse();
if (first.status().filter(ResponseStatus.COMPLETED::equals).isEmpty())
throw new IllegalStateException(
"Response ended with status " + first.status().orElse(null));
var call =
first.output().stream()
.flatMap(item -> item.functionCall().stream())
.filter(item -> item.name().equals("get_test_results"))
.findFirst()
.orElseThrow(() -> new IllegalStateException("Expected get_test_results call"));
// Use the real response and call IDs with a local demo result.
var result =
"{\"test\":\"test_fizz_buzz\",\"failure\":\"Expected FizzBuzz for 15, got Fizz.\"}";
conn.send(
ResponsesClientEvent.ofResponseCreate(
ResponsesClientEvent.ResponseCreate.builder()
.model("gpt-6-astra")
.streamId("main")
.store(false)
.previousResponseId(first.id())
.addTool(tool)
.toolChoice(ToolChoiceOptions.NONE)
.inputOfResponse(
List.of(
ResponseInputItem.ofFunctionCallOutput(
ResponseInputItem.FunctionCallOutput.builder()
.callId(call.callId())
.output(result)
.build()),
ResponseInputItem.ofEasyInputMessage(
EasyInputMessage.builder()
.role(EasyInputMessage.Role.USER)
.content("Now optimize it.")
.build())))
.build()));
var response = conn.finalResponse();
if (response.status().filter(ResponseStatus.COMPLETED::equals).isEmpty())
throw new IllegalStateException(
"Response ended with status " + response.status().orElse(null));
response.output().stream()
.flatMap(item -> item.message().stream())
.flatMap(message -> message.content().stream())
.flatMap(content -> content.outputText().stream())
.forEach(text -> System.out.println(text.text()));
} finally {
client.close();
} 1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73 require "async"
require "openai"
require "json"
def wait_for_response(connection)
while (event = connection.receive)
case event.type.to_s
when "response.completed" then return event.response
when "response.failed", "response.incomplete", "error"
raise "Response failed: #{event.to_json}"
end
end
raise "Connection closed before the response finished"
end
tools = [
{
type: "function",
name: "get_test_results",
description: "Read the demo test results.",
parameters: {
type: "object",
properties: {},
required: [],
additionalProperties: false
},
strict: true
}
]
client = OpenAI::Client.new
Sync do |task|
task.with_timeout(120) do
client.responses.connect(request_options: { timeout: 10 }) do |connection|
connection.response.create(
stream_id: "main", model: "gpt-6-astra", store: false,
input: "Find the failing test and suggest a fix.", tools: tools,
tool_choice: {
type: "function",
name: "get_test_results"
}, parallel_tool_calls: false
)
response = wait_for_response(connection)
call = response.output.grep(OpenAI::Responses::ResponseFunctionToolCall).first
unless call && call.name == "get_test_results" && JSON.parse(call.arguments) == {}
raise "Expected a get_test_results call with no arguments"
end
# Demo data. Replace this with your test runner.
result = {
test: "test_fizz_buzz",
failure: 'Expected "FizzBuzz" for 15, got "Fizz".'
}
connection.response.create(
stream_id: "main", model: "gpt-6-astra", store: false,
previous_response_id: response.id,
input: [
{
type: "function_call_output",
call_id: call.call_id,
output: JSON.generate(result)
},
{
role: "user",
content: "Now optimize it."
}
],
tools: tools, tool_choice: "none"
)
puts(wait_for_response(connection).output_text)
end
end
end
WebSocket 模式使用 previous_response_id 串接回應的方式與 HTTP 模式相同,但可以透過仍開啟的連線,以較低的延遲接續回應。
WebSocket 連線開啟期間,服務會將近期回應的狀態保存在該連線專屬的記憶體快取中。使用 stream_id 時,每個通道都會快取自己的最新回應。從該回應接續時,服務可以重用這條連線上的狀態,加快接續速度。服務只在記憶體中保留先前的回應狀態,不會將其寫入磁碟,因此 WebSocket 模式可搭配 store=false 和零資料保留(ZDR)使用。
如果記憶體快取中找不到 previous_response_id,服務的行為取決於你是否儲存回應:
使用 store=true 時,如果仍有已儲存的狀態,服務可能會從中載入較舊回應 ID 對應的狀態。此時仍可能接續回應,但無法享有記憶體快取帶來的低延遲。
使用 store=false(含 ZDR)時,沒有已儲存的狀態可供還原。如果快取中沒有該 ID,請求會傳回 previous_response_not_found。
如果在同一通道上接續回應時傳回 4xx 或 5xx,服務會從該連線的快取中移除 previous_response_id 所指的回應。如果跨通道建立分支時傳回錯誤,服務會保留共用的父回應,讓來源通道可以繼續執行。
使用壓縮時,有兩種不同的接續方式:
伺服器端壓縮(context_management)
啟用伺服器端壓縮後(在 context_management 中指定 compact_threshold),壓縮會在 /responses 正常產生回應的過程中進行。在 WebSocket 模式下,照常接續即可:傳送下一個 response.create 事件,帶上最新的 previous_response_id,並只提供新增的輸入項目。
獨立的 /responses/compact 端點 會傳回壓縮後的新輸入視窗,而非回應 ID。壓縮完成後,請將該視窗連同下一批使用者或工具項目放入 input,在 WebSocket 連線上建立新回應。
省略 previous_response_id 或將其設為 null,即可開始新的回應鏈。請原樣傳入壓縮後的輸出,不要刪減傳回的視窗內容。
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47 import { toResponseInputItems } from "openai/lib/responses/ResponseInputItems" ;
// Compact your current window with an HTTP request.
const compacted = await client.responses. compact ({
model: "gpt-6-astra" ,
input: longInputItems,
});
const nextInput = toResponseInputItems (compacted.output);
nextInput. push ({
type: "message" ,
role: "user" ,
content: [{ type: "input_text" , text: "Continue from here." }],
});
// Start a new response on the WebSocket using the compacted window.
const ws = new ResponsesWS (client);
try {
ws. send ({
type: "response.create" ,
stream_id: "main" ,
model: "gpt-6-astra" ,
store: false ,
input: nextInput,
tools: [],
});
let completed = false ;
for await ( const event of ws) {
if (event.type === "error" ) throw event.error;
if (event.type !== "message" ) continue ;
const message = event.message;
if (message.type === "response.output_text.delta" ) {
process.stdout. write (message.delta);
} else if (message.type === "response.completed" ) {
completed = true ;
break ;
} else if (
message.type === "response.failed" ||
message.type === "response.incomplete"
) {
throw new Error ( JSON . stringify (message));
}
}
if ( ! completed)
throw new Error ( "Connection closed before the response finished." );
} finally {
ws. close ();
} 1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39 from typing import cast
from openai import OpenAI
from openai.types.responses import ResponseInputParam
# Compact your current window (HTTP call).
compacted = client.responses.compact(
model="gpt-6-astra",
input=long_input_items_array,
)
next_input = cast(
ResponseInputParam,
[item.to_dict() for item in compacted.output],
)
next_input.append(
{
"type": "message",
"role": "user",
"content": [{"type": "input_text", "text": "Continue from here."}],
}
)
# Start a new response on the WebSocket using the compacted window.
with client.responses.connect() as connection:
connection.response.create(
stream_id="main",
model="gpt-6-astra",
store=False,
input=next_input,
tools=[],
)
for event in connection:
if event.type == "response.completed":
print(event.response.output_text)
break
if event.type in {"response.failed", "response.incomplete", "error"}:
raise RuntimeError(event.to_json())
else:
raise RuntimeError("Connection closed before the response finished.") 1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41 require "async"
require "openai"
def wait_for_response(connection)
while (event = connection.receive)
case event.type.to_s
when "response.completed" then return event.response
when "response.failed", "response.incomplete", "error"
raise "Response failed: #{event.to_json}"
end
end
raise "Connection closed before the response finished"
end
client = OpenAI::Client.new
compacted = client.responses.compact(
model: "gpt-6-astra",
input: [
{
role: :user,
content: "Find the failing test."
}
]
)
next_input = compacted.output.map(&:to_h)
next_input << {
role: :user,
content: "Continue from here."
}
Sync do |task|
task.with_timeout(120) do
client.responses.connect(request_options: { timeout: 10 }) do |connection|
connection.response.create(
stream_id: "main", model: "gpt-6-astra", store: false,
input: next_input, tools: []
)
puts(wait_for_response(connection).output_text)
end
end
end
使用 stream_id 參數,就能在同一條連線上維持多個平行對話。請接連傳送彼此獨立的 response.create 事件,並分別指定不同的 stream_id 值。伺服器可以在同一條連線上同時執行這些請求。由於各個對話的事件可能交錯抵達,請使用單一迴圈讀取事件,再依 stream_id 分派每個事件。
stream_id 用來識別同一條 WebSocket 連線上依序處理請求的通道。請區分 stream_id 和 previous_response_id 的用途:
stream_id 決定事件要送往哪個通道,以及哪些請求要依先進先出的順序執行。
previous_response_id 決定對話要從哪個回應接續。
兩者分工後,就能在同一條連線上平行執行獨立對話,也能從既有對話建立分支。
一條 WebSocket 連線
├─ stream_id="planner" 擬定部署計畫
└─ stream_id="research" 列出部署風險
具有相同 stream_id 的請求會依先進先出的順序執行,執行時間不會重疊。具有不同 stream_id 值的請求則可以同時執行。
在同一條連線上,具名通道與預設通道合計最多可同時有 16 個進行中的回應。連線仍會接受更多 response.create 事件,並將其排入佇列,直到有進行中的回應完成。
每條連線最多接受 32 個不同的具名 stream_id 值。隱含的預設通道不計入具名串流的上限。達到上限後,請重用現有的 stream_id 或開啟新連線。
若要從已完成的回應建立分支,請將該回應的 ID 作為 previous_response_id,連同新的 stream_id 一起傳送。只要該回應仍可供使用,新串流就能繼承其上下文,原本的串流也可以繼續執行。分支啟動後,兩個分支使用不同的串流 ID,因此可以同時執行。
使用 store=false(含 ZDR)時,只有在該連線的快取中仍有父回應的情況下,才能跨通道建立分支。如果建立分支的請求仍在佇列中等候,而來源通道在此期間進入下一輪或發生錯誤,父回應可能會在分支啟動前從快取中移除,此時建立分支的請求會傳回 previous_response_not_found。請等到分支通道發出 response.in_progress 事件,再讓來源通道進入下一輪。你也可以在重試時將 previous_response_id 設為 null,並重新傳送完整的輸入上下文。
main: resp_1 ──▶ resp_2 ──▶ resp_3
╲
critic: resp_4 ──▶ resp_5
重用 stream_id 但未提供 previous_response_id 時,會建立新的回應,不會接續原本的對話。
主要呼叫如下:
# 在同一條連線上執行兩個獨立的對話。
send_create(connection, "planner", "擬定部署計畫。")
send_create(connection, "research", "列出部署風險。")
# 從 planner 的回應建立分支,再接續原本的分支,讓兩者平行執行。
send_create(
connection,
"critic",
"找出這份計畫的疏漏。",
previous_response_id=planner_response_id,
)
wait_for_in_progress(connection, "critic")
send_create(
connection,
"planner",
"補上回復至先前版本的步驟。",
previous_response_id=planner_response_id,
)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120 import OpenAI from "openai" ;
import type { ResponsesStreamMessage } from "openai/resources/responses/internal-base.mjs" ;
import { ResponsesWS } from "openai/resources/responses/ws" ;
const client = new OpenAI ();
const latestResponseIdByLane = new Map ();
function sendCreate (
ws : ResponsesWS ,
streamId : string ,
text : string ,
previousResponseId = latestResponseIdByLane. get (streamId)
) {
ws. send ({
type: "response.create" ,
stream_id: streamId,
model: "gpt-6-astra" ,
store: false ,
input: [
{
type: "message" ,
role: "user" ,
content: [{ type: "input_text" , text }],
},
],
previous_response_id: previousResponseId,
});
}
async function readMessage (
events : AsyncIterableIterator < ResponsesStreamMessage >
) {
while ( true ) {
const { value : event , done } = await events. next ();
if (done)
throw new Error ( "Connection closed before all responses finished." );
if (event.type === "error" ) throw event.error;
if (event.type !== "message" ) continue ;
const message = event.message;
if (
message.type === "response.failed" ||
message.type === "response.incomplete"
) {
throw new Error (
`Lane ${ message . stream_id } failed: ${ JSON . stringify ( message ) }`
);
}
return message;
}
}
async function drainUntilComplete (
events : AsyncIterableIterator < ResponsesStreamMessage >,
expectedStreamIds : Iterable < unknown > | null | undefined
) {
const remaining = new Set (expectedStreamIds);
while (remaining.size > 0 ) {
const message = await readMessage (events);
const streamId = message.stream_id;
if ( ! streamId || ! remaining. has (streamId)) continue ;
if (message.type === "response.completed" ) {
latestResponseIdByLane. set (streamId, message.response.id);
remaining. delete (streamId);
}
}
}
async function waitForInProgress (
events : AsyncIterableIterator < ResponsesStreamMessage >,
streamId : string
) {
while ( true ) {
const message = await readMessage (events);
if (
message.type === "response.in_progress" &&
message.stream_id === streamId
)
return ;
}
}
const ws = new ResponsesWS (client);
// Keep one iterator so events stay queued while moving between phases.
const events = ws. stream ();
try {
// Run two independent conversations in parallel.
sendCreate (
ws,
"planner" ,
"Draft a deployment plan for a stateless API service."
);
sendCreate (
ws,
"research" ,
"List common deployment risks for a stateless API service."
);
await drainUntilComplete (events, new Set ([ "planner" , "research" ]));
// Fork the planner conversation and continue its original branch in parallel.
const plannerResponseId = latestResponseIdByLane. get ( "planner" );
sendCreate (
ws,
"critic" ,
"Find gaps in this deployment plan." ,
plannerResponseId
);
// Let the fork load its parent before advancing the original lane's cache.
await waitForInProgress (events, "critic" );
sendCreate (
ws,
"planner" ,
"Add rollback and monitoring steps to the plan." ,
plannerResponseId
);
await drainUntilComplete (events, new Set ([ "critic" , "planner" ]));
} finally {
await events. return ?.();
ws. close ();
} 1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93 from openai import OpenAI
from openai.resources.responses.responses import ResponsesConnection
client = OpenAI()
latest_response_id_by_lane: dict[str, str] = {}
def send_create(
connection: ResponsesConnection,
stream_id: str,
text: str,
previous_response_id: str | None = None,
):
if previous_response_id is None:
previous_response_id = latest_response_id_by_lane.get(stream_id)
connection.response.create(
stream_id=stream_id,
model="gpt-6-astra",
store=False,
input=[
{
"type": "message",
"role": "user",
"content": [{"type": "input_text", "text": text}],
}
],
previous_response_id=previous_response_id,
)
def drain_until_complete(
connection: ResponsesConnection, expected_stream_ids: set[str]
):
completed: set[str] = set()
for event in connection:
stream_id = event.stream_id
if event.type == "error" and stream_id is None:
raise RuntimeError(f"Connection error: {event.to_json()}")
if stream_id is None or stream_id not in expected_stream_ids:
continue
if event.type == "response.completed":
latest_response_id_by_lane[stream_id] = event.response.id
completed.add(stream_id)
if completed == expected_stream_ids:
return
elif event.type in {"response.failed", "response.incomplete", "error"}:
raise RuntimeError(f"Lane {stream_id} failed: {event.to_json()}")
raise RuntimeError("Connection closed before all responses finished.")
def wait_for_in_progress(connection: ResponsesConnection, expected_stream_id: str):
for event in connection:
if event.type == "error" and event.stream_id is None:
raise RuntimeError(f"Connection error: {event.to_json()}")
if event.stream_id != expected_stream_id:
continue
if event.type == "response.in_progress":
return
if event.type in {"response.failed", "response.incomplete", "error"}:
raise RuntimeError(f"Lane {expected_stream_id} failed: {event.to_json()}")
raise RuntimeError("Connection closed before the fork started.")
with client.responses.connect() as connection:
# 1. Run two independent conversations in parallel.
send_create(
connection, "planner", "Draft a deployment plan for a stateless API service."
)
send_create(
connection,
"research",
"List common deployment risks for a stateless API service.",
)
drain_until_complete(connection, {"planner", "research"})
# 2. Fork the planner conversation and continue the original branch in parallel.
planner_response_id = latest_response_id_by_lane["planner"]
send_create(
connection,
"critic",
"Find gaps in this deployment plan.",
previous_response_id=planner_response_id,
)
# Let the fork bind its parent before advancing the original lane.
wait_for_in_progress(connection, "critic")
send_create(
connection,
"planner",
"Add rollback and monitoring steps to the plan.",
previous_response_id=planner_response_id,
)
drain_until_complete(connection, {"critic", "planner"}) 1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64 require "async"
require "openai"
require "json"
def send_create(connection, stream_id, text, previous_response_id = nil)
payload = {
stream_id: stream_id,
model: "gpt-6-astra",
store: false,
input: [
{
role: "user",
content: text
}
]
}
payload[:previous_response_id] = previous_response_id if previous_response_id
connection.response.create(**payload)
end
def read_event(connection)
event = connection.receive or raise "Connection closed before all responses finished"
if ["response.failed", "response.incomplete", "error"].include?(event.type.to_s)
raise "Response failed: #{event.to_json}"
end
event
end
def drain_responses(connection, lanes, latest_ids)
remaining = lanes.dup
until remaining.empty?
event = read_event(connection)
next unless event.type.to_s == "response.completed"
lane = event.stream_id
next unless remaining.include?(lane)
latest_ids[lane] = event.response.id
remaining.delete(lane)
end
end
client = OpenAI::Client.new
Sync do |task|
task.with_timeout(120) do
client.responses.connect(request_options: { timeout: 10 }) do |connection|
latest_ids = {}
send_create(connection, "planner", "Draft a deployment plan for a stateless API service.")
send_create(connection, "research", "List common deployment risks for a stateless API service.")
drain_responses(connection, ["planner", "research"], latest_ids)
parent_id = latest_ids.fetch("planner")
send_create(connection, "critic", "Find gaps in this deployment plan.", parent_id)
# Let the fork load its parent before advancing the original lane's cache.
loop do
event = read_event(connection)
break if event.type.to_s == "response.in_progress" && event.stream_id == "critic"
end
send_create(connection, "planner", "Add rollback and monitoring steps.", parent_id)
drain_responses(connection, ["critic", "planner"], latest_ids)
puts(JSON.generate(latest_ids))
end
end
end
stream_id 的長度必須介於 1 到 256 個字元之間,且只能包含字母、數字、底線(_)、連字號(-)和句點(.)。請只在 WebSocket 的 response.create 事件中使用這個欄位,不要在 HTTP POST /v1/responses 請求中加入它。
對於具名串流,伺服器事件會包含對應的 stream_id,結束事件和個別請求的錯誤事件也不例外。
如果省略 stream_id,請求會使用隱含的預設通道,其事件不會包含 stream_id。除此之外,預設通道的排序與並行規則與具名串流相同。空字串不是有效的 stream_id 值。若要選用預設通道,請省略這個欄位。
每個回應中的事件都遵循既有的 Responses 串流事件模型。不同通道的事件可能交錯出現。
具有相同 stream_id 的請求會依先進先出的順序執行,執行時間不會重疊。不同通道上的請求可以同時執行。
每條連線最多可維持 60 分鐘。達到上限時,請重新連線。
當連線關閉或達到 60 分鐘上限時,該連線上所有通道的快取都會消失。請開啟新的 WebSocket 連線,並依各通道的情況選用下列方式恢復執行:
如果先前已儲存回應(store=true),而且有有效的回應 ID,請使用 previous_response_id 和新增的輸入項目接續該通道的對話。
如果無法接續某個通道的對話(例如使用 store=false 或 ZDR,或收到 previous_response_not_found),請將 previous_response_id 設為 null(或省略該欄位)以建立新回應,並傳送該通道下一輪所需的完整輸入上下文。
如果已使用 /responses/compact 壓縮上下文,請以傳回的壓縮視窗作為新回應的基礎 input,再附加最新的使用者或工具項目。
如果伺服器能判定錯誤屬於哪個具名通道,錯誤事件就會包含 stream_id。個別請求發生錯誤後,其他通道仍可繼續執行。
previous_response_not_found
1 2 3 4 5 6 7 8 9 10 11 {
"type" : "error" ,
"status" : 400 ,
"stream_id" : "main" ,
"error" : {
"type" : "invalid_request_error" ,
"code" : "previous_response_not_found" ,
"message" : "Previous response with id 'resp_abc' not found." ,
"param" : "previous_response_id"
}
}
invalid_stream_id
1 2 3 4 5 6 7 8 9 10 {
"type" : "error" ,
"status" : 400 ,
"error" : {
"type" : "invalid_request_error" ,
"code" : "invalid_stream_id" ,
"message" : "The 'stream_id' field must be a non-empty string with at most 256 characters and may only contain letters, numbers, underscores, hyphens, and periods." ,
"param" : "stream_id"
}
}
websocket_stream_limit_reached
1 2 3 4 5 6 7 8 9 10 11 {
"type" : "error" ,
"status" : 400 ,
"stream_id" : "agent_33" ,
"error" : {
"type" : "invalid_request_error" ,
"code" : "websocket_stream_limit_reached" ,
"message" : "This WebSocket connection has reached its maximum number of distinct stream IDs (32). Reuse an existing stream_id or open a new WebSocket connection." ,
"param" : "stream_id"
}
}
websocket_connection_limit_reached
1 2 3 4 5 6 7 8 9 {
"type" : "error" ,
"error" : {
"type" : "invalid_request_error" ,
"code" : "websocket_connection_limit_reached" ,
"message" : "Responses websocket connection limit reached (60 minutes). Create a new websocket connection to continue."
},
"status" : 400
}