如果事先知道許多輸出 Token,就可以使用預測輸出來加快 Chat Completions 的 API 回應。最常見的情境是重新產生只需小幅修改的文字檔或程式碼檔案。你可以透過 Chat Completions 的 prediction 請求參數提供預測內容。
目前,最新版的 gpt-4o、gpt-4o-mini、gpt-4.1、gpt-4.1-mini 和 gpt-4.1-nano 模型都支援預測輸出。以下說明如何使用預測輸出縮短應用程式的回應時間。
如果只需小幅修改文字文件或程式碼檔案,預測輸出特別適合用來重新產生整份檔案。假設你想請 GPT-4o 模型重構一段 JavaScript 程式碼,將 username 屬性在 User 類別中的名稱改為 email:
1
2
3
4
5
6
7class User {
firstName = "";
lastName = "";
username = "";
}
export default User;
除了上面的第 4 行,檔案其餘內容都會維持原樣。以程式碼檔案的現有內容作為預測文字,就能更快重新產生整份檔案。檔案越大,累計省下的時間就越多。
以下範例示範如何在 SDK 中使用 prediction 參數。由於預期模型的最終輸出會與原始程式碼檔案非常相似,因此直接以原始檔案的內容作為預測文字。
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41import OpenAI from "openai";
const code = `
class User {
firstName = "";
lastName = "";
username = "";
}
export default User;
`.trim();
const openai = new OpenAI();
const refactorPrompt = `
Replace the "username" property with an "email" property. Respond only
with code, and with no markdown formatting.
`;
const completion = await openai.chat.completions.create({
model: "gpt-4.1",
messages: [
{
role: "user",
content: refactorPrompt,
},
{
role: "user",
content: code,
},
],
store: true,
prediction: {
type: "content",
content: code,
},
});
// Inspect returned data
console.log(completion);
console.log(completion.choices[0].message.content);
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30from openai import OpenAI
code = """
class User {
firstName = "";
lastName = "";
username = "";
}
export default User;
""".strip()
refactor_prompt = """
Replace the "username" property with an "email" property. Respond only
with code, and with no markdown formatting.
"""
client = OpenAI()
completion = client.chat.completions.create(
model="gpt-4.1",
messages=[
{"role": "user", "content": refactor_prompt},
{"role": "user", "content": code},
],
prediction={"type": "content", "content": code},
)
print(completion)
print(completion.choices[0].message.content)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42package main
import (
"context"
"fmt"
"strings"
"github.com/openai/openai-go/v3"
"github.com/openai/openai-go/v3/shared"
)
func main() {
client := openai.NewClient()
code := strings.TrimSpace(`
class User {
firstName = "";
lastName = "";
username = "";
}
export default User;
`)
refactorPrompt := strings.TrimSpace(`
Replace the "username" property with an "email" property. Respond only
with code, and with no markdown formatting.
`)
completion, err := client.Chat.Completions.New(context.Background(), openai.ChatCompletionNewParams{
Model: shared.ChatModelGPT4_1,
Messages: []openai.ChatCompletionMessageParamUnion{
openai.UserMessage(refactorPrompt),
openai.UserMessage(code),
},
Store: openai.Bool(true),
Prediction: openai.ChatCompletionPredictionContentParam{
Content: openai.ChatCompletionPredictionContentContentUnionParam{OfString: openai.String(code)},
},
})
if err != nil {
panic(err)
}
fmt.Println(completion.Choices[0].Message.Content)
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31import com.openai.client.OpenAIClient;
import com.openai.client.okhttp.OpenAIOkHttpClient;
import com.openai.models.chat.completions.ChatCompletionCreateParams;
import com.openai.models.chat.completions.ChatCompletionPredictionContent;
String code =
"""
class User {
firstName: string = "";
lastName: string = "";
username: string = "";
}
export default User;
""";
String refactorPrompt =
"Replace the \"username\" property with an \"email\" property. "
+ "Respond only with code, and with no markdown formatting.";
ChatCompletionCreateParams params =
ChatCompletionCreateParams.builder()
.model("gpt-4.1")
.addUserMessage(refactorPrompt)
.addUserMessage(code)
.prediction(ChatCompletionPredictionContent.builder().content(code).build())
.store(true)
.build();
client.chat().completions().create(params).choices().stream()
.flatMap(choice -> choice.message().content().stream())
.forEach(System.out::println);
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32using OpenAI.Chat;
#pragma warning disable OPENAI001
string key = Environment.GetEnvironmentVariable("OPENAI_API_KEY")!;
string model = "gpt-4.1";
ChatClient client = new(model, key);
string code =
"""
class User {
firstName = "";
lastName = "";
username = "";
}
export default User;
""";
ChatCompletionOptions options = new()
{
OutputPrediction = ChatOutputPrediction.CreateStaticContentPrediction(code),
};
ChatCompletion completion = await client.CompleteChatAsync(
[
new UserChatMessage(
"Replace the username property with an email property. Respond only with code, and with no markdown formatting."
),
new UserChatMessage(code),
],
options
);
Console.WriteLine(completion.Content[0].Text);
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36require "openai"
client = OpenAI::Client.new
code = <<~CODE
class User {
firstName: string = "";
lastName: string = "";
username: string = "";
}
export default User;
CODE
refactor_prompt = <<~PROMPT
Replace the "username" property with an "email" property. Respond only
with code, and with no markdown formatting.
PROMPT
completion = client.chat.completions.create(
model: "gpt-4.1",
messages: [
{
role: :user,
content: refactor_prompt
},
{
role: :user,
content: code
}
],
prediction: {
type: :content,
content: code
},
store: true
)
puts(completion.choices.fetch(0).message.content)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20curl https://api.openai.com/v1/chat/completions \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $OPENAI_API_KEY" \
-d '{
"model": "gpt-4.1",
"messages": [
{
"role": "user",
"content": "Replace the username property with an email property. Respond only with code, and with no markdown formatting."
},
{
"role": "user",
"content": "$CODE_CONTENT_HERE"
}
],
"prediction": {
"type": "content",
"content": "$CODE_CONTENT_HERE"
}
}'
模型除了傳回重構後的程式碼,也會傳回用量資料。以下是省略 choices 欄位的回應節錄:
12345678910111213141516171819{
"id": "chatcmpl-xxx",
"object": "chat.completion",
"created": 1786652188,
"model": "gpt-4.1-2025-04-14",
"usage": {
"prompt_tokens": 59,
"completion_tokens": 24,
"total_tokens": 83,
"prompt_tokens_details": { "cached_tokens": 0, "audio_tokens": 0 },
"completion_tokens_details": {
"reasoning_tokens": 0,
"audio_tokens": 0,
"accepted_prediction_tokens": 14,
"rejected_prediction_tokens": 2
}
},
"system_fingerprint": "fp_6ddb4f7408"
}
請留意 accepted_prediction_tokens 和 rejected_prediction_tokens 這兩個欄位,它們都位於 usage 物件中。在此範例中,模型採用了 14 個預測 Token 來加快回應,並拒絕了 2 個。
遭拒的 Token 仍按 API 產生的其他輸出 Token 的費率計費,因此使用預測輸出可能會增加請求費用。
如果 API 以串流方式傳回回應,使用預測輸出能進一步縮短回應時間。以下沿用前面的程式碼重構範例,這次改用 OpenAI SDK 的串流功能。
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43import OpenAI from "openai";
const code = `
class User {
firstName = "";
lastName = "";
username = "";
}
export default User;
`.trim();
const openai = new OpenAI();
const refactorPrompt = `
Replace the "username" property with an "email" property. Respond only
with code, and with no markdown formatting.
`;
const completion = await openai.chat.completions.create({
model: "gpt-4.1",
messages: [
{
role: "user",
content: refactorPrompt,
},
{
role: "user",
content: code,
},
],
store: true,
prediction: {
type: "content",
content: code,
},
stream: true,
});
// Inspect returned data
for await (const chunk of completion) {
process.stdout.write(chunk.choices[0]?.delta?.content || "");
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32from openai import OpenAI
code = """
class User {
firstName = "";
lastName = "";
username = "";
}
export default User;
""".strip()
refactor_prompt = """
Replace the "username" property with an "email" property. Respond only
with code, and with no markdown formatting.
"""
client = OpenAI()
stream = client.chat.completions.create(
model="gpt-4.1",
messages=[
{"role": "user", "content": refactor_prompt},
{"role": "user", "content": code},
],
prediction={"type": "content", "content": code},
stream=True,
)
for chunk in stream:
if chunk.choices[0].delta.content is not None:
print(chunk.choices[0].delta.content, end="")
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46package main
import (
"context"
"fmt"
"strings"
"github.com/openai/openai-go/v3"
"github.com/openai/openai-go/v3/shared"
)
func main() {
client := openai.NewClient()
code := strings.TrimSpace(`
class User {
firstName = "";
lastName = "";
username = "";
}
export default User;
`)
refactorPrompt := strings.TrimSpace(`
Replace the "username" property with an "email" property. Respond only
with code, and with no markdown formatting.
`)
stream := client.Chat.Completions.NewStreaming(context.Background(), openai.ChatCompletionNewParams{
Model: shared.ChatModelGPT4_1,
Messages: []openai.ChatCompletionMessageParamUnion{
openai.UserMessage(refactorPrompt),
openai.UserMessage(code),
},
Store: openai.Bool(true),
Prediction: openai.ChatCompletionPredictionContentParam{
Content: openai.ChatCompletionPredictionContentContentUnionParam{OfString: openai.String(code)},
},
})
for stream.Next() {
if len(stream.Current().Choices) > 0 {
fmt.Print(stream.Current().Choices[0].Delta.Content)
}
}
if err := stream.Err(); err != nil {
panic(err)
}
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37import com.openai.client.OpenAIClient;
import com.openai.client.okhttp.OpenAIOkHttpClient;
import com.openai.core.http.StreamResponse;
import com.openai.models.chat.completions.ChatCompletionChunk;
import com.openai.models.chat.completions.ChatCompletionCreateParams;
import com.openai.models.chat.completions.ChatCompletionPredictionContent;
String code =
"""
class User {
firstName: string = "";
lastName: string = "";
username: string = "";
}
export default User;
""";
String refactorPrompt =
"Replace the \"username\" property with an \"email\" property. "
+ "Respond only with code, and with no markdown formatting.";
ChatCompletionCreateParams params =
ChatCompletionCreateParams.builder()
.model("gpt-4.1")
.addUserMessage(refactorPrompt)
.addUserMessage(code)
.prediction(ChatCompletionPredictionContent.builder().content(code).build())
.store(true)
.build();
try (StreamResponse<ChatCompletionChunk> stream =
client.chat().completions().createStreaming(params)) {
stream.stream()
.flatMap(chunk -> chunk.choices().stream())
.flatMap(choice -> choice.delta().content().stream())
.forEach(System.out::print);
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39using OpenAI.Chat;
#pragma warning disable OPENAI001
string key = Environment.GetEnvironmentVariable("OPENAI_API_KEY")!;
string model = "gpt-4.1";
ChatClient client = new(model, key);
string code =
"""
class User {
firstName = "";
lastName = "";
username = "";
}
export default User;
""";
ChatCompletionOptions options = new()
{
OutputPrediction = ChatOutputPrediction.CreateStaticContentPrediction(code),
};
await foreach (
StreamingChatCompletionUpdate update in client.CompleteChatStreamingAsync(
[
new UserChatMessage(
"Replace the username property with an email property. Respond only with code, and with no markdown formatting."
),
new UserChatMessage(code),
],
options
)
)
{
foreach (ChatMessageContentPart part in update.ContentUpdate)
{
Console.Write(part.Text);
}
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36require "openai"
client = OpenAI::Client.new
code = <<~CODE
class User {
firstName: string = "";
lastName: string = "";
username: string = "";
}
export default User;
CODE
refactor_prompt = <<~PROMPT
Replace the "username" property with an "email" property. Respond only
with code, and with no markdown formatting.
PROMPT
stream = client.chat.completions.stream(
model: "gpt-4.1",
messages: [
{
role: :user,
content: refactor_prompt
},
{
role: :user,
content: code
}
],
prediction: {
type: :content,
content: code
},
store: true
)
stream.text.each { |text| print(text) }
無論你提供的預測文字出現在模型回應的哪個位置,都能縮短回應時間。假設預測文字是以下簡單的 Hono 伺服器程式碼:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25import { serve } from "@hono/node-server";
import { serveStatic } from "@hono/node-server/serve-static";
import { Hono } from "hono";
const app = new Hono();
app.get("/api", (c) => {
return c.text("Hello Hono!");
});
// You will need to build the client code first: `pnpm run ui:build`.
app.use(
"/*",
serveStatic({
rewriteRequestPath: (path) => `./dist${path}`,
})
);
const port = 3000;
console.log(`Server is running on port ${port}`);
serve({
fetch: app.fetch,
port,
});
你可以使用類似以下的提示詞,請模型重新產生檔案:
請在這個應用程式中新增一條 GET 路由,讓它傳回文字「hello world」。
加入這條路由後,請重新產生整份應用程式檔案,不要加上任何額外的 Markdown 格式。
模型可能會傳回以下內容:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29import { serve } from "@hono/node-server";
import { serveStatic } from "@hono/node-server/serve-static";
import { Hono } from "hono";
const app = new Hono();
app.get("/api", (c) => {
return c.text("Hello Hono!");
});
app.get("/hello", (c) => {
return c.text("hello world");
});
// You will need to build the client code first: `pnpm run ui:build`.
app.use(
"/*",
serveStatic({
rewriteRequestPath: (path) => `./dist${path}`,
})
);
const port = 3000;
console.log(`Server is running on port ${port}`);
serve({
fetch: app.fetch,
port,
});
即使預測文字分別出現在新增內容的前後,你仍可以在以下省略 choices 欄位的回應節錄中看到模型採用了多少個預測 Token:
12345678910111213141516171819{
"id": "chatcmpl-xxx",
"object": "chat.completion",
"created": 1731014771,
"model": "gpt-4o-2024-08-06",
"usage": {
"prompt_tokens": 203,
"completion_tokens": 159,
"total_tokens": 362,
"prompt_tokens_details": { "cached_tokens": 0, "audio_tokens": 0 },
"completion_tokens_details": {
"reasoning_tokens": 0,
"audio_tokens": 0,
"accepted_prediction_tokens": 60,
"rejected_prediction_tokens": 0
}
},
"system_fingerprint": "fp_9ee9e968ea"
}
這次模型沒有拒絕任何預測 Token,因為最終回應用到了原始檔案中作為預測文字的全部內容。太好了!🔥
使用預測輸出時,請留意以下事項與限制。
- 只有 GPT-4o、GPT-4o mini、GPT-4.1、GPT-4.1 mini 和 GPT-4.1 nano 系列模型支援預測輸出。
- 提供預測內容時,即使其中部分 Token 未用於最終回應,這些 Token 仍會按輸出 Token 的費率計費。
rejected_prediction_tokens 屬性(位於 usage 物件中)會顯示最終回應未使用的預測 Token 數量。
- 使用預測輸出時,以下 API 參數有使用限制或不受支援:
n:不支援大於 1 的值
logprobs:不支援
presence_penalty:不支援大於 0 的值
frequency_penalty:不支援大於 0 的值
audio:預測輸出無法與音訊輸入和輸出搭配使用
modalities:僅支援 text 模態
max_completion_tokens:不支援
tools:預測輸出目前不支援函式呼叫