選擇您開發時使用的 Realtime 模型。若使用 GPT-Live,請參閱為 GPT-Live 撰寫提示詞。
gpt-realtime-2 是我們最先進的推理語音模型,適用於低延遲的語音到語音應用。它能先思考再說話,且相較於早期的即時互動模型,能更可靠地遵循指示、使用更大的上下文視窗,並更精準地呼叫工具。
若要充分發揮這些改進的效益,設計提示詞時應更明確地表達意圖。清楚定義助理的職責、決策時機、工具呼叫行為與防護機制:它應該做什麼、何時執行,以及應該避免什麼。
從簡單的提示詞開始,不要一開始就加入過多指示。先使用最精簡的提示詞並執行評估,再只針對測試中未達預期的行為補充指示。
選擇模型
| 模型 | 適用情境 | 提示詞設計重點 |
|---|---|---|
gpt-realtime-2 | 需要最強的即時推理、工具使用與指示遵循能力。 | 調整推理強度、前置說明、工具使用政策、實體值的精確擷取,以及長時間工作階段的狀態。 |
gpt-realtime-1.5 | 需要快速、可靠的非推理語音到語音模型。 | 遵循即時互動提示詞的核心結構,並測試對延遲敏感的行為。 |
Realtime 2.0 提示詞指南
當語音智慧體需要更強的推理、工具選擇、
實體值精確處理或長時間工作階段狀態管理能力時,請使用 gpt-realtime-2。
先從 reasoning.effort: "low" 開始,測試預設的前置說明行為,
並明確界定執行寫入動作前,哪些情況需要確認。
Realtime 2 有哪些變化
為 Realtime 2 設計提示詞時,應將它視為具備推理能力的語音智慧體,而非基本的語音機器人。
| 變化 | 對提示詞的影響 |
|---|---|
| 推理 | 處理複雜任務時,允許模型在開口或呼叫工具前先在內部推理。使用前置說明,避免尷尬的沉默或不必要的填充語。 |
| 提示詞的精確度更重要 | 以明確的觸發條件、動作與例外規則,取代「提供幫助」這類籠統指引:何時行動、做什麼,以及何時不該執行。 |
| 指示衝突造成的代價更高 | 除非確實必要,否則請移除彼此重疊的 always、never、only 和 must 規則。當規則互相衝突時,請明定優先順序。 |
| 工具使用行為更容易引導 | 明確說明助理何時應立即行動、詢問缺少的資訊、確認必須精確無誤的細節、在失敗後重試,或升級處理。 |
| 前置說明成為核心行為 | 模型可能會在較長的推理或工具使用流程開始前,先以簡短語音說明進度。請引導模型何時提供前置說明、應簡短到什麼程度,以及何時省略。 |
| 擴大的上下文視窗 | gpt-realtime-2 將即時模型的上下文視窗從 32k 擴大至 128k Token,更適合長時間工作階段與較大的系統提示詞。 |
前置說明不是隱藏的思路鏈,而是簡短的語音進度說明,例如「我現在幫你查詢那筆訂單」。請勿要求模型揭露私有推理內容。
建議的提示詞結構
使用簡短且有標題的區段,讓模型能迅速找到相關指示。
# Role and Objective
# Personality and Tone
# Language
# Reasoning
# Message Channels
# Preambles
# Verbosity
# Tools
# Unclear Audio
# Entity Capture
# Long Context Behavior
# Escalation並非每種使用情境都需要所有區段。加入與產品相關的區段即可。
設定推理強度
gpt-realtime-2 可以用較高的延遲換取更深入的推理。請選擇仍能讓助理具備足夠智慧來完成工作流程的最低推理等級。
對大多數正式環境中的語音智慧體,請先從 low 開始,再依據任務複雜度、可接受的延遲與失敗代價調高或調低。
| 推理強度 | 適用情境 | 範例 |
|---|---|---|
minimal | 最低延遲是首要考量,且任務簡單。 | 智慧家庭指令、計時器、簡單的行事曆查詢。 |
low | 需要快速回應,同時具備基本推理能力。 | 客戶支援、訂單查詢、簡單的政策問題。 |
medium | 助理必須透過推理完成多步驟任務。 | 技術支援、診斷、複雜的路由分派。 |
high | 更深入的推理能實質提高成功率。 | 精確度要求高的工作流程、升級處理決策、具有限制條件的任務。 |
xhigh | 最深入的推理值得付出額外的延遲與成本。 | 複雜規劃、關鍵問題分流、影響重大的工具編排。 |
除了 API 設定,也要引導模型何時推理,以及推理到什麼程度。
## Reasoning
- For direct answers, simple lookups, and short confirmations, respond quickly and do not reason.
- For multi-step tasks, tool decisions, troubleshooting, or escalation, reason before acting.
- Do not perform extended reasoning when the user's audio is unclear; ask for clarification instead.有目的地使用前置說明
前置說明是簡短的語音進度說明,能讓語音智慧體在推理、查詢資訊或呼叫工具時,仍讓人感覺回應即時。運用得當時,能讓使用者放心,知道助理正在處理。運用不當時,則會成為多餘的填充語,讓使用者覺得等待更久。
gpt-realtime-2 預設會產生前置說明。請先測試預設行為;若不符合產品體驗,再以明確指示加以調整。

## Preambles
Use short preambles only when they help the user understand that work is happening.
### When to use a preamble
Use a preamble when:
- you are about to call a tool that may take noticeable time;
- you need to reason through a multi-step request;
- you are checking records, availability, account state, or policy details;
- you are preparing an escalation or handoff;
- silence would make the assistant feel unresponsive.
When a preamble is needed, output it immediately before substantive reasoning or tool use.
### When to not use a preamble
Do not use a preamble when:
- the answer is direct and can be given immediately;
- the user is only confirming, correcting, or declining something;
- the audio is unclear and you need clarification;
- the latest audio is silence, background noise, hold music, TV audio, or side conversation;
- the tool call is lightweight and the user would not benefit from an update.
### Preamble style
When using a preamble:
- keep it natural, calm, and concise;
- vary the wording across turns;
- describe the action, not the internal reasoning;
- avoid filler.
Avoid phrases like:
- "Let me think..."
- "Hmm..."
- "One moment while I process that..."
- "I am now going to access the tool..."
### Preamble length
Use one short sentence.
Do not exceed two short sentences unless the user needs an explanation before a high-impact action.
### Prefer
- "I'll check that order now."
- "I'll look up your appointment details."
- "I'll verify that before we make any changes."
- "I'll check the policy and then give you the next step."
- "I'll pull that up so we can make sure it's the right account."
### Avoid
- "Let me think about that for a second."
- "Please wait while I process your request."
- "I'm going to use my tools now."
- "Interesting question. I will reason through this carefully."控制回覆長度
在提示詞中明確指定各類任務應提供多少細節,最能讓 gpt-realtime-2 遵循長度指引。與其只告訴模型「保持簡潔」,不如定義各種情境下何謂簡潔:直接回答、工具結果、疑難排解、比較與升級處理,可能各自需要不同的回覆長度。
## Verbosity
- Direct answers: Use 1-2 short sentences.
- Clarifying questions: Ask one question at a time.
- Tool results: Summarize the result first, then give only the next useful action.
- Product or option comparisons: Include key differences, tradeoffs, and who each option fits.
- Troubleshooting: Give one step at a time unless the user asks for the full procedure.
- Escalations: Briefly explain why escalation is needed and what will happen next.範例:
使用者:我應該選哪個方案?
助理:如果你想要最低的費用,請選擇 Basic。如果你需要團隊權限和共用帳單,請選擇 Pro。如果你重視合規審查或管理員控制功能,請選擇企業方案。
設計工具行為
gpt-realtime-2 的工具呼叫能力更強,但工具行為仍取決於提示詞和工具規格的設計。如果提示詞未定義何時該採取行動、提問、確認或處理失敗,助理可能會過早呼叫工具、提出不必要的問題,或重複執行失敗的呼叫。
設定工具呼叫的積極程度
唯讀、低風險的動作適合較積極地呼叫工具。如果工具會修改資料、對外部產生影響,或需要精確的識別碼,則較適合降低呼叫的積極程度。
| 工具類型 | 預設行為 |
|---|---|
| 唯讀、低風險的查詢 | 意圖和必要欄位都明確時,即可呼叫。 |
| 需要精確識別碼的唯讀操作 | 查詢前先確認識別碼。 |
| 使用者可見的通訊 | 傳送前先草擬內容或提供摘要。 |
| 帳戶變更 | 呼叫前先確認。 |
| 購買、取消、付款 | 呼叫前先確認金額、對象和後果。 |
| 無法復原或影響重大的動作 | 取得明確確認,並在適當時提供轉交處理的選項。 |
當你的工具同時包含讀取和寫入動作時,可採用這套兼顧兩者的預設行為,再依使用情境調整。
## Tools
Use only the tools explicitly provided in the current tool list. Do not invent, assume, simulate, or rename tools.
For read-only tools:
- Call the tool when the user's intent is clear and all required fields are available.
- Do not ask for confirmation unless the lookup depends on a high-precision identifier or there is meaningful risk of using the wrong record.
- Ask a clarification question only if a required field is missing, ambiguous, or conflicting.
For write tools or external actions:
- Summarize the intended action before calling the tool.
- Include the key consequence, such as what will be changed, sent, canceled, ordered, or charged.
- Ask for confirmation.
- Do not call the tool until the user clearly confirms.
For exact identifiers:
- Treat order IDs, tracking numbers, account numbers, confirmation codes, phone numbers, and email addresses as high precision.
- Normalize only when the field type is clear.
- Confirm the final value before account-specific lookups, validation, or write actions.
After tool calls:
- Only say an action was completed after the tool call succeeds.
- If the tool fails, explain the failure briefly, avoid raw errors, and give the user a clear next step.高風險範例:
使用者:請用我的卡片支付剩餘款項。
不佳的做法:
助理:已從你的卡片扣款。
良好的做法:
助理:跟你確認一下,你希望我從已儲存的卡片扣款 $248.16,以支付剩餘款項。要繼續嗎?
處理工具失敗並恢復流程
工具失敗也是對話的一部分。良好的復原處理應說明發生了什麼事,並讓使用者清楚知道下一步該怎麼做。
不要以相同方式處理所有失敗。復原行為應取決於工具類型、失敗情況,以及對使用者的影響。有些失敗應直接重試,不必告知使用者;另一些則需要請使用者澄清、更正識別碼、確認新的動作,或選擇其他處理方式。
## Tool Failures
If a tool call fails:
1. Briefly explain what failed in user-friendly language.
2. Do not blame the user or expose raw tool errors.
3. If the failure may be due to an exact identifier, read back the value used and ask the user to correct it.
4. If the failure may be temporary, offer to retry once.
5. If the same failure happens repeatedly, offer an alternate path or escalation.
Do not repeatedly call the same tool with the same arguments after failure.
Do not ask for a different identifier until you have first checked whether the captured value was correct.不佳的做法:
助理:發生了問題。
良好的做法:
助理:我找不到符合 O R D 連字號 3 1 2 5 B 2 3 的資料。我有哪個部分聽錯了嗎?
讓提示詞與可用工具保持同步
Realtime 模型會積極提供協助。如果提示詞提及實際上無法使用的工具,或工具清單與提示詞不一致,模型可能會捏造工具名稱,或假裝已完成動作。
例如,如果提示詞提及 lookup_order,但提供的工具名稱是 search_orders,模型可能會使用錯誤的名稱呼叫工具,或模擬該動作。
## Tool Availability
Use only the tools that are explicitly provided in the current tool list.
Do not invent, assume, or simulate tools. If a tool is mentioned in the instructions but is not present in the tool list, treat it as unavailable.
If the user requests an action that requires an unavailable tool:
1. Do not pretend to complete the action.
2. Briefly explain that the tool is not available.
3. Offer the closest supported next step.
Only say an action was completed after the relevant tool call succeeds.使用附錄中用於稽核提示詞的後設提示詞,檢查正式環境的提示詞是否存在矛盾、缺少工具,或包含容易出錯的指示。
處理靜音和背景音訊
語音智慧體預設傾向於做出回應。在正式環境中,它們經常接收到不應以語音回應的音訊,例如靜音、背景噪音、電話等候音樂、電視聲音,或旁人的交談。
當助理應保持安靜並繼續聆聽時,請使用不執行任何操作的等待工具。這個工具讓模型能採取有效且不需發聲的動作,避免它說出「我在這裡」或「我沒聽清楚」之類的話。
工具設計:
{
"name": "wait_for_user",
"description": "Call this when the latest audio does not need a spoken response, such as silence, background noise, hold music, TV audio, side conversation, or speech not addressed to the assistant. This tool helps end the turn without a spoken reply.",
"parameters": {
"type": "object",
"properties": {},
"required": []
}
}搭配以下提示詞指示:
## Handling Silence and Background Noise
If the latest audio is silence, background noise, hold music, TV audio, side conversation, or speech not addressed to you, call `wait_for_user`.
Do not respond conversationally after calling this tool.
Do not say "I'm here," "I didn't catch that," "Take your time," or "Let me know when you're ready."
Resume normal responses only when the user clearly addresses you or asks for help.這種做法適用於非對助理說話的音訊,不適用於不清楚的使用者請求。如果使用者明顯在對助理說話,但內容無法辨識,就應請使用者澄清。
依用途運用訊息通道
gpt-realtime-2 可在 commentary 通道產生使用者可見的中間訊息,並在 final 通道產生面向使用者的最終回覆。如果行為需依出現的通道而有所不同,請針對各通道給予指示。
| 通道 | 使用者可見? | 用途 |
|---|---|---|
commentary | 是 | 前置說明和工具呼叫。 |
final | 是 | 面向使用者的最終訊息。 |
例如,工具呼叫會在 commentary 通道進行。如果你希望助理在使用工具之前、期間或之後說些話,請明確指定它應在 commentary 通道採取的行為。
Before calling tools in the commentary channel, briefly tell the user what you are doing.gpt-realtime-2 可在單一回合中輸出多個回應階段。API 輸出透過 response.done 事件區分這些階段;該事件包含 phase 值,用來指出內容屬於 commentary 還是最終答案。
你可以利用這個欄位,在應用程式中以不同方式處理各個階段。例如,commentary 可以作為簡短的中途進度更新來播放或顯示,而 final_answer 則可保留給助理完成後的回覆。
response.output[0].phase: "commentary"
response.output[1].phase: "final_answer"處理不清楚的音訊
模型應只根據有把握理解的音訊採取行動。如果音訊不清楚,模型應以簡短的問題請使用者澄清,而不是猜測。
不要讓模型推測缺漏的字詞、呼叫工具、擷取實體、產生前置說明,或花時間在內部推理中嘗試還原使用者可能說過的內容。
## Unclear Audio
- Only respond to clear audio or text.
- If the user's audio is not clear, ask for clarification using a short English phrase such as "Sorry, could you repeat that clearly?"
- Don't repeat the same unclear-audio clarification twice.
- Treat audio as unclear if it is ambiguous, noisy, silent, unintelligible, partially cut off, or if you are unsure of the exact words the user said.
- Do not guess what the user meant from unclear audio.
- Do not reason when the audio is unclear.
- Do not provide a preamble or call tools in the commentary channel when the audio is unclear.範例:
使用者音訊:「查詢訂單三一——」[中斷]
不佳的做法:
助理:我現在就查詢訂單 31。
良好的做法:
助理:我只聽到部分訂單編號。可以請你逐一念出每個數字嗎?
準確擷取實體
許多即時工作流程都需要準確的值,例如訂單 ID、物流追蹤號碼、電子郵件地址、確認碼、帳號、理賠編號、工單 ID、客服案件編號和電話號碼。
透過語音準確取得這些值並不容易。使用者可能說得很快、以不同方式分組念出數字、只拼讀部分內容、夾雜語助詞、說到一半自行更正,或念出發音相近的字元。只要一個數字有誤,就可能導致查詢失敗或查到錯誤的帳戶。
擷取實體時應採取保守做法。一次收集一個值,只將明確的內容正規化,在呼叫工具前確認需要高度準確的值,並確保每次更正後都能恢復流程。
一次收集一個實體
工作流程需要多個值時,請逐一收集。這能避免不同欄位的內容混在一起,尤其是在語音對話中。
## Entity Collection Order
Collect required values one at a time.
- Ask for only the next missing value.
- Do not ask for multiple values in the same turn.
- Before asking, check whether the value was already provided earlier in the conversation or the session.
- If a possible value already exists, confirm it with the user before using it.
Example:
"I see tracking number ABC-54321 from earlier. Should I use that one, or do you have a different tracking number?"
Do not call tools until the current value has been collected, validated, and confirmed.處理逐字拼讀的字元
當使用者逐一拼讀 ID、代碼、姓名或電子郵件地址的字元時,請使用這個方法。口述形式只是輸入,並非最終值。
## Spelled-Out Characters
When a user dictates an ID, code, or email character by character, treat the spoken sequence as one compact value. Preserve explicitly spoken separators like dash, dot, underscore, slash, or plus; otherwise do not add spaces or separators.
Examples:
- "A B C one two three" -> "ABC123"
- "B C dash nine eight seven" -> "BC-987"
- "J O H N at example dot com" -> "john@example.com"
Do not insert spaces between spelled-out characters unless the user explicitly says the value contains spaces.謹慎正規化口述數字
對於數字識別碼,使用者可能逐位念出、分組念出,或用自然的數字說法表達。如果欄位需要一串連續數字,請將清楚的口述數字轉換成數字字元。
## Spoken Number Handling
Convert spoken numbers into digits when collecting numeric identifiers.
Examples:
- "one two three four" -> "1234"
- "one twenty three" -> "123"
- "one nineteen" -> "119"
- "ninety nine eleven" -> "9911"
- "nine thousand nine hundred eleven" -> "9911"
If multiple interpretations are plausible, ask the user to clarify before using the value.
Example:
"I heard either 119 or 1-19. Could you repeat the number digit by digit?"在呼叫工具前確認識別碼完全正確
訂單 ID、物流追蹤號碼、帳號、理賠編號、確認碼及類似識別碼,都是需要高度準確的欄位。在工具呼叫中使用這些值之前,請先確認。
對於數字識別碼,請逐位複誦。將整串數字當成一個數值念出,可能會讓錯誤不易被察覺。
範例:
助理:跟你確認一下,我聽到的是 8……3……5……2……1,對嗎?
如果使用者更正了一個字元或數字,請在呼叫工具前複誦更正後的完整值。
範例:
助理:了解。我記下的是 8……3……5……7……1,這樣正確嗎?
## Exact Identifier Confirmation
Before calling tools with high-precision identifiers:
- Confirm the final normalized value with the user.
- Read numeric identifiers back digit by digit.
- Do not use guessed, partial, or ambiguous values.
- If the user corrects the value, repeat the full corrected value before calling the tool.逐一確認電子郵件地址的字元
電子郵件地址是重要的資料。句點、連字號、底線、重複字母及發音相近的名稱,都可能導致帳戶查詢失敗,或將訊息寄到錯誤的地址。
請使用者拼讀電子郵件地址:
助理:可以請你逐一念出電子郵件地址的每個字元嗎?這樣我才能確認完全正確。
複誦時,請確認最終地址完全正確:
助理:跟你確認一下,是 c-h-e-n at example dot com,對嗎?
## Email Confirmation
Email addresses must be captured exactly.
If the user says the email naturally without spelling it out, ask them to repeat it character by character.
Example:
"Could you spell the email address character by character so I can make sure I have it exactly right?"
When reading an email back, confirm the exact final email address.
Example:
"Just to confirm, that is c-h-e-n at example dot com, right?"實體收集工作流程
避免按字面解讀指示的陷阱
相較於先前的即時模型,gpt-realtime-2 更傾向於按字面遵循指示。在舊模型上表現良好的提示詞,可能需要調整。
請使用精確的措辭。模型可能優先遵循指示的字面表述,而非你原本期望的整體行為。範圍過廣或過於僵硬的規則,可能以出乎意料的方式主導助理的行為,尤其是在多項規則重疊時。
請謹慎使用 must、only、never 和 always 等限制性用語。只有在行為確實不可或缺時才使用,不要將它們當成一般的強調方式。過度使用硬性限制,可能讓助理變得僵化、過度謹慎,或無法處理合理的例外情況。
建議明確界定適用範圍:
For write actions that modify user data, ask for confirmation before calling the tool.避免適用範圍過廣:
Always ask for confirmation before doing anything.範圍過廣的版本,可能導致助理在執行無害的唯讀查詢前,也要求不必要的確認,例如查詢訂單狀態、取得可用情況,或讀取帳戶資訊。
按字面解讀的範例
一般提示詞撰寫建議:
- 優先使用明確的指示,不要只暗示意圖。
- 除非行為確實必須嚴格固定,否則請避免不必要的限制性用語。
- 盡量減少互相矛盾的指引。
- 謹慎使用層層疊加或互相衝突的優先順序指示。
- 逐步測試提示詞。細微的措辭變更,也可能大幅影響行為。
- 從先前的即時模型遷移時,請預期部分提示詞需要重新調整結構,才能取得最佳效果。
分別控制語言與口音
語言與口音應分別控制。
使用者的口音不等於他們想使用的語言。使用者可能帶著印地語、西班牙語、法語或華語口音說英語,但仍希望收到英語回應。
避免使用範圍過廣的語言指示,例如:
Mirror the user.
Respond naturally in the user's language.
Switch languages when appropriate.
Sound local.
Adapt to the user's accent.這些指示的範圍太廣。模型可能將口音、語助詞、簡短的應和語,或零星的外語詞彙,解讀為切換語言的理由。
英語使用規則
## Language
English is the default response language.
- Do not infer language from accent alone.
- Ignore short filler sounds, backchannels, and isolated foreign words for language detection.
- Only switch languages if the user explicitly asks or provides a substantive utterance in another language.
- If language confidence is low, ask a short clarification instead of guessing.
- Keep preambles, spoken bridges, tool-related messages, and final answers in the same language.
- Accent adaptation must not change the response language.多語言使用規則
## Language
Default to English unless the user clearly uses another language.
Switch languages only when:
- the user explicitly asks to use another language;
- the user provides a substantive utterance in another language. A substantive utterance means the user gives a complete request, question, or correction in another language, not just a greeting, name, address, filler word, or borrowed phrase.
Do not switch languages based on:
- accent;
- pronunciation;
- filler words;
- short backchannels;
- names;
- addresses;
- isolated foreign words.
If uncertain, ask:
"Would you like me to continue in English or [LANGUAGE]?"口音控制
gpt-realtime-2 能更確實地遵循口音指示,但模糊的口音提示詞可能導致口音偏移,或意外切換語言。
口音控制提示詞明確指定下列事項時,效果最佳:
- 目標口音;
- 哪些特徵應維持穩定;
- 預期的語速、重音與韻律;
- 調整口音是否應影響語言選擇。
避免使用:
Sound Australian.改用:
## Accent
Speak English with a light Australian accent.
- Keep the accent stable from the first word to the last.
- Use natural Australian vowel shaping, but keep speech easy to understand.
- Do not exaggerate the accent.
- Do not change response language based on the user's accent.自訂語音
當標準語音無法穩定滿足品牌、口音或角色需求時,請使用自訂語音。
提示詞可以引導口音、語速與表達方式,但無法完全取代語音設計。若使用案例需要一致的品牌聲音特色或精準的口音,請考慮使用自訂語音。
自訂語音僅供經核准的客戶使用。如需存取權,請聯絡您的客戶團隊。
在長時間工作階段中維持狀態
gpt-realtime-2 將即時上下文視窗從 32k 擴大至 128k Token,使其更適合長時間工作階段。對於密集的雙向對話,128k Token 大致可視為約 1–2 小時的密集原始音訊上下文。實際容量會隨工具使用、內部推理、插入的紀錄及其他工作階段細節而變動。
在長上下文使用案例中,當 gpt-realtime-2 能分辨哪些資訊是最新資訊、哪些是背景資料,以及來源衝突時應忽略哪些內容,就能發揮最佳表現。請勿依賴模型從原始逐字稿或大量未整理的上下文中推斷來源優先順序。請以結構化方式組織內容。
若工作階段開始時需要提供大量上下文,例如擷取的紀錄、先前的對話歷史、政策、摘要、帳戶備註或背景文件,請採用結構化格式。
從較早的即時模型遷移
從較早的即時模型遷移時,應將提示詞視為控制行為的介面,而不只是要搬移的文字。
- 使用 Codex 或能力強大的推理模型,依照最新的 Realtime 提示詞指引重整提示詞。請附上本提示詞指南的連結,讓遷移工作以最佳實務為依據。
- 將推理強度設為
low,不要使用預設值。只有在工作流程需要更深入的規劃時,才提高推理強度。 - 檢查工具名稱、參數、列舉值、JSON 結構描述及其他設定,確保它們與預期的實作一致。
- 移除過時的範例。針對正常流程、語意不明、中斷、工具呼叫及備援行為,新增簡短範例。
- 比較遷移前後具代表性的對話。使用既有評估檢查是否出現退步,並記錄刻意調整的行為。
- 最後再進行一次一致性檢查。確認提示詞清楚區分強制要求、預設行為、工具規則、安全規則及備援行為。
- 執行評估、檢視具代表性的失敗案例,並反覆調整提示詞,直到能可靠地呈現目標行為。
Realtime 1.5 提示詞指南
gpt-realtime-1.5 是 Realtime API 中的語音到語音模型。gpt-realtime 的提示詞指引同樣適用於此模型。
語音到語音系統是讓語音成為核心 AI 介面的關鍵。gpt-realtime-1.5 支援穩健且實用的即時語音智慧體,能大規模處理關鍵任務工作流程。
相較於較早的即時預覽模型,gpt-realtime-1.5 的指令遵循能力更強、工具呼叫更可靠、語音品質更好,整體體驗也更流暢。這些改進讓應用程式能實際從串接式處理方式轉向真正的即時體驗,降低延遲,並產生聽起來更自然、更有表現力的回應。
有些提示詞技巧能改善即時模型的表現,卻不適合直接套用於文字模型。本指南先提供建議的提示詞架構,再逐一說明各部分,搭配實用技巧、可直接複製的簡短模式,以及可依使用案例調整的範例。
一般技巧
- 持續反覆調整:細微的措辭變化,就可能決定行為是否符合預期。
- 例如:在處理不清楚音訊的指令中,我們將「聽不見」改為「無法理解」,改善了模型處理含雜訊輸入的能力。
- 優先使用條列,而非段落:清楚、簡短的條列比長篇段落更有效。
- 以範例引導:模型會密切遵循範例語句。
- 表達精確:與 GPT-5 類似,含糊或互相衝突的指令會降低表現。
- 控制語言:若出現非預期的語言切換,請將輸出固定為目標語言。
- 減少重複:加入用語多樣性規則,減少機械式措辭。
- 使用大寫文字強調重點:將關鍵規則寫成大寫,能讓它們更醒目,也更容易讓模型遵循。
- 將非文字規則改寫為文字:不要寫「IF x > 3 THEN ESCALATE」,改寫為「失敗超過三次時,轉交處理」。
提示詞結構
妥善組織提示詞,可讓模型更容易理解上下文,並在多輪對話中保持一致。這也能讓您更容易反覆調整提示詞,以及修改有問題的區段。
- 作用:在系統提示詞中使用清楚且有標題的區段,讓模型能找到並遵循相關指令。每個區段應只聚焦於一件事。
- 調整方式:新增特定領域所需的區段,例如合規、品牌政策。移除不需要的區段,例如沒有發音問題時,可移除參考發音區段。
範例
# Role & Objective — who you are and what “success” means
# Personality & Tone — the voice and style to maintain
# Context — retrieved context, relevant info
# Reference Pronunciations — phonetic guides for tricky words
# Tools — names, usage rules, and preambles
# Instructions / Rules — do’s, don’ts, and approach
# Conversation Flow — states, goals, and transitions
# Safety & Escalation — fallback and handoff logic角色與目標
此區段定義智慧體的身分,以及什麼情況才算「完成」。範例呈現兩種不同身分,示範明確設定角色與目標後,模型會如何嚴格遵循。
- 適用時機:模型未採用您需要的人物設定、角色,或未依照所需的任務範圍行事。
- 作用:固定語音智慧體的身分,讓回應符合該角色描述。
- 調整方式:依使用案例修改角色。
範例(模型採用特定口音)
# Role & Objective
You are a Quebecois French-speaking customer service bot. Your task is to answer the user's question.較早的即時預覽模型:
gpt-realtime-1.5:
範例(模型扮演角色)
# Role & Objective
You are a high-energy game-show host guiding the caller to guess a secret number from 1 to 100 to win 1,000,000$.較早的即時預覽模型:
gpt-realtime-1.5:
相較於較早的即時預覽模型,gpt-realtime-1.5 能更可靠地扮演指定角色。
個性與語氣
gpt-realtime-1.5 在模仿特定個性或語氣時,能良好地遵循指令。您可以依使用案例的預期需求,調整語音體驗與表達方式。
- 適用時機:回應聽起來平淡、過於冗長,或在多輪對話中不一致。
- 作用:設定聲音風格、簡潔程度與語速,讓回覆聽起來自然且一致。
- 調整方式:調整親切程度、正式程度與預設長度。對於受法規監管的領域,應優先採用中立、精確的表達方式。新增與使用案例相關的其他小節。
範例
# Personality & Tone
## Personality
- Friendly, calm and approachable expert customer service assistant.
## Tone
- Warm, concise, confident, never fawning.
## Length
2–3 sentences per turn.範例(多種情緒)
# Personality & Tone
- Start your response very happy
- Midway, change to sad
- At the end change your mood to very angrygpt-realtime-1.5:
模型能遵循複雜指令,並在整段音訊回應中切換三種情緒。
語速指示
在 Realtime API 中,speed 參數改變的是播放速度,而不是模型組織語音的方式。若要讓說話速度真正加快,請加入引導語速與節奏的指示。
- 適用時機:使用者希望語速更快,但單靠播放速度(透過 speed 參數)無法改善說話風格。
- 作用:調整說話風格(簡潔程度、節奏),不受用戶端播放速度影響。
- 調整方式:依使用情境的需求修改語速指示。
範例
# Personality & Tone
## Personality
- Friendly, calm and approachable expert customer service assistant.
## Tone
- Warm, concise, confident, never fawning.
## Length
- 2–3 sentences per turn.
## Pacing
- Deliver your audio response fast, but do not sound rushed.
- Do not modify the content of your response, only increase speaking speed for the same response.較早的 realtime 預覽模型:
gpt-realtime-1.5:
透過明確的語速與節奏指示,gpt-realtime-1.5 可以明顯加快語速,同時不會聽起來過於急促。
語言限制
語言限制可確保模型持續以指定語言回應,即使有背景雜音或多語言輸入等困難情況也一樣。
- 適用時機:需要防止模型在多語言或嘈雜環境中意外切換語言。
- 作用:將輸出固定為所選語言,避免意外切換語言。
- 調整方式:將「英語」換成你的目標語言,或依使用情境加入更複雜的指示。
範例(固定使用一種語言)
# Personality & Tone
## Personality
- Friendly, calm and approachable expert customer service assistant.
## Tone
- Warm, concise, confident, never fawning.
## Length
- 2–3 sentences per turn.
## Language
- The conversation will be only in English.
- Do not respond in any other language even if the user asks.
- If the user speaks another language, politely explain that support is limited to English.以下是 gpt-realtime-1.5 套用指示後的回應。

範例(模型教授語言)
# Role & Objective
- You are a friendly, knowledgeable voice tutor for French learners.
- Your goal is to help the user improve their French speaking and listening skills through engaging conversation and clear explanations.
- Balance immersive French practice with supportive English guidance to ensure understanding and progress.
# Personality & Tone
## Personality
- Friendly, calm and approachable expert customer service assistant.
## Tone
- Warm, concise, confident, never fawning.
## Length
- 2–3 sentences per turn.
## Language
### Explanations
Use English when explaining grammar, vocabulary, or cultural context.
### Conversation
Speak in French when conducting practice, giving examples, or engaging in dialogue.以下是 gpt-realtime-1.5 套用指示後的回應。

模型能根據自訂指示,在不同語言之間切換。
減少重複
realtime 模型能緊密遵循範例語句,維持品牌風格,但也可能過度使用這些語句,讓回應聽起來生硬或重複。加入限制重複的規則,有助於在保持清晰表達和品牌語氣的同時,讓措辭更有變化。
- 適用時機:模型在不同回合或工作階段中,反覆使用相同的開場白、填充詞或句型。
- 作用:加入要求措辭變化的限制,減少重複語句、鼓勵使用同義詞和不同句型,同時保留必要術語。
- 調整方式:調整嚴格程度(例如「每 N 個回合內,相同開場白最多只能使用一次」),將必須保留的語句(法律、合規、品牌用語)列入允許清單,並在重視一致性的情況下,允許更固定的措辭。
範例
# Personality & Tone
## Personality
- Friendly, calm and approachable expert customer service assistant.
## Tone
- Warm, concise, confident, never fawning.
## Length
- 2–3 sentences per turn.
## Language
- The conversation will be only in English.
- Do not respond in any other language even if the user asks.
- If the user speaks another language, politely explain that support is limited to English.
## Variety
- Do not repeat the same sentence twice.
- Vary your responses so they don't sound robotic.以下是 gpt-realtime-1.5 套用指示 前 的回應。模型會重複使用同一句確認語句:Got it。

以下是 gpt-realtime-1.5 套用指示 後 的回應。

現在,模型能變換回應和確認語句,不再聽起來像機器人。
參考發音
本節說明如何確保模型在語音互動中,正確唸出重要的字詞、數字、名稱和術語。
- 適用時機:模型經常唸錯品牌名稱、技術術語或地名。
- 作用:透過發音提示,提升信任感與表達的清晰度。
- 調整方式:清單保持簡短,並在聽到發音錯誤時更新。
範例
# Reference Pronunciations
When voicing these words, use the respective pronunciations:
- Pronounce “SQL” as “sequel.”
- Pronounce “PostgreSQL” as “post-gress.”
- Pronounce “Kyiv” as “KEE-iv.”
- Pronounce "Huawei" as “HWAH-way”較早的 realtime 預覽模型:
gpt-realtime-1.5:
透過參考發音指示,gpt-realtime-1.5 能將 SQL 正確唸成「sequel」。
英數字元發音
Realtime S2S 在覆誦關鍵資訊(電話號碼、信用卡號、訂單 ID)時,可能會把數字或字母唸得模糊不清,或連在一起。明確要求逐字元確認,可避免聽錯,並讓合成語音更清晰。
- 適用時機:模型難以準確擷取或覆誦電話號碼、卡號、2FA 驗證碼、訂單 ID、序號、地址、戶號,或英數混合字串。
- 作用:強制模型逐一唸出字元,並在字元之間加入分隔,再向使用者確認;如有修正,則再次確認。也可選擇使用拼讀詞來區分字母(例如「A,也就是 Alpha 的 A」)。
範例(一般指示區段)
# Instructions/Rules
- When reading numbers or codes, speak each character separately, separated by hyphens (e.g., 4-1-5).
- Repeat EXACTLY the provided number; do not omit any digits.提示:如果你採用以對話流程設計提示詞的策略,可以指定哪些對話狀態需要套用英數字元發音指示。
範例(對話狀態中的指示)
(摘自我們的 openai-realtime-agents 提示詞中的對話流程)
{
"id": "3_get_and_verify_phone",
"description": "Request phone number and verify by repeating it back.",
"instructions": [
"Politely request the user’s phone number.",
"Once provided, confirm it by repeating each digit and ask if it’s correct.",
"If the user corrects you, confirm AGAIN to make sure you understand.",
],
"examples": [
"I'll need some more information to access your account if that's okay. May I have your phone number, please?",
"You said 0-2-1-5-5-5-1-2-3-4, correct?",
"You said 4-5-6-7-8-9-0-1-2-3, correct?"
],
"transitions": [{
"next_step": "4_authentication_DOB",
"condition": "Once phone number is confirmed"
}]
}以下是 gpt-realtime-1.5 套用指示 前 的回應。
沒問題!號碼是 55119765423。如果還有其他需要,請告訴我!
以下是 gpt-realtime-1.5 套用指示 後 的回應。
沒問題!號碼是:5-5-1-1-1-9-7-6-5-4-2-3。如果還有其他需要,請告訴我!
指示
本節提供提示詞撰寫指引,說明如何指示模型完成任務、採用最佳實務,並修正可能出現的問題。
你可能不會意外,我們建議採用與 GPT-4.1 相似的提示詞模式,以獲得最佳效果。
遵循指示
與 GPT-4.1 和 GPT-5 一樣,如果指示互相衝突、有歧義或不清楚,gpt-realtime-1.5 的表現就會變差。
- 適用時機:模型的輸出偏離規則、跳過階段,或誤用工具。
- 作用:在上線前,利用 LLM 指出歧義、衝突和缺少的定義。
指示品質檢查提示詞(可在 ChatGPT 中或透過 API 使用)
將以下提示詞提供給 GPT-5,找出你的提示詞中有問題且可以修正的部分。
## Role & Objective
You are a **Prompt-Critique Expert**.
Examine a user-supplied LLM prompt and surface any weaknesses following the instructions below.
## Instructions
Review the prompt that is meant for an LLM to follow and identify the following issues:
- Ambiguity: Could any wording be interpreted in more than one way?
- Lacking Definitions: Are there any class labels, terms, or concepts that are not defined that might be misinterpreted by an LLM?
- Conflicting, missing, or vague instructions: Are directions incomplete or contradictory?
- Unstated assumptions: Does the prompt assume the model has to be able to do something that is not explicitly stated?
## Do **NOT** list issues of the following types:
- Invent new instructions, tool calls, or external information. You do not know what tools need to be added that are missing.
- Issues that you are unsure about.
## Output Format
"""
# Issues
- Numbered list; include brief quote snippets.
# Improvements
- Numbered list; provide the revised lines you would change and how you would change them.
# Revised Prompt
- Revised prompt where you have applied all your improvements surgically with minimal edits to the original prompt
"""提示詞最佳化元提示詞(可在 ChatGPT 中或透過 API 使用)
這個元提示詞能針對特定失敗模式,協助你改善基礎系統提示詞。提供目前的提示詞,並描述你觀察到的問題,模型(GPT-5)就會提出改良版本,收緊限制並減少問題發生。
Here's my current prompt to an LLM:
[BEGIN OF CURRENT PROMPT]
{CURRENT_PROMPT}
[END OF CURRENT PROMPT]
But I see this issue happening from the LLM:
[BEGIN OF ISSUE]
{ISSUE}
[END OF ISSUE]
Can you provide some variants of the prompt so that the model can better understand the constraints to alleviate the issue?沒有音訊或音訊不清楚
有時模型會以為自己聽到了什麼,並嘗試回應。你可以加入自訂指示,告訴模型在聽到不清楚的音訊或使用者輸入時該如何處理。請根據使用案例調整預期行為。例如,你可能希望模型重複上一個問題,而不是請使用者澄清。
- 適用時機:背景雜音、不完整的字詞或靜音觸發了不必要的回覆。
- 作用:避免無端回應,並自然地請使用者澄清。
- 調整方式:根據使用案例,選擇請使用者澄清,或重複上一個問題。
範例(咳嗽與不清楚的音訊)
# Instructions/Rules
...
## Unclear audio
- Always respond in the same language the user is speaking in, if unintelligible.
- Only respond to clear audio or text.
- If the user's audio is not clear (e.g. ambiguous input/background noise/silent/unintelligible) or if you did not fully hear or understand the user, ask for clarification using {preferred_language} phrases.以下是 gpt-realtime-1.5 套用指示 後 的回應。
在這個範例中,模型聽到我 (非常) 大聲的咳嗽和不清楚的音訊後,請我澄清。
背景音樂或聲響
模型在語音生成過程中,偶爾可能產生非預期的背景音樂、哼唱、有節奏的雜音或類似聲響的瑕疵。這些音訊瑕疵可能降低清晰度、分散使用者的注意力,或讓助理顯得不夠專業。以下指示有助於避免或大幅減少這類情況。
- 適用時機:發現 Realtime 音訊回應中出現非預期的音樂元素或音效。
- 作用:引導模型避免產生這些不必要的音訊瑕疵。
- 調整方式:針對你遇到的特定聲音模式調整指示,嘗試明確要求模型抑制這些聲音。
範例
# Instructions/Rules
...
- Do not include any sound effects or onomatopoeic expressions in your responses.工具
在這個區段中,告訴模型如何使用你的函式和工具。明確說明何時應該呼叫工具、何時不應該呼叫、需要收集哪些引數、呼叫執行期間該說什麼,以及如何處理錯誤或不完整的結果。
工具選擇
gpt-realtime-1.5 會確實遵循指示。不過,如果指示與模型實際可存取的資源不符,例如提示詞提到了「未」透過工具清單傳入的工具,就可能導致不良回應。
- 適用時機:提示詞提到了實際上無法使用的工具。
- 作用:審查可用工具與系統提示詞,確保兩者一致。
範例
# Tools
## lookup_account(email_or_phone)
...
## check_outage(address)
...我們需要確保提到的工具都可用,而且 各處的描述沒有互相矛盾:
[
{
"name": "lookup_account",
"description": "Retrieve a customer account using either an email or phone number to enable verification and account-specific actions.",
"parameters": {
...
},
{
"name": "check_outage",
"description": "Check for network outages affecting a given service address and return status and ETA if applicable.",
"parameters": {
...
}
]工具呼叫前置說明
在某些使用案例中,讓 Realtime 模型在呼叫工具的同時提供音訊回應,能減少使用者感受到的延遲,改善使用體驗。你可以根據使用案例調整示範語句。
- 適用時機:使用者需要在工具呼叫的同時立即得知請求已收到;這有助於減少等待感。
- 作用:在工具呼叫前加入簡短、一致的前置說明。
範例
# Tools
- Before any tool call, say one short line like “I’m checking that now.” Then call the tool immediately.以下是 gpt-realtime-1.5 套用指示後的回應。

套用指示後,模型會在呼叫工具的同時,以語音回應「我正在查詢」。
工具呼叫前置說明 + 示範語句
如果你想更精確地控制模型在呼叫工具時使用哪一類語句,可以在工具規格的描述中加入示範語句。
範例
tools = [
{
"name": "lookup_account",
"description": """Retrieve a customer account using either an email or phone number to enable verification and account-specific actions.
Preamble sample phrases:
- For security, I’ll pull up your account using the email on file.
- Let me look up your account by {email} now.
- I’m fetching the account linked to {phone} to verify access.
- One moment—I’m opening your account details.""",
"parameters": {
"type": "object",
"properties": {
"email": {"type": "string"},
"phone": {"type": "string"},
},
"additionalProperties": False,
},
},
{
"name": "check_outage",
"description": """Check for network outages affecting a given service address and return status and ETA if applicable.
Preamble sample phrases:
- I’ll check for any outages at {service_address} right now.
- Let me look up network status for your area.
- I’m checking whether there’s an active outage impacting your address.
- One sec—verifying service status and any posted ETA.""",
"parameters": {
"type": "object",
"properties": {
"service_address": {"type": "string"},
},
"required": ["service_address"],
"additionalProperties": False,
},
},
]不經確認的工具呼叫
模型有時可能會在呼叫工具前要求確認。在某些使用案例中,這會讓模型顯得不夠主動,影響終端使用者的體驗。
- 適用時機:智慧體在明顯應該呼叫工具的情況下,仍會先徵求同意。
- 作用:免去不必要的反覆確認。
範例
# Tools
- When calling a tool, do not ask for any user confirmation. Be proactive以下是 gpt-realtime-1.5 套用指示 後 的回應。

在這個範例中,你可以看到即時模型沒有產生任何音訊回應,而是直接呼叫對應的工具。
提示:如果你發現模型太急於呼叫工具,可以試著緩和措辭。例如,將「積極主動」這類較強烈的用語換成較溫和的說法,有助於引導模型採取更從容、不那麼急切的做法。
工具呼叫表現
隨著使用案例變得更複雜、可用工具數量增加,明確引導模型何時使用各項工具就變得至關重要,而何時不該使用也同樣重要。清楚的使用規則不僅能提高工具呼叫的準確性,也能幫助模型在正確的時機選擇合適的工具。
- 適用時機:模型的工具呼叫表現不佳,需要明確的指示來減少誤用。
- 作用:加入指示,說明何時應「使用/避免使用」各項工具。你也可以加入工具呼叫順序的指示(呼叫工具 A 後,可以呼叫工具 B 或 C)。
範例
# Tools
- When you call any tools, you must output at the same time a response letting the user know that you are calling the tool.
## lookup_account(email_or_phone)
Use when: verifying identity or viewing plan/outage flags.
Do NOT use when: the user is clearly anonymous and only asks general questions.
## check_outage(address)
Use when: user reports connectivity issues or slow speeds.
Do NOT use when: question is billing-only.
## refund_credit(account_id, minutes)
Use when: confirmed outage > 240 minutes in the past 7 days.
Do NOT use when: outage is unconfirmed; route to Diagnose → check_outage first.
## schedule_technician(account_id, window)
Use when: repeated failures after reboot and outage status = false.
Do NOT use when: outage status = true (send status + ETA instead).
## escalate_to_human(account_id, reason)
Use when: user seems very frustrated, abuse/harassment, repeated failures, billing disputes >$50, or user requests escalation.提示:如果工具呼叫可能因無法預料的原因失敗,請加入清楚的失敗處理指示,讓模型能妥善回應。
各工具的行為設定
你可以針對特定工具細部調整模型的行為,而不必對所有工具套用同一條全域規則。例如,你可能希望模型主動呼叫讀取工具,但呼叫寫入工具前必須取得明確確認。
- 適用時機:關於主動性、確認或前置說明的全域指示,無法適用於每一項工具。
- 作用:為各項工具加入個別的行為規則,定義模型應該立即呼叫工具、先取得確認,還是在呼叫前先說一段前置說明。
範例
# TOOLS
- For the tools marked PROACTIVE: do not ask for confirmation from the user and do not output a preamble.
- For the tools marked as CONFIRMATION FIRST: always ask for confirmation to the user.
- For the tools marked as PREAMBLES: Before any tool call, say one short line like “I’m checking that now.” Then call the tool immediately.
## lookup_account(email_or_phone) — PROACTIVE
Use when: verifying identity or accessing billing.
Do NOT use when: caller refuses to identify after second request.
## check_outage(address) — PREAMBLES
Use when: caller reports failed connection or speed lower than 10 Mbps.
Do NOT use when: purely billing OR when internet speed is above 10 Mbps.
If either condition applies, inform the customer you cannot assist and hang up.
## refund_credit(account_id, minutes) — CONFIRMATION FIRST
Use when: confirmed outage > 240 minutes in the past 7 days (credit 60 minutes).
Do NOT use when: outage unconfirmed.
Confirmation phrase: “I can issue a credit for this outage—would you like me to go ahead?”
## schedule_technician(account_id, window) — CONFIRMATION FIRST
Use when: reboot + line checks fail AND outage=false.
Windows: “10am–12pm ET” or “2pm–4pm ET”.
Confirmation phrase: “I can schedule a technician to visit—should I book that for you?”
## escalate_to_human(account_id, reason) — PREAMBLES
Use when: harassment, threats, self-harm, repeated failure, billing disputes > $50, caller is frustrated, or caller requests escalation.
Preamble: “Let me connect you to a senior agent who can assist further.”工具輸出格式
某些工具輸出可能不符合模型的訓練資料分布,尤其是必須逐字重述的長字串。在訓練資料中,工具輸出通常是具有具名欄位的 JSON 物件。如果工具傳回原始字串,再另外要求模型「完全照原文重述」,模型可能更容易改寫、截斷內容,或混入自己的前置說明。
一個實用的解法是讓工具輸出採用一般工具結果的形式,並以機器能明確理解的方式表達逐字重述的要求。
-
適用時機: 工具傳回 冗長或複雜的結構化內容 (多句指示、交接資料、ID/連結、政策摘要、多步驟程序等),而你發現模型會 截斷、改寫、遺漏欄位、調整順序,或混入自己的前置說明/補充說明。
-
作用: 將工具輸出包在 精簡、明確的 JSON 外層結構 中(例如
response_text加上require_repeat_verbatim、format或content_type等旗標),使回應更 符合訓練資料分布 ,並讓 機器能清楚理解預期的內容呈現方式。 -
調整方式: 讓結構描述保持 精簡且穩定。在 工具使用指示 中與 工具定義 旁,清楚記載預期的工具輸出結構(例如:「如果
require_repeat_verbatim為 true,請原封不動地輸出response_text,不要加入其他內容」,或「請原樣呈現response_text;不要新增、遺漏工具輸出的欄位,也不要調整欄位順序。」)。
範例
範例:原始字串(較容易出錯)
工具傳回:
I just sent you an email with the verification link. Please open it and click “Confirm”.模型有時會這樣回覆:
-
「我已寄送驗證連結到您的電子郵件信箱……」(改述)
-
省略最後一句(截斷)
-
加入額外的話語(「還有其他需要幫忙的嗎?」)
範例:以 JSON 包裝(更符合訓練資料分布、更可靠)
工具回傳:
{
"response_text": "I just sent you an email with the verification link. Please open it and click “Confirm”.",
"require_repeat_verbatim": true
}由於這看起來像典型的工具結果(JSON 物件),模型通常更容易做到以下幾點:
-
辨識應以哪些內容為準(response_text)
-
理解輸出方式的限制(require_repeat_verbatim)
-
完整重現工具輸出,不截斷內容,也不加入額外的話語
改述監督工具的回覆(回應者與思考者架構)
在許多語音架構中,即時模型擔任回應者(與使用者說話),而能力更強的文字模型則擔任思考者(負責規劃、查詢政策及完成 SOP)。文字回覆不一定適合直接說出口,因此回應者必須在生成音訊前,將思考者的文字改述成適合語音表達的回覆。
- 適用時機:回應者收到思考者的回覆後,說出的內容聽起來像機器人、過於冗長或不自然。
- 作用:加入明確指示,引導回應者將思考者的文字改述為簡短、自然且以口語表達為主的回覆。
- 調整方式:調整措辭風格、開場白及長度限制,以符合使用案例的需求。
範例
# Tools
## Supervisor Tool
Name: getNextResponseFromSupervisor(relevantContextFromLastUserMessage: string)
When to call:
- Any request outside the allow list.
- Any factual, policy, account, or process question.
- Any action that might require internal lookups or system changes.
When not to call:
- Simple greetings and basic chitchat.
- Requests to repeat or clarify.
- Collecting parameters for later Supervisor use:
- phone_number for account help (getUserAccountInfo)
- zip_code for store lookup (findNearestStore)
- topic or keyword for policy lookup (lookupPolicyDocument)
Usage rules and preamble:
1) Say a neutral filler phrase to the user, then immediately call the tool. Approved fillers: “One moment.”, “Let me check.”, “Just a second.”, “Give me a moment.”, “Let me see.”, “Let me look into that.” Fillers must not imply success or failure.
2) Do not mention the “Supervisor” when responding with filler phrase.
3) relevantContextFromLastUserMessage is a one-line summary of the latest user message; use an empty string if nothing salient.
4) After the tool returns, apply Rephrase Supervisor and send your reply.
### Rephrase Supervisor
- Start with a brief conversational opener using active language, then flow into the answer (for example: “Thanks for waiting—”, “Just finished checking that.”, “I’ve got that pulled up now.”).
- Keep it short: no more than 2 sentences.
- Use this template: opener + one-sentence gist + up to 3 key details + a quick confirmation or choice (for example: “Does that match what you expected?”, “Want me to review options?”).
- Read numbers for speech: money naturally (“$45.20” → “forty-five dollars and twenty cents”), phone numbers 3-3-4, addresses with individual digits, dates/times plainly (“August twelfth”, “three-thirty p.m.”).以下是未加入改述指示的範例:
助理:您目前的信用卡餘額為正數,金額為 32,323,232 AUD。
以下是同一個範例加入改述指示後的結果:
助理:剛剛查好了,您的信用卡帳戶有三千兩百三十二萬三千兩百三十二元的結餘。您上次的付款已於八月一日處理完成。這與您預期的一致嗎?
常用工具
gpt-realtime-1.5 已接受訓練,能有效使用下列常用工具。如果您的使用案例需要類似行為,請讓工具名稱、簽章和描述盡量接近這些範例,以最大程度提高可靠性,並更貼近訓練資料分布。
以下是模型已接受訓練的幾項重要常用工具:
範例
# answer(question: string)
Description: Call this when the customer asks a question that you don't have an answer to or asks to perform an action.
# escalate_to_human()
Description: Call this when a customer asks for escalation, or to talk to someone else, or expresses dissatisfaction with the call.
# finish_session()
Description: Call this when a customer says they're done with the session or doesn't want to continue. If it's ambiguous, confirm with the customer before calling.對話流程
本節說明如何將對話劃分為清楚且目標明確的階段,讓模型確切知道每一步該做什麼。這包括定義各階段的目的、執行指示,以及轉往下一階段的具體「退出條件」。這樣能避免模型停滯、略過步驟或過早跳到後續階段,確保對話從問候到問題解決都井然有序。
此外,依照不同的對話狀態組織提示詞,也能讓您更容易找出錯誤模式,更有效地反覆改進。
- 適用時機:對話顯得雜亂無章、在達成目標前停滯,或模型難以有效完成目標。
- 作用:將互動劃分為多個階段,並為每個階段訂定明確的目標、指示及退出條件。
- 調整方式:依照您的工作流程重新命名各階段;修改各階段的指示,使其符合預期行為;讓「何時退出」的條件具體且精簡。
範例
# Conversation Flow
## 1) Greeting
Goal: Set tone and invite the reason for calling.
How to respond:
- Identify as NorthLoop Internet Support.
- Keep the opener brief and invite the caller’s goal.
- Confirm that customer is a Northloop customer
Exit to Discovery: Caller states they are a Northloop customer and mentions an initial goal or symptom.
## 2) Discover
Goal: Classify the issue and capture minimal details.
How to respond:
- Determine billing vs connectivity with one targeted question.
- For connectivity: collect the service address.
- For billing/account: collect email or phone used on the account.
Exit when: Intent and address (for connectivity) or email/phone (for billing) are known.
## 3) Verify
Goal: Confirm identity and retrieve the account.
How to respond:
- Once you have email or phone, call lookup_account(email_or_phone).
- If lookup fails, try the alternate identifier once; otherwise proceed with general guidance or offer escalation if account actions are required.
Exit when: Account ID is returned.
## 4) Diagnose
Goal: Decide outage vs local issue.
How to respond:
- For connectivity, call check_outage(address).
- If outage=true, skip local steps; move to Resolve with outage context.
- If outage=false, guide a short reboot/cabling check; confirm each step’s result before continuing.
Exit when: Root cause known.
## 5) Resolve
Goal: Apply fix, credit, or appointment.
How to respond:
- If confirmed outage > 240 minutes in the last 7 days, call refund_credit(account_id, 60).
- If outage=false and issue persists after basic checks, offer “10am–12pm ET” or “2pm–4pm ET” and call schedule_technician(account_id, chosen window).
- If the local fix worked, state the result and next steps briefly.
Exit when: A fix/credit/appointment has been applied and acknowledged by the caller.
## 6) Confirm/Close
Goal: Confirm outcome and end cleanly.
How to respond:
- Restate the result and any next step (e.g., stabilization window or tech ETA).
- Invite final questions; close politely if none.
Exit when: Caller declines more help.示範語句
示範語句可作為模型的「參考範例」,呈現您希望模型遵循的風格、精簡程度和語氣,同時不將它限制在單一僵化的回覆中。
- 適用時機:回覆缺乏您的品牌風格,或風格不一致。
- 作用:提供可供模型靈活變化的示範語句,讓回覆保持自然、簡短。
- 調整方式:將範例換成符合品牌風格的語句;保留「不要每次都使用」的提醒。
範例
# Sample Phrases
- Below are sample examples that you should use for inspiration. DO NOT ALWAYS USE THESE EXAMPLES, VARY YOUR RESPONSES.
Acknowledgements: “On it.” “One moment.” “Good question.”
Clarifiers: “Do you want A or B?” “What’s the deadline?”
Bridges: “Here’s the quick plan.” “Let’s keep it simple.”
Empathy (brief): “That’s frustrating—let’s fix it.”
Closers: “Anything else before we wrap?” “Happy to help next time.”注意:如果您的語音系統總是只重複示範語句,讓語音體驗變得更像機器人,請嘗試加入措辭多樣性的限制。我們曾觀察到這個方法能解決此問題。
對話流程 + 示範語句
在對話流程的不同狀態中加入示範語句,能讓模型知道什麼樣的回覆才合適,是一種實用的做法:
範例
# Conversation Flow
## 1) Greeting
Goal: Set tone and invite the reason for calling.
How to respond:
- Identify as NorthLoop Internet Support.
- Keep the opener brief and invite the caller’s goal.
Sample phrases (do not always repeat the same phrases, vary your responses):
- “Thanks for calling NorthLoop Internet—how can I help today?”
- “You’ve reached NorthLoop Support. What’s going on with your service?”
- “Hi there—tell me what you’d like help with.”
Exit when: Caller states an initial goal or symptom.
## 2) Discover
Goal: Classify the issue and capture minimal details.
How to respond:
- Determine billing vs connectivity with one targeted question.
- For connectivity: collect the service address.
- For billing/account: collect email or phone used on the account.
Sample phrases (do not always repeat the same phrases, vary your responses):
- “Is this about your bill or your internet speed?”
- “What address are you using for the connection?”
- “What’s the email or phone number on the account?”
Exit when: Intent and address (for connectivity) or email/phone (for billing) are known.
## 3) Verify
Goal: Confirm identity and retrieve the account.
How to respond:
- Once you have email or phone, call lookup_account(email_or_phone).
- If lookup fails, try the alternate identifier once; otherwise proceed with general guidance or offer escalation if account actions are required.
Sample phrases:
- “Thanks—looking up your account now.”
- “If that doesn’t pull up, what’s the other contact—email or phone?”
- “Found your account. I’ll take care of this.”
Exit when: Account ID is returned.
## 4) Diagnose
Goal: Decide outage vs local issue.
How to respond:
- For connectivity, call check_outage(address).
- If outage=true, skip local steps; move to Resolve with outage context.
- If outage=false, guide a short reboot/cabling check; confirm each step’s result before continuing.
Sample phrases (do not always repeat the same phrases, vary your responses):
- “I’m running a quick outage check for your area.”
- “No outage reported—let’s try a fast modem reboot.”
- “Please confirm the modem lights: is the internet light solid or blinking?”
Exit when: Root cause known.
## 5) Resolve
Goal: Apply fix, credit, or appointment.
How to respond:
- If confirmed outage > 240 minutes in the last 7 days, call refund_credit(account_id, 60).
- If outage=false and issue persists after basic checks, offer “10am–12pm ET” or “2pm–4pm ET” and call schedule_technician(account_id, chosen window).
- If the local fix worked, state the result and next steps briefly.
Sample phrases (do not always repeat the same phrases, vary your responses):
- “There’s been an extended outage—adding a 60-minute bill credit now.”
- “No outage—let’s book a technician. I can do 10am–12pm ET or 2pm–4pm ET.”
- “Credit applied—you’ll see it on your next bill.”
Exit when: A fix/credit/appointment has been applied and acknowledged by the caller.
## 6) Confirm/Close
Goal: Confirm outcome and end cleanly.
How to respond:
- Restate the result and any next step (e.g., stabilization window or tech ETA).
- Invite final questions; close politely if none.
Sample phrases (do not always repeat the same phrases, vary your responses):
- “We’re all set: [credit applied / appointment booked / service restored].”
- “You should see stable speeds within a few minutes.”
- “Your technician window is 10am–12pm ET.”
Exit when: Caller declines more help.
進階對話流程
隨著使用案例越來越複雜,您需要一種能隨需求擴充,同時維持模型效能的結構。關鍵在於兼顧可維護性與簡單性:過多僵化的狀態可能使模型負荷過重,影響表現,也讓對話顯得機械化。
更好的做法是設計能降低模型理解負擔的流程。以有結構但具彈性的方式處理狀態,能讓模型更容易保持專注、迅速回應,進而改善使用者體驗。
管理複雜情境的兩種常見模式如下:
- 以狀態機建立對話流程
- 透過 session.updates 建立動態對話流程
以狀態機建立對話流程
以 JSON 結構定義對話,將狀態與轉換規則一併納入。這樣能更容易評估涵蓋範圍、找出邊界情況,並追蹤歷次變更。由於這個結構以程式碼形式儲存,您可以隨著流程演進進行版本管理、比較差異及擴充。狀態機也能讓您精細控制對話從一個狀態轉換到另一個狀態的確切方式與時機。
範例
# Conversation States
[
{
"id": "1_greeting",
"description": "Begin each conversation with a warm, friendly greeting, identifying the service and offering help.",
"instructions": [
"Use the company name 'Snowy Peak Boards' and provide a warm welcome.",
"Let them know upfront that for any account-specific assistance, you’ll need some verification details."
],
"examples": [
"Hello, this is Snowy Peak Boards. Thanks for reaching out! How can I help you today?"
],
"transitions": [{
"next_step": "2_get_first_name",
"condition": "Once greeting is complete."
}, {
"next_step": "3_get_and_verify_phone",
"condition": "If the user provides their first name."
}]
},
{
"id": "2_get_first_name",
"description": "Ask for the user’s name (first name only).",
"instructions": [
"Politely ask, 'Who do I have the pleasure of speaking with?'",
"Do NOT verify or spell back the name; just accept it."
],
"examples": [
"Who do I have the pleasure of speaking with?"
],
"transitions": [{
"next_step": "3_get_and_verify_phone",
"condition": "Once name is obtained, OR name is already provided."
}]
},
{
"id": "3_get_and_verify_phone",
"description": "Request phone number and verify by repeating it back.",
"instructions": [
"Politely request the user’s phone number.",
"Once provided, confirm it by repeating each digit and ask if it’s correct.",
"If the user corrects you, confirm AGAIN to make sure you understand.",
],
"examples": [
"I'll need some more information to access your account if that's okay. May I have your phone number, please?",
"You said 0-2-1-5-5-5-1-2-3-4, correct?",
"You said 4-5-6-7-8-9-0-1-2-3, correct?"
],
"transitions": [{
"next_step": "4_authentication_DOB",
"condition": "Once phone number is confirmed"
}]
},
...動態對話流程
在這種模式中,系統會根據目前狀態更新系統提示詞和工具清單,讓對話即時調整。您只需提供與當前對話階段相關的內容,而不是一次將所有可能用到的規則和工具都提供給模型。
當某個狀態的結束條件達成時,您可以使用 session.update 進行轉換,將提示詞和工具替換為下一階段所需的內容。
這種做法能降低模型的認知負荷,讓它更容易處理複雜任務,不受不必要的上下文干擾。
範例
from typing import Literal
State = Literal["verify", "resolve"]
# Allowed transitions
TRANSITIONS: dict[State, list[State]] = {
"verify": ["resolve"],
"resolve": [], # terminal
}
def build_state_change_tool(current: State) -> dict:
allowed = TRANSITIONS[current]
readable = ", ".join(allowed) if allowed else "no further states (terminal)"
return {
"type": "function",
"name": "set_conversation_state",
"description": (
f"Switch the conversation phase. Current: '{current}'. "
f"You may switch only to: {readable}. "
"Call this AFTER exit criteria are satisfied."
),
"parameters": {
"type": "object",
"properties": {"next_state": {"type": "string", "enum": allowed}},
"required": ["next_state"],
},
}
# Minimal business tools per state
TOOLS_BY_STATE: dict[State, list[dict]] = {
"verify": [
{
"type": "function",
"name": "lookup_account",
"description": "Fetch account by email or phone.",
"parameters": {
"type": "object",
"properties": {"email_or_phone": {"type": "string"}},
"required": ["email_or_phone"],
},
}
],
"resolve": [
{
"type": "function",
"name": "schedule_technician",
"description": "Book a technician visit.",
"parameters": {
"type": "object",
"properties": {
"account_id": {"type": "string"},
"window": {"type": "string", "enum": ["10-12 ET", "14-16 ET"]},
},
"required": ["account_id", "window"],
},
}
],
}
# Short, phase-specific instructions
INSTRUCTIONS_BY_STATE: dict[State, str] = {
"verify": (
"# Role & Objective\n"
"Verify identity to access the account.\n\n"
"# Conversation (Verify)\n"
"- Ask for the email or phone on the account.\n"
"- Read back digits one-by-one (e.g., '4-1-5… Is that correct?').\n"
"Exit when: Account ID is returned.\n"
'When exit is satisfied: call set_conversation_state(next_state="resolve").'
),
"resolve": (
"# Role & Objective\n"
"Apply a fix by booking a technician.\n\n"
"# Conversation (Resolve)\n"
"- Offer two windows: '10–12 ET' or '2–4 ET'.\n"
"- Book the chosen window.\n"
"Exit when: Appointment is confirmed.\n"
"When exit is satisfied: end the call politely."
),
}
def build_session_update(state: State) -> dict:
"""Return the JSON payload for a Realtime `session.update` event."""
return {
"type": "session.update",
"session": {
"instructions": INSTRUCTIONS_BY_STATE[state],
"tools": TOOLS_BY_STATE[state] + [build_state_change_tool(state)],
},
}安全與轉交處理
對 Realtime 語音智慧體而言,具備可靠的轉交真人處理方式通常很重要。在本節中,您應根據使用案例,修改有關何時轉交處理的指示。
- 適用時機:模型難以正確判斷何時應轉交真人或備援系統處理
- 作用:定義快速、可靠的轉交處理流程,以及應說的話。
- 調整方式:填入您自己的觸發門檻,以及模型必須說的話。
範例
# Safety & Escalation
When to escalate (no extra troubleshooting):
- Safety risk (self-harm, threats, harassment)
- User explicitly asks for a human
- Severe dissatisfaction (e.g., “extremely frustrated,” repeated complaints, profanity)
- **2** failed tool attempts on the same task **or** **3** consecutive no-match/no-input events
- Out-of-scope or restricted (e.g., real-time news, financial/legal/medical advice)
What to say at the same time as calling the escalate_to_human tool (MANDATORY):
- “Thanks for your patience—I’m connecting you with a specialist now.”
- Then call the tool: `escalate_to_human`
Examples that would require escalation:
- “This is the third time the reset didn’t work. Just get me a person.”
- “I am extremely frustrated!”第一個範例展示 gpt-4o-realtime-preview-2025-06-03 使用這項指示後的對話回覆。

第二個範例展示 gpt-realtime-1.5 使用這項指示後的對話回覆。

gpt-realtime-1.5 能更可靠地遵循指示,並轉交人工處理。
後續步驟
若使用 GPT-Live:
- 參閱委派與工具,了解後端提示詞與由應用程式管理的上下文。
- 透過 WebRTC 或 WebSockets 連線。如需整合電話通訊,請參閱電話通訊與 SIP。
- 從對話品質與經驗證的任務成果兩方面評估語音智慧體。
若使用 Realtime:
- 參閱先前的即時互動提示詞指南,查看更多
gpt-realtime-1.5範例。 - 參閱 Realtime 評估指南,測試具代表性的語音智慧體行為。
- 透過 WebRTC、WebSockets 或 SIP 連線。
- 了解 Realtime 對話生命週期,並查看 Realtime 費用。