较新的语言模型可以处理和分析图像输入,这种能力称为 视觉。GPT Image 模型可以根据文本和图像输入创建新图像或编辑现有图像。
根据您是要分析图像还是生成图像,选择相应的端点:
如需了解我们模型支持的输入和输出模态,请参阅模型页面。
使用 Images API 时,选择 gpt-image-2.5-sunburst 来根据文本生成图像或编辑现有图像。使用 Responses API 时,选择支持图像生成工具的主线模型;该工具会负责选择 GPT Image 模型。
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20import OpenAI from "openai";
const openai = new OpenAI();
const response = await openai.responses.create({
model: "gpt-6-astra",
input:
"Generate an image of gray tabby cat hugging an otter with an orange scarf",
tools: [{ type: "image_generation" }],
});
// Save the image to a file
const imageData = response.output
.filter((output) => output.type === "image_generation_call")
.map((output) => output.result);
if (imageData.length > 0) {
const imageBase64 = imageData[0];
const fs = await import("fs");
fs.writeFileSync("cat_and_otter.png", Buffer.from(imageBase64, "base64"));
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22from openai import OpenAI
import base64
client = OpenAI()
response = client.responses.create(
model="gpt-6-astra",
input="Generate an image of gray tabby cat hugging an otter with an orange scarf",
tools=[{"type": "image_generation"}],
)
# Save the image to a file
image_data = [
output.result
for output in response.output
if output.type == "image_generation_call"
]
if image_data:
image_base64 = image_data[0]
with open("cat_and_otter.png", "wb") as f:
f.write(base64.b64decode(image_base64))
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43package main
import (
"context"
"encoding/base64"
"os"
"github.com/openai/openai-go/v3"
"github.com/openai/openai-go/v3/responses"
)
func main() {
client := openai.NewClient()
response, err := client.Responses.New(context.Background(), responses.ResponseNewParams{
Model: "gpt-6-astra",
Input: responses.ResponseNewParamsInputUnion{
OfString: openai.String("Generate an image of a gray tabby cat hugging an otter with an orange scarf."),
},
Tools: []responses.ToolUnionParam{{
OfImageGeneration: &responses.ToolImageGenerationParam{},
}},
})
if err != nil {
panic(err)
}
for _, output := range response.Output {
if output.Type != "image_generation_call" {
continue
}
image, err := base64.StdEncoding.DecodeString(output.AsImageGenerationCall().Result)
if err != nil {
panic(err)
}
if err := os.WriteFile("cat_and_otter.png", image, 0o600); err != nil {
panic(err)
}
return
}
panic("response did not include an image generation call")
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23import com.openai.client.OpenAIClient;
import com.openai.client.okhttp.OpenAIOkHttpClient;
import com.openai.models.responses.ResponseCreateParams;
import com.openai.models.responses.Tool;
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;
import java.util.Base64;
ResponseCreateParams params =
ResponseCreateParams.builder()
.model("gpt-6-astra")
.input("Generate an image of a gray tabby cat hugging an otter with an orange scarf.")
.addTool(Tool.ImageGeneration.builder().build())
.build();
String imageResult =
client.responses().create(params).output().stream()
.flatMap(item -> item.imageGenerationCall().stream())
.flatMap(call -> call.result().stream())
.findFirst()
.orElseThrow(() -> new IllegalStateException("No generated image returned"));
Files.write(Path.of("cat_and_otter.png"), Base64.getDecoder().decode(imageResult));
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28using OpenAI.Responses;
#pragma warning disable OPENAI001
string key = Environment.GetEnvironmentVariable("OPENAI_API_KEY")!;
ResponsesClient client = new(key);
CreateResponseOptions options = new()
{
Model = "gpt-6-astra",
};
options.InputItems.Add(
ResponseItem.CreateUserMessageItem(
"Generate an image of a gray tabby cat hugging an otter with an orange scarf."
)
);
options.Tools.Add(
ResponseTool.CreateImageGenerationTool(model: "gpt-image-2")
);
ResponseResult response = await client.CreateResponseAsync(options);
ImageGenerationCallResponseItem image = response
.OutputItems.OfType<ImageGenerationCallResponseItem>()
.FirstOrDefault()
?? throw new InvalidOperationException("No generated image was returned.");
await File.WriteAllBytesAsync(
"cat_and_otter.png",
image.ImageResultBytes.ToArray()
);
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21require "base64"
require "openai"
client = OpenAI::Client.new
response = client.responses.create(
model: "gpt-6-astra",
input: "Generate an image of a gray tabby cat hugging an otter with an orange scarf.",
tools: [{ type: :image_generation }]
)
image_call = response.output.find do |item|
item.is_a?(OpenAI::Models::Responses::ResponseOutputItem::ImageGenerationCall)
end
unless image_call.is_a?(OpenAI::Models::Responses::ResponseOutputItem::ImageGenerationCall)
raise "No image generation call returned"
end
File.binwrite(
"cat_and_otter.png",
Base64.strict_decode64(image_call.result)
)
1
2
3
4
5
6
7
8openai responses create \
--model gpt-6-astra \
--raw-output \
--transform 'output.#(type=="image_generation_call").result' <<'YAML' | base64 --decode > cat_and_otter.png
tools:
- type: image_generation
input: Generate an image of a gray tabby cat hugging an otter with an orange scarf.
YAML
GPT Image 模型无需参考图像即可利用世界知识。例如,要求生成一柜半宝石的提示可以生成包含紫水晶、粉晶和玉石等可辨认宝石的场景。
使用具备视觉能力的模型描述图像、读取可见文本,并回答有关物体、形状、颜色或纹理的问题。使用模型的回答时,请考虑其局限性。
使用完整的图像 URL 或 Base64 编码的数据 URL,提供要分析的图像。
您可以在 content 数组中包含多张图像,从而在单个请求中提供多张图像作为输入,但请注意,图像会计为 Token,并据此计费。
传入 URL
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22import OpenAI from "openai";
const openai = new OpenAI();
const response = await openai.chat.completions.create({
model: "gpt-6-astra",
messages: [
{
role: "user",
content: [
{ type: "text", text: "What is in this image?" },
{
type: "image_url",
image_url: {
url: "https://api.nga.gov/iiif/a2e6da57-3cd1-4235-b20e-95dcaefed6c8/full/!800,800/0/default.jpg",
},
},
],
},
],
});
console.log(response.choices[0].message.content);
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23from openai import OpenAI
client = OpenAI()
response = client.chat.completions.create(
model="gpt-6-astra",
messages=[
{
"role": "user",
"content": [
{"type": "text", "text": "What's in this image?"},
{
"type": "image_url",
"image_url": {
"url": "https://api.nga.gov/iiif/a2e6da57-3cd1-4235-b20e-95dcaefed6c8/full/!800,800/0/default.jpg",
},
},
],
}
],
)
print(response.choices[0].message.content)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29package main
import (
"context"
"fmt"
"github.com/openai/openai-go/v3"
)
func main() {
client := openai.NewClient()
completion, err := client.Chat.Completions.New(context.Background(), openai.ChatCompletionNewParams{
Model: "gpt-6-astra",
Messages: []openai.ChatCompletionMessageParamUnion{
openai.UserMessage([]openai.ChatCompletionContentPartUnionParam{
openai.TextContentPart("What's in this image?"),
openai.ImageContentPart(openai.ChatCompletionContentPartImageImageURLParam{
URL: "https://api.nga.gov/iiif/a2e6da57-3cd1-4235-b20e-95dcaefed6c8/full/!800,800/0/default.jpg",
}),
}),
},
})
if err != nil {
panic(err)
}
fmt.Println(completion.Choices[0].Message.Content)
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32import com.openai.client.OpenAIClient;
import com.openai.client.okhttp.OpenAIOkHttpClient;
import com.openai.models.chat.completions.ChatCompletionContentPart;
import com.openai.models.chat.completions.ChatCompletionContentPartImage;
import com.openai.models.chat.completions.ChatCompletionContentPartText;
import com.openai.models.chat.completions.ChatCompletionCreateParams;
import com.openai.models.chat.completions.ChatCompletionUserMessageParam;
import java.util.List;
String imageUrl =
"https://api.nga.gov/iiif/a2e6da57-3cd1-4235-b20e-95dcaefed6c8/full/!800,800/0/default.jpg";
ChatCompletionContentPart text =
ChatCompletionContentPart.ofText(
ChatCompletionContentPartText.builder().text("What's in this image?").build());
ChatCompletionContentPart image =
ChatCompletionContentPart.ofImageUrl(
ChatCompletionContentPartImage.builder()
.imageUrl(ChatCompletionContentPartImage.ImageUrl.builder().url(imageUrl).build())
.build());
ChatCompletionCreateParams params =
ChatCompletionCreateParams.builder()
.model("gpt-6-astra")
.addMessage(
ChatCompletionUserMessageParam.builder()
.contentOfArrayOfContentParts(List.of(text, image))
.build())
.build();
client.chat().completions().create(params).choices().stream()
.flatMap(choice -> choice.message().content().stream())
.forEach(System.out::println);
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19using OpenAI.Chat;
string key = Environment.GetEnvironmentVariable("OPENAI_API_KEY")!;
string model = "gpt-6-astra";
ChatClient client = new(model, key);
Uri imageUrl = new(
"https://api.nga.gov/iiif/a2e6da57-3cd1-4235-b20e-95dcaefed6c8/full/!800,800/0/default.jpg"
);
UserChatMessage message = new(
[
ChatMessageContentPart.CreateTextPart("What is in this image?"),
ChatMessageContentPart.CreateImagePart(imageUrl),
]
);
ChatCompletion completion = await client.CompleteChatAsync(message);
Console.WriteLine(completion.Content[0].Text);
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26require "openai"
client = OpenAI::Client.new
completion = client.chat.completions.create(
model: "gpt-6-astra",
messages: [
{
role: :user,
content: [
{
type: :text,
text: "What's in this image?"
},
{
type: :image_url,
image_url: {
url: "https://api.nga.gov/iiif/a2e6da57-3cd1-4235-b20e-95dcaefed6c8/full/!800,800/0/default.jpg"
}
}
]
}
]
)
puts(completion.choices.fetch(0).message.content)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24curl https://api.openai.com/v1/chat/completions \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $OPENAI_API_KEY" \
-d '{
"model": "gpt-6-astra",
"messages": [
{
"role": "user",
"content": [
{
"type": "text",
"text": "What is in this image?"
},
{
"type": "image_url",
"image_url": {
"url": "https://api.nga.gov/iiif/a2e6da57-3cd1-4235-b20e-95dcaefed6c8/full/!800,800/0/default.jpg"
}
}
]
}
],
"max_completion_tokens": 300
}'
传入 Base64 编码的图像
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27import fs from "fs";
import OpenAI from "openai";
const openai = new OpenAI();
const imagePath = "fixtures/example.jpg";
const base64Image = fs.readFileSync(imagePath, "base64");
const completion = await openai.chat.completions.create({
model: "gpt-6-astra",
messages: [
{
role: "user",
content: [
{ type: "text", text: "what's in this image?" },
{
type: "image_url",
image_url: {
url: `data:image/jpeg;base64,${base64Image}`,
},
},
],
},
],
});
console.log(completion.choices[0].message.content);
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37import base64
from openai import OpenAI
client = OpenAI()
# Function to encode the image
def encode_image(image_path):
with open(image_path, "rb") as image_file:
return base64.b64encode(image_file.read()).decode("utf-8")
# Path to your image
image_path = "path_to_your_image.jpg"
# Getting the Base64 string
base64_image = encode_image(image_path)
completion = client.chat.completions.create(
model="gpt-6-astra",
messages=[
{
"role": "user",
"content": [
{"type": "text", "text": "what's in this image?"},
{
"type": "image_url",
"image_url": {
"url": f"data:image/jpeg;base64,{base64_image}",
},
},
],
}
],
)
print(completion.choices[0].message.content)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34package main
import (
"context"
"encoding/base64"
"fmt"
"os"
"github.com/openai/openai-go/v3"
)
func main() {
client := openai.NewClient()
image, err := os.ReadFile("image.png")
if err != nil {
panic(err)
}
imageURL := "data:image/png;base64," + base64.StdEncoding.EncodeToString(image)
completion, err := client.Chat.Completions.New(context.Background(), openai.ChatCompletionNewParams{
Model: "gpt-6-astra",
Messages: []openai.ChatCompletionMessageParamUnion{
openai.UserMessage([]openai.ChatCompletionContentPartUnionParam{
openai.TextContentPart("What's in this image?"),
openai.ImageContentPart(openai.ChatCompletionContentPartImageImageURLParam{URL: imageURL}),
}),
},
})
if err != nil {
panic(err)
}
fmt.Println(completion.Choices[0].Message.Content)
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39import com.openai.client.OpenAIClient;
import com.openai.client.okhttp.OpenAIOkHttpClient;
import com.openai.models.chat.completions.ChatCompletionContentPart;
import com.openai.models.chat.completions.ChatCompletionContentPartImage;
import com.openai.models.chat.completions.ChatCompletionContentPartText;
import com.openai.models.chat.completions.ChatCompletionCreateParams;
import com.openai.models.chat.completions.ChatCompletionUserMessageParam;
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;
import java.util.Base64;
import java.util.List;
String imageUrl =
"data:image/jpeg;base64,"
+ Base64.getEncoder()
.encodeToString(
Files.readAllBytes(Path.of(System.getenv("OPENAI_EXAMPLE_IMAGE_PATH"))));
ChatCompletionContentPart text =
ChatCompletionContentPart.ofText(
ChatCompletionContentPartText.builder().text("What's in this image?").build());
ChatCompletionContentPart image =
ChatCompletionContentPart.ofImageUrl(
ChatCompletionContentPartImage.builder()
.imageUrl(ChatCompletionContentPartImage.ImageUrl.builder().url(imageUrl).build())
.build());
ChatCompletionCreateParams params =
ChatCompletionCreateParams.builder()
.model("gpt-6-astra")
.addMessage(
ChatCompletionUserMessageParam.builder()
.contentOfArrayOfContentParts(List.of(text, image))
.build())
.build();
client.chat().completions().create(params).choices().stream()
.flatMap(choice -> choice.message().content().stream())
.forEach(System.out::println);
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24using OpenAI.Chat;
string key = Environment.GetEnvironmentVariable("OPENAI_API_KEY")!;
string model = "gpt-6-astra";
ChatClient client = new(model, key);
Uri imageUrl = new(
"https://openai-documentation.vercel.app/images/cat_and_otter.png"
);
using HttpClient http = new();
BinaryData image = BinaryData.FromBytes(
await http.GetByteArrayAsync(imageUrl)
);
UserChatMessage message = new(
[
ChatMessageContentPart.CreateTextPart("What's in this image?"),
ChatMessageContentPart.CreateImagePart(image, "image/png"),
]
);
ChatCompletion completion = await client.CompleteChatAsync(message);
Console.WriteLine(completion.Content[0].Text);
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26require "base64"
require "openai"
client = OpenAI::Client.new
image = Base64.strict_encode64(File.binread("image.png"))
completion = client.chat.completions.create(
model: "gpt-6-astra",
messages: [
{
role: :user,
content: [
{
type: :text,
text: "What's in this image?"
},
{
type: :image_url,
image_url: { url: "data:image/png;base64,#{image}" }
}
]
}
]
)
puts(completion.choices.fetch(0).message.content)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23BASE64_IMAGE=$(base64 < path_to_your_image.jpg) && curl https://api.openai.com/v1/chat/completions -H "Content-Type: application/json" -H "Authorization: Bearer $OPENAI_API_KEY" -d @- <<EOF
{
"model": "gpt-6-astra",
"messages": [
{
"role": "user",
"content": [
{
"type": "text",
"text": "What is in this image?"
},
{
"type": "image_url",
"image_url": {
"url": "data:image/jpeg;base64,$BASE64_IMAGE"
}
}
]
}
],
"max_completion_tokens": 300
}
EOF
通过以下任一方式提供要分析的图像:
- 提供图像文件的完整 URL
- 以 Base64 编码的数据 URL 形式提供图像
- 提供文件 ID(通过 Files API 创建)
您可以在 content 数组中包含多张图像,从而在单个请求中提供多张图像作为输入,但请注意,图像会计为 Token,并据此计费。
传入 URL
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23import OpenAI from "openai";
const openai = new OpenAI();
const response = await openai.responses.create({
model: "gpt-6-astra",
input: [
{
role: "user",
content: [
{ type: "input_text", text: "what's in this image?" },
{
type: "input_image",
image_url:
"https://api.nga.gov/iiif/a2e6da57-3cd1-4235-b20e-95dcaefed6c8/full/!800,800/0/default.jpg",
detail: "auto",
},
],
},
],
});
console.log(response.output_text);
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21from openai import OpenAI
client = OpenAI()
response = client.responses.create(
model="gpt-6-astra",
input=[
{
"role": "user",
"content": [
{"type": "input_text", "text": "what's in this image?"},
{
"type": "input_image",
"image_url": "https://api.nga.gov/iiif/a2e6da57-3cd1-4235-b20e-95dcaefed6c8/full/!800,800/0/default.jpg",
},
],
}
],
)
print(response.output_text)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36package main
import (
"context"
"fmt"
"github.com/openai/openai-go/v3"
"github.com/openai/openai-go/v3/responses"
)
func main() {
client := openai.NewClient()
response, err := client.Responses.New(context.Background(), responses.ResponseNewParams{
Model: "gpt-6-astra",
Input: responses.ResponseNewParamsInputUnion{
OfInputItemList: responses.ResponseInputParam{
responses.ResponseInputItemParamOfMessage(
responses.ResponseInputMessageContentListParam{
responses.ResponseInputContentParamOfInputText("What's in this image?"),
{OfInputImage: &responses.ResponseInputImageParam{
Detail: responses.ResponseInputImageDetailAuto,
ImageURL: openai.String("https://api.nga.gov/iiif/a2e6da57-3cd1-4235-b20e-95dcaefed6c8/full/!800,800/0/default.jpg"),
}},
},
responses.EasyInputMessageRoleUser,
),
},
},
})
if err != nil {
panic(err)
}
fmt.Println(response.OutputText())
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31import com.openai.client.OpenAIClient;
import com.openai.client.okhttp.OpenAIOkHttpClient;
import com.openai.models.responses.ResponseCreateParams;
import com.openai.models.responses.ResponseInputImage;
import com.openai.models.responses.ResponseInputItem;
import java.util.List;
ResponseInputItem imageInput =
ResponseInputItem.ofMessage(
ResponseInputItem.Message.builder()
.role(ResponseInputItem.Message.Role.USER)
.addInputTextContent("What's in this image?")
.addContent(
ResponseInputImage.builder()
.detail(ResponseInputImage.Detail.AUTO)
.imageUrl(
"https://api.nga.gov/iiif/a2e6da57-3cd1-4235-b20e-95dcaefed6c8/full/!800,800/0/default.jpg")
.build())
.build());
ResponseCreateParams params =
ResponseCreateParams.builder()
.model("gpt-6-astra")
.inputOfResponse(List.of(imageInput))
.build();
client.responses().create(params).output().stream()
.flatMap(item -> item.message().stream())
.flatMap(message -> message.content().stream())
.flatMap(content -> content.outputText().stream())
.forEach(text -> System.out.println(text.text()));
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23using OpenAI.Responses;
#pragma warning disable OPENAI001
string key = Environment.GetEnvironmentVariable("OPENAI_API_KEY")!;
ResponsesClient client = new(key);
Uri imageUrl = new(
"https://api.nga.gov/iiif/a2e6da57-3cd1-4235-b20e-95dcaefed6c8/full/!800,800/0/default.jpg"
);
ResponseResult response = await client.CreateResponseAsync(
"gpt-6-astra",
[
ResponseItem.CreateUserMessageItem(
[
ResponseContentPart.CreateInputTextPart("What is in this image?"),
ResponseContentPart.CreateInputImagePart(imageUrl),
]
),
]
);
Console.WriteLine(response.GetOutputText());
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25require "openai"
client = OpenAI::Client.new
response = client.responses.create(
model: "gpt-6-astra",
input: [
{
role: :user,
content: [
{
type: :input_text,
text: "What's in this image?"
},
{
type: :input_image,
detail: :auto,
image_url: "https://api.nga.gov/iiif/a2e6da57-3cd1-4235-b20e-95dcaefed6c8/full/!800,800/0/default.jpg"
}
]
}
]
)
puts(response.output_text)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18curl https://api.openai.com/v1/responses \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $OPENAI_API_KEY" \
-d '{
"model": "gpt-6-astra",
"input": [
{
"role": "user",
"content": [
{"type": "input_text", "text": "what is in this image?"},
{
"type": "input_image",
"image_url": "https://api.nga.gov/iiif/a2e6da57-3cd1-4235-b20e-95dcaefed6c8/full/!800,800/0/default.jpg"
}
]
}
]
}'
1
2
3
4
5
6
7
8
9
10
11
12openai responses create \
--model gpt-6-astra \
--raw-output \
--transform 'output.#(type=="message").content.0.text' <<'YAML'
input:
- role: user
content:
- type: input_text
text: What is in this image?
- type: input_image
image_url: https://api.nga.gov/iiif/a2e6da57-3cd1-4235-b20e-95dcaefed6c8/full/!800,800/0/default.jpg
YAML
传入 Base64 编码的图像
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26import fs from "fs";
import OpenAI from "openai";
const openai = new OpenAI();
const imagePath = "fixtures/example.jpg";
const base64Image = fs.readFileSync(imagePath, "base64");
const response = await openai.responses.create({
model: "gpt-6-astra",
input: [
{
role: "user",
content: [
{ type: "input_text", text: "what's in this image?" },
{
type: "input_image",
image_url: `data:image/jpeg;base64,${base64Image}`,
detail: "auto",
},
],
},
],
});
console.log(response.output_text);
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36import base64
from openai import OpenAI
client = OpenAI()
# Function to encode the image
def encode_image(image_path):
with open(image_path, "rb") as image_file:
return base64.b64encode(image_file.read()).decode("utf-8")
# Path to your image
image_path = "path_to_your_image.jpg"
# Getting the Base64 string
base64_image = encode_image(image_path)
response = client.responses.create(
model="gpt-6-astra",
input=[
{
"role": "user",
"content": [
{"type": "input_text", "text": "what's in this image?"},
{
"type": "input_image",
"image_url": f"data:image/jpeg;base64,{base64_image}",
},
],
}
],
)
print(response.output_text)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43package main
import (
"context"
"encoding/base64"
"fmt"
"os"
"github.com/openai/openai-go/v3"
"github.com/openai/openai-go/v3/responses"
)
func main() {
client := openai.NewClient()
image, err := os.ReadFile("image.png")
if err != nil {
panic(err)
}
imageURL := "data:image/png;base64," + base64.StdEncoding.EncodeToString(image)
response, err := client.Responses.New(context.Background(), responses.ResponseNewParams{
Model: "gpt-6-astra",
Input: responses.ResponseNewParamsInputUnion{
OfInputItemList: responses.ResponseInputParam{
responses.ResponseInputItemParamOfMessage(
responses.ResponseInputMessageContentListParam{
responses.ResponseInputContentParamOfInputText("What's in this image?"),
{OfInputImage: &responses.ResponseInputImageParam{
Detail: responses.ResponseInputImageDetailAuto,
ImageURL: openai.String(imageURL),
}},
},
responses.EasyInputMessageRoleUser,
),
},
},
})
if err != nil {
panic(err)
}
fmt.Println(response.OutputText())
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39import com.openai.client.OpenAIClient;
import com.openai.client.okhttp.OpenAIOkHttpClient;
import com.openai.models.responses.ResponseCreateParams;
import com.openai.models.responses.ResponseInputImage;
import com.openai.models.responses.ResponseInputItem;
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;
import java.util.Base64;
import java.util.List;
String imageBase64 =
Base64.getEncoder()
.encodeToString(
Files.readAllBytes(Path.of(System.getenv("OPENAI_EXAMPLE_IMAGE_PATH"))));
ResponseInputItem imageInput =
ResponseInputItem.ofMessage(
ResponseInputItem.Message.builder()
.role(ResponseInputItem.Message.Role.USER)
.addInputTextContent("What's in this image?")
.addContent(
ResponseInputImage.builder()
.detail(ResponseInputImage.Detail.AUTO)
.imageUrl("data:image/png;base64," + imageBase64)
.build())
.build());
ResponseCreateParams params =
ResponseCreateParams.builder()
.model("gpt-6-astra")
.inputOfResponse(List.of(imageInput))
.build();
client.responses().create(params).output().stream()
.flatMap(item -> item.message().stream())
.flatMap(message -> message.content().stream())
.flatMap(content -> content.outputText().stream())
.forEach(text -> System.out.println(text.text()));
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47using OpenAI.Responses;
#pragma warning disable OPENAI001
string key = Environment.GetEnvironmentVariable("OPENAI_API_KEY")!;
ResponsesClient client = new(key);
Uri imageUrl = new(
"https://openai-documentation.vercel.app/images/cat_and_otter.png"
);
using HttpClient http = new();
// Download an image as a stream.
using Stream stream = await http.GetStreamAsync(imageUrl);
BinaryData imageData = BinaryData.FromStream(stream, "image/png");
ResponseResult response1 = await client.CreateResponseAsync(
"gpt-6-astra",
[
ResponseItem.CreateUserMessageItem(
[
ResponseContentPart.CreateInputTextPart("What is in this image?"),
ResponseContentPart.CreateInputImagePart(imageData),
]
),
]
);
Console.WriteLine($"From image stream: {response1.GetOutputText()}");
// Download an image as a byte array.
byte[] bytes = await http.GetByteArrayAsync(imageUrl);
imageData = BinaryData.FromBytes(bytes, "image/png");
ResponseResult response2 = await client.CreateResponseAsync(
"gpt-6-astra",
[
ResponseItem.CreateUserMessageItem(
[
ResponseContentPart.CreateInputTextPart("What is in this image?"),
ResponseContentPart.CreateInputImagePart(imageData),
]
),
]
);
Console.WriteLine($"From byte array: {response2.GetOutputText()}");
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27require "base64"
require "openai"
client = OpenAI::Client.new
image = Base64.strict_encode64(File.binread("image.png"))
response = client.responses.create(
model: "gpt-6-astra",
input: [
{
role: :user,
content: [
{
type: :input_text,
text: "What's in this image?"
},
{
type: :input_image,
detail: :auto,
image_url: "data:image/png;base64,#{image}"
}
]
}
]
)
puts(response.output_text)
传入文件 ID
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36import OpenAI from "openai";
import fs from "fs";
const openai = new OpenAI();
// Function to create a file with the Files API
async function createFile(filePath) {
const fileContent = fs.createReadStream(filePath);
const result = await openai.files.create({
file: fileContent,
purpose: "vision",
});
return result.id;
}
// Getting the file ID
const fileId = await createFile("fixtures/example.jpg");
const response = await openai.responses.create({
model: "gpt-6-astra",
input: [
{
role: "user",
content: [
{ type: "input_text", text: "what's in this image?" },
{
type: "input_image",
file_id: fileId,
detail: "auto",
},
],
},
],
});
console.log(response.output_text);
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35from openai import OpenAI
client = OpenAI()
# Function to create a file with the Files API
def create_file(file_path):
with open(file_path, "rb") as file_content:
result = client.files.create(
file=file_content,
purpose="vision",
)
return result.id
# Getting the file ID
file_id = create_file("path_to_your_image.jpg")
response = client.responses.create(
model="gpt-6-astra",
input=[
{
"role": "user",
"content": [
{"type": "input_text", "text": "what's in this image?"},
{
"type": "input_image",
"file_id": file_id,
},
],
}
],
)
print(response.output_text)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50package main
import (
"context"
"fmt"
"os"
"github.com/openai/openai-go/v3"
"github.com/openai/openai-go/v3/responses"
)
func main() {
client := openai.NewClient()
file, err := os.Open("image.png")
if err != nil {
panic(err)
}
defer file.Close()
uploaded, err := client.Files.New(context.Background(), openai.FileNewParams{
File: file,
Purpose: openai.FilePurposeVision,
})
if err != nil {
panic(err)
}
response, err := client.Responses.New(context.Background(), responses.ResponseNewParams{
Model: "gpt-6-astra",
Input: responses.ResponseNewParamsInputUnion{
OfInputItemList: responses.ResponseInputParam{
responses.ResponseInputItemParamOfMessage(
responses.ResponseInputMessageContentListParam{
responses.ResponseInputContentParamOfInputText("What's in this image?"),
{OfInputImage: &responses.ResponseInputImageParam{
Detail: responses.ResponseInputImageDetailAuto,
FileID: openai.String(uploaded.ID),
}},
},
responses.EasyInputMessageRoleUser,
),
},
},
})
if err != nil {
panic(err)
}
fmt.Println(response.OutputText())
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43import com.openai.client.OpenAIClient;
import com.openai.client.okhttp.OpenAIOkHttpClient;
import com.openai.models.files.FileCreateParams;
import com.openai.models.files.FilePurpose;
import com.openai.models.responses.ResponseCreateParams;
import com.openai.models.responses.ResponseInputImage;
import com.openai.models.responses.ResponseInputItem;
import java.nio.file.Path;
import java.util.List;
var file =
client
.files()
.create(
FileCreateParams.builder()
.file(Path.of(System.getenv("OPENAI_EXAMPLE_FILE_PATH")))
.purpose(FilePurpose.VISION)
.build());
var response =
client
.responses()
.create(
ResponseCreateParams.builder()
.model("gpt-6-astra")
.inputOfResponse(
List.of(
ResponseInputItem.ofMessage(
ResponseInputItem.Message.builder()
.role(ResponseInputItem.Message.Role.USER)
.addInputTextContent("What's in this image?")
.addContent(
ResponseInputImage.builder()
.detail(ResponseInputImage.Detail.AUTO)
.fileId(file.id())
.build())
.build())))
.build());
response.output().stream()
.flatMap(item -> item.message().stream())
.flatMap(message -> message.content().stream())
.flatMap(content -> content.outputText().stream())
.forEach(text -> System.out.println(text.text()));
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38using OpenAI.Files;
using OpenAI.Responses;
#pragma warning disable OPENAI001
string key = Environment.GetEnvironmentVariable("OPENAI_API_KEY")!;
ResponsesClient client = new(key);
string filename = "cat_and_otter.png";
Uri imageUrl = new(
$"https://openai-documentation.vercel.app/images/{filename}"
);
using HttpClient http = new();
// Download an image as a stream.
using Stream stream = await http.GetStreamAsync(imageUrl);
OpenAIFileClient files = new(key);
OpenAIFile file = await files.UploadFileAsync(
stream,
filename,
FileUploadPurpose.Vision
);
ResponseResult response = await client.CreateResponseAsync(
"gpt-6-astra",
[
ResponseItem.CreateUserMessageItem(
[
ResponseContentPart.CreateInputTextPart("what's in this image?"),
ResponseContentPart.CreateInputImagePart(file.Id),
]
),
]
);
Console.WriteLine(response.GetOutputText());
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30require "openai"
require "pathname"
client = OpenAI::Client.new
uploaded = client.files.create(
file: Pathname("image.png"),
purpose: :vision
)
response = client.responses.create(
model: "gpt-6-astra",
input: [
{
role: :user,
content: [
{
type: :input_text,
text: "What's in this image?"
},
{
type: :input_image,
detail: :auto,
file_id: uploaded.id
}
]
}
]
)
puts(response.output_text)
请使用受支持的图像文件,并确保图像足够清晰,以便模型分析。
| 要求 | 支持的输入 |
|---|
| 文件类型 | PNG(.png)、JPEG(.jpeg 或 .jpg)、WEBP(.webp)和非动画 GIF(.gif) |
| 请求大小 | 每个请求的总载荷最多为 512 MB |
| 图像数量 | 每个请求最多包含 1,500 张图像 |
对于基于图像块的图像输入,在应用所选模型和 detail 级别的缩放规则后,API 支持每张图像最多包含 30,000 个图像块。此限制适用于所有受支持的细节级别,并针对每张图像单独计算,而非针对请求中的图像块总数。
特定模型和细节级别规定的较低缩放预算仍然适用。处理后仍超过 30,000 个图像块限制的图像将被拒绝,系统不会自动缩放图像以满足此限制。请减小图像尺寸后重试。
图像 Token 和提示的其余部分也必须符合模型的输入和上下文限制。Token 估算结果并不保证请求满足所有输入限制。图像的使用必须遵守我们的使用政策。
detail 参数控制图像预处理。支持的值因模型而异:low、high、original 或 auto。如果省略此参数,Responses API 和 Chat Completions API 均默认使用 auto。模型尺寸处理表列出了相应的处理方式。
1
2
3
4"image_url": {
"url": "https://api.nga.gov/iiif/a2e6da57-3cd1-4235-b20e-95dcaefed6c8/full/!800,800/0/default.jpg",
"detail": "original"
},
1
2
3
4
5{
"type": "input_image",
"image_url": "https://api.nga.gov/iiif/a2e6da57-3cd1-4235-b20e-95dcaefed6c8/full/!800,800/0/default.jpg",
"detail": "original"
}
请根据以下指引选择细节级别:
| 细节级别 | 最适合 |
|---|
low | 粗略理解图像。缩放方式和 Token 用量取决于模型;low 使用的 Token 不一定比 high 少。 |
high | 无需精确原图坐标时的标准高保真图像理解。 |
original | 在模型支持的情况下,用于尺寸较大、信息密集、对空间位置敏感或用于计算机使用的图像。 |
auto | 使用模型的默认尺寸处理方式,具体见模型尺寸处理表。 |
对于需要精细视觉细节或精确坐标的任务,例如光学字符识别(OCR)、小物体检测或计算机使用,请在模型支持时使用 "detail": "original"。原始细节级别仍可能缩放图像,以满足模型的像素尺寸限制或缩放图像块预算,但不会为了满足单独设定的 30,000 个图像块拒绝阈值而缩放。对于对坐标敏感的任务,请在发送前缩放图像以满足这些限制,并将返回的坐标映射回原图。有关坐标处理,请参阅计算机使用指南。
下表汇总了通用视觉模型的尺寸处理方式。其他模型和专用变体可能采用不同的限制。所有尺寸调整都会保持宽高比,且不会放大较小的图像。
| 模型系列 |
支持的细节级别 |
图像块与缩放处理方式 |
|---|
gpt-6-astra | low、high、original、
auto
| low 将图像尺寸限制在 512 × 512 像素以内。high 允许最多
2,500 个图像块,且最大边长为 65,535 像素。这两项限制同时适用。
original 保留图像的原始尺寸,但如果图像
任一边长超过 65,535 像素,则会将其缩小至符合
该限制。如果处理后的图像需要超过
30,000 个图像块,API 会拒绝
该请求;不会为满足图像块数量限制而调整图像尺寸。
auto 的尺寸处理方式与 original 相同。
|
gpt-5.6-sol、gpt-5.6-terra、
gpt-5.6-luna
| low、high、original、
auto
| low 将图像限制在 512 × 512 像素以内。high 将图像限制在
2048 × 2048 像素和 2,500 个图像块以内。original
保留图像的原始尺寸,但任一边超过 65,535 像素的图像
会被缩小以满足该限制。如果处理后的
图像需要超过
30,000 个图像块,API 将拒绝
该请求;图像不会为满足图像块数量限制而缩放。
auto 采用与 original 相同的尺寸处理方式。
|
gpt-5.5 | low、high、original、
auto
| low 将图像限制在 512 × 512 像素以内。high 最多允许
2,500 个图像块,且最长边不超过 2048 像素。original
最多允许 10,000 个图像块,且最长边不超过 6000 像素。两项
限制同时适用。auto 采用与
original 相同的尺寸处理方式。
|
gpt-5.4、gpt-5.4-mini、gpt-5.4-nano
| low、high、original、
auto
| low 的最长边限制为 2048 像素,图像块预算为 6,144 个,
因此它可能比 high 使用更多 Token。
high 最多允许 2,500 个图像块,且最长边
不超过 2048 像素。original 最多允许 10,000 个图像块,且
最长边不超过 6000 像素。两项限制同时适用。auto
采用与 high 相同的尺寸处理方式。
|
gpt-5.2、gpt-4.1-mini
| low、high、auto
| 这些细节级别采用相同的尺寸限制:最长边不超过 2048 像素,
图像块预算为 6,144 个。不支持 original
。 |
gpt-5.1、gpt-4.1、gpt-4o、
gpt-4o-mini
| low、high、auto
| low 使用固定数量的 Token。high 和
auto 使用
基于瓦片的尺寸处理规则。
|
视觉模型将图像输入转换为可计费的输入 Token。图像输入成本计算器和本节中的图像块/瓦片规则适用于视觉模型的输入,不适用于 GPT Image 的图像生成或编辑。有关后者的单独定价,请参阅GPT Image 模型输入。
图像 Token 也会计入您的每分钟 Token 数(TPM)限制。计算器按标准输入费率估算一张图像的成本,不包含提示的其余部分或模型输出。
使用图像输入成本计算器,根据模型、图像尺寸和细节级别估算单张图像的输入 Token 数和成本。
某些模型通过用 32px x 32px 的图像块覆盖图像来进行 Token 化处理。许多模型与细节级别的组合都规定了缩放图像块预算。首先,API 会将图像调整到所选细节级别的像素尺寸限制以内,同时保留宽高比,并将尺寸舍入为整数像素,且不会放大较小的图像。随后按以下步骤确定 Token 成本:
A. 在应用像素尺寸限制后,计算覆盖图像需要多少个 32px x 32px 的图像块。图像块可以超出图像边界。
patch_count = ceil(width/32)×ceil(height/32)
B. 如果所选模型和细节级别规定了缩放图像块预算,且图像超出该预算,则按比例缩小图像。否则,跳过此步骤。调整缩放比例,确保转换为整数像素尺寸并计算覆盖所需的图像块数量后,仍不超出预算。在计算最终尺寸之前,始终保留完整精度。
shrink_factor = sqrt((32^2 * patch_budget) / (width * height))
adjusted_shrink_factor = shrink_factor * min(
floor(width * shrink_factor / 32) / (width * shrink_factor / 32),
floor(height * shrink_factor / 32) / (height * shrink_factor / 32)
)
C. 如果步骤 B 缩放了图像,将缩放后的最终宽度和高度向下取整为整数像素。计算覆盖所得图像所需的图像块数量。这就是应用模型乘数之前的图像 Token 数。如果设有图像块预算,此数量不会超出该预算。
resized_patch_count = ceil(resized_width/32)×ceil(resized_height/32)
如果此数量超过 30,000 个图像块,API 将拒绝请求。请在应用 Token 乘数之前检查此限制。
D. 将图像块数量乘以模型的乘数并向上取整,得到可计费的图像输入 Token 数。按模型的输入价格对这些 Token 计费一次;该乘数不适用于提示中的其他 Token,也不应再次应用于价格。
| 模型 | 乘数 |
|---|
gpt-6-astra | 1.2 |
gpt-5.6-sol | 1.2 |
gpt-5.6-terra | 1.2 |
gpt-5.6-luna | 1.2 |
gpt-5.5 | 1.2 |
gpt-5.4 | 1.2 |
gpt-5.4-mini | 1.2 |
gpt-5.4-nano | 1.2 |
gpt-5.2 | 1.2 |
gpt-5-mini* | 1.2 |
gpt-5-nano* | 1.5 |
gpt-4.1-mini | 1.62 |
gpt-4.1-nano*(2025-04-14 快照) | 2.46 |
o4-mini* | 1.72 |
对于 gpt-4.1-mini,此规则适用于 2025-04-14 快照。
* 已弃用,并已计划停用。有关日期和替代模型,请参阅弃用时间表。计算器和上方的模型尺寸调整行为表均未包含这些模型。
gpt-6-astra 在 detail: high 设置下的图像 Token 计算示例
此组合的最大边长为 65,535 像素,图像块预算为 2,500 个,乘数为 1.2 倍。
- 一张 1024 × 1024 的图像需要
32 × 32 = 1024 个图像块,无需调整尺寸。计费的图像输入 Token 数为 ceil(1024 × 1.2) = 1229。
- 一张 2048 × 2048 的图像最初需要
64 × 64 = 4096 个图像块。受图像块预算限制,图像会缩小至 1600 × 1600 像素,即 50 × 50 = 2500 个图像块。估算的 Token 数为 ceil(2500 × 1.2) = 3000。
- 一张 4096 × 512 的图像会保持原始尺寸:图像块数量为
128 × 16 = 2048,Token 数为 ceil(2048 × 1.2) = 2458。
计费过程中的浮点数舍入可能使最终数量与估算值相差一个 Token。
下表中的模型将基础 Token 数与图像瓦片的 Token 数相加:
| 模型 | 基础 Token 数 | 每个瓦片的 Token 数 |
|---|
gpt-5.1 | 70 | 140 |
gpt-5* | 70 | 140 |
gpt-4o、gpt-4.1 | 85 | 170 |
gpt-4o-mini | 2833 | 5667 |
o1*、o1-pro*、o3* | 75 | 150 |
* 已弃用,并已计划停用。有关日期和替代模型,请参阅弃用时间表。计算器和上方的模型尺寸调整行为表均未包含这些模型。
使用 "detail": "low" 时,无论图像尺寸如何,都只计入该模型的基础 Token 数。使用 "detail": "high" 或 "detail": "auto" 时:
- 保持宽高比,将图像缩小至 2048px x 2048px 的正方形范围内。较小的图像不会放大。
- 如果最短边超过 768px,则将其缩小至 768px,并将另一边的尺寸向下取整。
- 计算覆盖图像所需的边长为 512px 的正方形数量。每个正方形消耗该模型规定的每个瓦片的 Token 数。
- 将模型的基础 Token 数与瓦片的 Token 总数相加。
GPT Image 模型在生成和编辑图像时采用单独的图像 Token 定价。视觉计算器不估算这些模型的输入或输出费用。有关当前费率,请参阅图像生成定价;有关生成和编辑工作流,请参阅图像生成指南。
以下输入 Token 规则适用于 gpt-image-1。使用基于瓦片的图像尺寸调整方式,但将最短边缩小至 512px,而非 768px。Token 用量取决于图像尺寸以及 Images API 中的 input_fidelity 参数。
当输入保真度设为低时,基础消耗为 65 个图像 Token,每个瓦片消耗 129 个图像 Token。
使用高输入保真度时,除上述图像 Token 外,我们还会根据图像的宽高比增加一定数量的 Token。
- 如果您的图像为正方形,我们会额外增加 4160 个输入图像 Token。
- 如果图像更接近纵向或横向比例,我们会额外增加 6240 个 Token。
有关图像输入 Token 的定价,请参阅图像定价部分。
视觉模型可能出错。设计应用时,请考虑以下局限性:
- 医学影像:该模型不适合解读 CT 扫描等专业医学影像,也不应被用于提供医疗建议。
- 非英语文本:处理包含日语或韩语等非拉丁文字的图像时,模型的表现可能不够理想。
- 小字号文本:放大图像中的文本以提高可读性。如果支持,使用
"detail": "original" 也有助于改善表现。
- 旋转:模型可能会错误解读旋转或倒置的文本和图像。
- 视觉元素:对于使用不同颜色或样式(例如实线、虚线或点线)的图表或文本,模型可能难以理解。
- 空间推理:模型难以完成需要精确空间定位的任务,例如识别国际象棋棋子的位置。
- 准确性:在某些情况下,模型可能生成不正确的描述或图注。
- 图像形状:模型难以处理全景和鱼眼图像。
- 元数据与尺寸调整:模型不处理原始文件名或元数据。图像在分析前可能会被调整尺寸,即使使用
original 细节级别也是如此。有关各模型适用的限制,请参阅模型尺寸调整行为。
- 计数:模型对图像中物体的计数可能是近似值。
- 验证码:出于安全原因,我们的系统会阻止提交验证码。