API Reference

Copy Markdown

Open in Claude

Open in ChatGPT

Open in Cursor

---

Copy Markdown

View as Markdown

Evals

Manage and run evals in the OpenAI platform.

Create eval

EvalCreateResponse evals().create(EvalCreateParamsparams, RequestOptionsrequestOptions = RequestOptions.none())

POST/evals

Delete an eval

EvalDeleteResponse evals().delete(EvalDeleteParamsparams = EvalDeleteParams.none(), RequestOptionsrequestOptions = RequestOptions.none())

DELETE/evals/{eval_id}

List evals

EvalListPage evals().list(EvalListParamsparams = EvalListParams.none(), RequestOptionsrequestOptions = RequestOptions.none())

GET/evals

Get an eval

EvalRetrieveResponse evals().retrieve(EvalRetrieveParamsparams = EvalRetrieveParams.none(), RequestOptionsrequestOptions = RequestOptions.none())

GET/evals/{eval_id}

Update an eval

EvalUpdateResponse evals().update(EvalUpdateParamsparams = EvalUpdateParams.none(), RequestOptionsrequestOptions = RequestOptions.none())

POST/evals/{eval_id}

ModelsExpand Collapse

class EvalCustomDataSourceConfig:

A CustomDataSourceConfig which specifies the schema of your item and optionally sample namespaces. The response schema defines the shape of the data that will be:

• Used to define your testing criteria and
• What data is required when creating a run

class EvalStoredCompletionsDataSourceConfig:

Deprecated in favor of LogsDataSourceConfig.

EvalsRuns

Manage and run evals in the OpenAI platform.

Cancel eval run

RunCancelResponse evals().runs().cancel(RunCancelParamsparams, RequestOptionsrequestOptions = RequestOptions.none())

POST/evals/{eval_id}/runs/{run_id}

Create eval run

RunCreateResponse evals().runs().create(RunCreateParamsparams, RequestOptionsrequestOptions = RequestOptions.none())

POST/evals/{eval_id}/runs

Delete eval run

RunDeleteResponse evals().runs().delete(RunDeleteParamsparams, RequestOptionsrequestOptions = RequestOptions.none())

DELETE/evals/{eval_id}/runs/{run_id}

Get eval runs

RunListPage evals().runs().list(RunListParamsparams = RunListParams.none(), RequestOptionsrequestOptions = RequestOptions.none())

GET/evals/{eval_id}/runs

Get an eval run

RunRetrieveResponse evals().runs().retrieve(RunRetrieveParamsparams, RequestOptionsrequestOptions = RequestOptions.none())

GET/evals/{eval_id}/runs/{run_id}

ModelsExpand Collapse

class CreateEvalCompletionsRunDataSource:

A CompletionsRunDataSource object describing a model sampling configuration.

class CreateEvalJsonlRunDataSource:

A JsonlRunDataSource object with that specifies a JSONL file that matches the eval

class EvalApiError:

An object representing an error response from the Eval API.

EvalsRunsOutput Items

Manage and run evals in the OpenAI platform.

Get eval run output items

OutputItemListPage evals().runs().outputItems().list(OutputItemListParamsparams, RequestOptionsrequestOptions = RequestOptions.none())

GET/evals/{eval_id}/runs/{run_id}/output_items

Get an output item of an eval run

OutputItemRetrieveResponse evals().runs().outputItems().retrieve(OutputItemRetrieveParamsparams, RequestOptionsrequestOptions = RequestOptions.none())

GET/evals/{eval_id}/runs/{run_id}/output_items/{output_item_id}