API Reference

Evals

Copy Markdown

Open in Claude

Open in ChatGPT

Open in Cursor

---

Copy Markdown

View as Markdown

Runs

Manage and run evals in the OpenAI platform.

Cancel eval run

evals.runs.cancel(run_id, **kwargs) -> RunCancelResponse { id, created_at, data_source, 11 more }

POST/evals/{eval_id}/runs/{run_id}

Create eval run

evals.runs.create(eval_id, **kwargs) -> RunCreateResponse { id, created_at, data_source, 11 more }

POST/evals/{eval_id}/runs

Delete eval run

evals.runs.delete(run_id, **kwargs) -> RunDeleteResponse { deleted, object, run_id }

DELETE/evals/{eval_id}/runs/{run_id}

Get eval runs

evals.runs.list(eval_id, **kwargs) -> CursorPage<RunListResponse { id, created_at, data_source, 11 more } >

GET/evals/{eval_id}/runs

Get an eval run

evals.runs.retrieve(run_id, **kwargs) -> RunRetrieveResponse { id, created_at, data_source, 11 more }

GET/evals/{eval_id}/runs/{run_id}

ModelsExpand Collapse

class CreateEvalCompletionsRunDataSource { source, type, input_messages, 2 more }

A CompletionsRunDataSource object describing a model sampling configuration.

class CreateEvalJSONLRunDataSource { source, type }

A JsonlRunDataSource object with that specifies a JSONL file that matches the eval

class EvalAPIError { code, message }

An object representing an error response from the Eval API.

class RunCancelResponse { id, created_at, data_source, 11 more }

A schema representing an evaluation run.

class RunCreateResponse { id, created_at, data_source, 11 more }

A schema representing an evaluation run.

class RunDeleteResponse { deleted, object, run_id }

class RunListResponse { id, created_at, data_source, 11 more }

A schema representing an evaluation run.

class RunRetrieveResponse { id, created_at, data_source, 11 more }

A schema representing an evaluation run.

RunsOutput Items

Manage and run evals in the OpenAI platform.

Get eval run output items

evals.runs.output_items.list(run_id, **kwargs) -> CursorPage<OutputItemListResponse { id, created_at, datasource_item, 7 more } >

GET/evals/{eval_id}/runs/{run_id}/output_items

Get an output item of an eval run

evals.runs.output_items.retrieve(output_item_id, **kwargs) -> OutputItemRetrieveResponse { id, created_at, datasource_item, 7 more }

GET/evals/{eval_id}/runs/{run_id}/output_items/{output_item_id}

ModelsExpand Collapse

class OutputItemListResponse { id, created_at, datasource_item, 7 more }

A schema representing an evaluation run output item.

class OutputItemRetrieveResponse { id, created_at, datasource_item, 7 more }

A schema representing an evaluation run output item.