{"name":"eval","stage":"ga","description":"Package eval is scoring a model on your own data, with a judge you choose.","operations":[{"operationId":"get_eval_datasets","method":"GET","href":"/v1/eval/datasets","summary":"Is the datasets your org has, each with its name, description, metadata and timestamps."},{"operationId":"post_eval_datasets","method":"POST","href":"/v1/eval/datasets","summary":"Writes a dataset — the named set of graded examples a run scores a model against — under the caller's org and answers 201 with it."},{"operationId":"delete_eval_datasets_by_name","method":"DELETE","href":"/v1/eval/datasets/{name}","summary":"Removes the named dataset of the caller's org AND all of its examples, in one transaction."},{"operationId":"get_eval_datasets_by_name","method":"GET","href":"/v1/eval/datasets/{name}","summary":"Returns one dataset of the caller's org by name, together with its live item count — the one read that answers how big the set actually is."},{"operationId":"get_eval_datasets_by_name_items","method":"GET","href":"/v1/eval/datasets/{name}/items","summary":"Is the examples in one of your datasets — the set is named in the path, because this collection only exists inside one."},{"operationId":"post_eval_datasets_by_name_items","method":"POST","href":"/v1/eval/datasets/{name}/items","summary":"Writes one graded example — its input, its expected output, free-form metadata and a status — into the dataset named in the path, and answers 201 with it."},{"operationId":"get_eval_evaluators","method":"GET","href":"/v1/eval/evaluators","summary":"Is the judges your org has defined, each with its judge model, criteria and the score name it writes under."},{"operationId":"post_eval_evaluators","method":"POST","href":"/v1/eval/evaluators","summary":"Saves a reusable judge for the caller's org — the judge model and the written criteria it grades against — and answers 201 with it."},{"operationId":"get_eval_metrics","method":"GET","href":"/v1/eval/metrics","summary":"Is your org's AI overview board over a window: totals (generations, prompt and completion tokens, cost in cents, errors, success rate, distinct models and users), a gap-filled time series, a per-model breakdown with the long tail folded into \"other\", and latency percentiles read from the GenAI spans."},{"operationId":"get_eval_rubrics","method":"GET","href":"/v1/eval/rubrics","summary":"Is the score shapes your org has declared — each name's data type, its numeric bounds and its allowed categories."},{"operationId":"post_eval_rubrics","method":"POST","href":"/v1/eval/rubrics","summary":"Defines the shape of one score name for the caller's org and answers 201 with it."},{"operationId":"get_eval_runs","method":"GET","href":"/v1/eval/runs","summary":"Is your past runs and how they scored — the dataset and model, the judge model, how many examples were attempted and how many scored, the average score, and when it happened."},{"operationId":"post_eval_runs","method":"POST","href":"/v1/eval/runs","summary":"Runs a real evaluation and answers the summary when it is finished — this is synchronous work, not a job id."},{"operationId":"get_eval_scores","method":"GET","href":"/v1/eval/scores","summary":"Is the score events your org has recorded, narrowed by any of name, runName and traceId."},{"operationId":"post_eval_scores","method":"POST","href":"/v1/eval/scores","summary":"Files one score event for the caller's org and answers 201 with it."},{"operationId":"get_eval_traces","method":"GET","href":"/v1/eval/traces","summary":"Is the traces behind your evaluations — one per model call an evaluation made, carrying its input, output, model and timing — narrowed by any of sessionId, runName and datasetName."}],"_links":{"describedby":{"href":"/v1/openapi.json?capability=eval"},"self":{"href":"/v1/eval"},"up":{"href":"/v1"}}}