Judge an agent's completed conversations

POST

evals

run

curl --request POST \
  --url https://api.sempera.org/v1/evals/run \
  --header 'Authorization: Bearer <token>' \
  --header 'Content-Type: application/json' \
  --data '
{
  "agent_id": "<string>"
}
'

import requests

url = "https://api.sempera.org/v1/evals/run"

payload = { "agent_id": "<string>" }
headers = {
    "Authorization": "Bearer <token>",
    "Content-Type": "application/json"
}

response = requests.post(url, json=payload, headers=headers)

print(response.text)

const options = {
  method: 'POST',
  headers: {Authorization: 'Bearer <token>', 'Content-Type': 'application/json'},
  body: JSON.stringify({agent_id: '<string>'})
};

fetch('https://api.sempera.org/v1/evals/run', options)
  .then(res => res.json())
  .then(res => console.log(res))
  .catch(err => console.error(err));

{
  "object": "eval_run",
  "agent_id": "<string>",
  "client": "<string>",
  "evaluated": 123,
  "unknown": 123,
  "skipped": 123,
  "run_id": "<string>"
}

{
  "error": {
    "code": "AGENT_ENV_UNPINNED",
    "message": "<string>",
    "request_id": "<string>",
    "field": "<string>",
    "scope": "<string>",
    "reason": "<string>"
  }
}

{
  "error": {
    "code": "AGENT_ENV_UNPINNED",
    "message": "<string>",
    "request_id": "<string>",
    "field": "<string>",
    "scope": "<string>",
    "reason": "<string>"
  }
}

{
  "error": {
    "code": "AGENT_ENV_UNPINNED",
    "message": "<string>",
    "request_id": "<string>",
    "field": "<string>",
    "scope": "<string>",
    "reason": "<string>"
  }
}

{
  "error": {
    "code": "AGENT_ENV_UNPINNED",
    "message": "<string>",
    "request_id": "<string>",
    "field": "<string>",
    "scope": "<string>",
    "reason": "<string>"
  }
}

{
  "error": {
    "code": "AGENT_ENV_UNPINNED",
    "message": "<string>",
    "request_id": "<string>",
    "field": "<string>",
    "scope": "<string>",
    "reason": "<string>"
  }
}

{
  "error": {
    "code": "AGENT_ENV_UNPINNED",
    "message": "<string>",
    "request_id": "<string>",
    "field": "<string>",
    "scope": "<string>",
    "reason": "<string>"
  }
}

Authorizations

Authorization

string

header

required

A workspace API key, sent as Authorization: Bearer sempera_sk_... Each operation names the one scope it needs. Scopes, key kinds and roles are in the Authentication guide.

Headers

Idempotency-Key

string

Replays the stored answer for a retry of the same request; 409 for a different request under the same key.

Maximum string length: 255

Body

application/json

agent_id

string

required

eval_id

string

Run one criterion of this workspace instead of all of them.

call_ids

string[]

Score only these calls.

Required array length: 1 - 200 elements

Response

Accepted

Evaluation counts and optional recorded run id. Evaluator diagnostics are not documented.

object

any

required

agent_id

string

required

client

string

required

evaluated

integer

required

unknown

integer

required

skipped

integer

required

run_id

string | null

Null when no run was recorded; absent on an evaluator failure.

⌘I

Judge an agent's completed conversations

curl --request POST \
  --url https://api.sempera.org/v1/evals/run \
  --header 'Authorization: Bearer <token>' \
  --header 'Content-Type: application/json' \
  --data '
{
  "agent_id": "<string>"
}
'

import requests

url = "https://api.sempera.org/v1/evals/run"

payload = { "agent_id": "<string>" }
headers = {
    "Authorization": "Bearer <token>",
    "Content-Type": "application/json"
}

response = requests.post(url, json=payload, headers=headers)

print(response.text)

const options = {
  method: 'POST',
  headers: {Authorization: 'Bearer <token>', 'Content-Type': 'application/json'},
  body: JSON.stringify({agent_id: '<string>'})
};

fetch('https://api.sempera.org/v1/evals/run', options)
  .then(res => res.json())
  .then(res => console.log(res))
  .catch(err => console.error(err));

{
  "object": "eval_run",
  "agent_id": "<string>",
  "client": "<string>",
  "evaluated": 123,
  "unknown": 123,
  "skipped": 123,
  "run_id": "<string>"
}

{
  "error": {
    "code": "AGENT_ENV_UNPINNED",
    "message": "<string>",
    "request_id": "<string>",
    "field": "<string>",
    "scope": "<string>",
    "reason": "<string>"
  }
}

{
  "error": {
    "code": "AGENT_ENV_UNPINNED",
    "message": "<string>",
    "request_id": "<string>",
    "field": "<string>",
    "scope": "<string>",
    "reason": "<string>"
  }
}

{
  "error": {
    "code": "AGENT_ENV_UNPINNED",
    "message": "<string>",
    "request_id": "<string>",
    "field": "<string>",
    "scope": "<string>",
    "reason": "<string>"
  }
}

{
  "error": {
    "code": "AGENT_ENV_UNPINNED",
    "message": "<string>",
    "request_id": "<string>",
    "field": "<string>",
    "scope": "<string>",
    "reason": "<string>"
  }
}

{
  "error": {
    "code": "AGENT_ENV_UNPINNED",
    "message": "<string>",
    "request_id": "<string>",
    "field": "<string>",
    "scope": "<string>",
    "reason": "<string>"
  }
}

{
  "error": {
    "code": "AGENT_ENV_UNPINNED",
    "message": "<string>",
    "request_id": "<string>",
    "field": "<string>",
    "scope": "<string>",
    "reason": "<string>"
  }
}