Docs

The testworld API gives an agent a run on a seeded environment: a simulated company with Gmail, Slack and Calendar. Every tool call the agent makes is recorded, and when the run ends the end state is scored against fixed checks.

You create a run, point your agent at the run's tools address, finish the run and read back one object that holds the trace and the evaluation.

The hosted API opens with early access. The address and the key come with the invitation. In this page $TESTWORLD_URL is the address and $TESTWORLD_KEY is the key. Get early access.

Four steps to a first run.

The examples were recorded against the engine; only the address and the key were replaced. Long answers are indented and cut with ... where this page says so. Ids and times are the recorded ones, so yours differ.

Step 1

Create a run

Name an environment and a task. Schedule the Q4 offsite is the title of a task of offsite-planning, so the run takes that task and its five checks. The idempotency_key makes a retry safe. The answer holds tools_url, mcp_url and the run's token.

curl -s -X POST $TESTWORLD_URL/v1/runs \
  -H "Authorization: Bearer $TESTWORLD_KEY" \
  -H "Content-Type: application/json" \
  -d '{"environment":"offsite-planning","task":"Schedule the Q4 offsite","idempotency_key":"docs-quickstart-1","agent":{"name":"ops-assistant","revision":"1"}}'

201 Created
{
  "id": "run_5296fdfe-b495-4c5b-8e30-8f767d4c47dc",
  "status": "created",
  "ended_by": null,
  "error": null,
  "environment": {
    "id": "env_186c5910-e7d2-4c3c-bb5c-29da5672fbbe",
    "slug": "offsite-planning",
    "version": {
      "id": "ver_c8ab753a-2102-403f-9a20-502f3528935c",
      "number": 1
    }
  },
  "task": {
    "id": "task_schedule_q4_offsite",
    "text": "The leadership team wants a full-day Q4 offsite in Oslo. Read the 'Q4 offsite' mail thread for the proposed dates, check the six leaders' calendars for a working day between 12 and 14 November 2026 when all of them are free, confirm the date in #leadership on Slack, create an all-day event named 'Q4 offsite' on that day with the six leaders as attendees, and announce the date in #general. Do not send email: everything goes through Slack and Calendar."
  },
  "agent": {
    "name": "ops-assistant",
    "revision": "1"
  },
  "tools_url": "$TESTWORLD_URL/v1/runs/run_5296fdfe-b495-4c5b-8e30-8f767d4c47dc/tools",
  "mcp_url": "$TESTWORLD_URL/v1/runs/run_5296fdfe-b495-4c5b-8e30-8f767d4c47dc/tools/mcp",
  "tools": [
    {
      "provider": "google.gmail",
      "surface": "rest",
      "url": "$TESTWORLD_URL/v1/runs/run_5296fdfe-b495-4c5b-8e30-8f767d4c47dc/tools/gmail.googleapis.com/gmail/v1"
    },
    {
      "provider": "google.gmail",
      "surface": "mcp",
      "url": "$TESTWORLD_URL/v1/runs/run_5296fdfe-b495-4c5b-8e30-8f767d4c47dc/tools/gmailmcp.googleapis.com/mcp/v1"
    },
    {
      "provider": "slack",
      "surface": "web-api",
      "url": "$TESTWORLD_URL/v1/runs/run_5296fdfe-b495-4c5b-8e30-8f767d4c47dc/tools/slack.com/api"
    },
    {
      "provider": "slack",
      "surface": "mcp",
      "url": "$TESTWORLD_URL/v1/runs/run_5296fdfe-b495-4c5b-8e30-8f767d4c47dc/tools/mcp.slack.com/mcp"
    },
    {
      "provider": "google.calendar",
      "surface": "rest",
      "url": "$TESTWORLD_URL/v1/runs/run_5296fdfe-b495-4c5b-8e30-8f767d4c47dc/tools/www.googleapis.com/calendar/v3"
    }
  ],
  "now": "2026-10-26T08:00:00.000Z",
  "calls": 0,
  "max_calls": 500,
  "idle_timeout_seconds": 600,
  "timeout_seconds": 3600,
  "created_at": "2026-11-11T21:10:27.220Z",
  "started_at": null,
  "expires_at": "2026-11-11T22:10:27.220Z",
  "finished_at": null,
  "scored_at": null,
  "metadata": {},
  "trace": null,
  "evaluation": null,
  "token": "tw_run_6cdb03085eae616dbb91402d469952ef"
}

Keep the three values for the next steps: RUN_ID is id, TOOLS_URL is tools_url, RUN_TOKEN is token.

Step 2

Point the agent at tools_url

The agent talks to Gmail, Slack and Calendar under tools_url, on the real APIs' shape, with the run token as its bearer token. The first call searches the mailbox:

curl -s "$TOOLS_URL/gmail.googleapis.com/gmail/v1/users/me/messages?q=offsite" \
  -H "Authorization: Bearer $RUN_TOKEN"

200 OK
{
  "messages": [
    {
      "id": "m_q4_1",
      "threadId": "t_q4offsite"
    },
    {
      "id": "m_q4_2",
      "threadId": "t_q4offsite"
    },
    {
      "id": "m_q4_3",
      "threadId": "t_q4offsite"
    },
    {
      "id": "m_venue",
      "threadId": "t_venue"
    }
  ],
  "resultSizeEstimate": 4
}

Then it posts in #leadership. Two simulated colleagues answer this post (see Reactions); their replies are in the trace.

curl -s -X POST $TOOLS_URL/slack.com/api/chat.postMessage \
  -H "Authorization: Bearer $RUN_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"channel":"C_LEADERSHIP","text":"Proposal for the Q4 offsite: Friday 13 November, all day. It is the only working day all six of you are free."}'

200 OK
{
  "ok": true,
  "channel": "C_LEADERSHIP",
  "ts": "1793001600.000000",
  "message": {
    "type": "message",
    "user": "U_ASSIST",
    "text": "Proposal for the Q4 offsite: Friday 13 November, all day. It is the only working day all six of you are free.",
    "ts": "1793001600.000000"
  }
}

The recorded agent made five calls in all: a Gmail search, a Calendar free/busy query for the six leaders, this post, a Calendar event for Friday 13 November and an announcement in #general. Step 4 reads them back from the run; the Trace section shows one call in full. An MCP client connects to mcp_url instead of the addresses under tools_url.

Step 3

Finish the run

Hand over the agent's final answer. The run is sealed and moves to evaluating. The answer cuts tools (the same rows as in step 1) and the trace's calls.

curl -s -X POST $TESTWORLD_URL/v1/runs/$RUN_ID/finish \
  -H "Authorization: Bearer $TESTWORLD_KEY" \
  -H "Content-Type: application/json" \
  -d '{"output":"Booked the Q4 offsite for Friday 13 November, all day, with the six leaders as attendees. Confirmed the date in #leadership and announced it in #general."}'

200 OK
{
  "id": "run_5296fdfe-b495-4c5b-8e30-8f767d4c47dc",
  "status": "evaluating",
  "ended_by": "finish",
  "error": null,
  "environment": {
    "id": "env_186c5910-e7d2-4c3c-bb5c-29da5672fbbe",
    "slug": "offsite-planning",
    "version": {
      "id": "ver_c8ab753a-2102-403f-9a20-502f3528935c",
      "number": 1
    }
  },
  "task": {
    "id": "task_schedule_q4_offsite",
    "text": "The leadership team wants a full-day Q4 offsite in Oslo. Read the 'Q4 offsite' mail thread for the proposed dates, check the six leaders' calendars for a working day between 12 and 14 November 2026 when all of them are free, confirm the date in #leadership on Slack, create an all-day event named 'Q4 offsite' on that day with the six leaders as attendees, and announce the date in #general. Do not send email: everything goes through Slack and Calendar."
  },
  "agent": {
    "name": "ops-assistant",
    "revision": "1"
  },
  "tools_url": "$TESTWORLD_URL/v1/runs/run_5296fdfe-b495-4c5b-8e30-8f767d4c47dc/tools",
  "mcp_url": "$TESTWORLD_URL/v1/runs/run_5296fdfe-b495-4c5b-8e30-8f767d4c47dc/tools/mcp",
  "tools": ...,
  "now": "2026-10-26T08:00:00.005Z",
  "calls": 5,
  "max_calls": 500,
  "idle_timeout_seconds": 600,
  "timeout_seconds": 3600,
  "created_at": "2026-11-11T21:10:27.220Z",
  "started_at": "2026-11-11T21:10:38.505Z",
  "expires_at": "2026-11-11T22:10:27.220Z",
  "finished_at": "2026-11-11T21:10:38.666Z",
  "scored_at": null,
  "metadata": {},
  "trace": {
    "calls": ...
  },
  "evaluation": null
}

Step 4

Read the result

Read the run until status is done. The trace and the evaluation are on the run. The answer cuts tools and the trace's calls (see Trace). All five template checks passed, so the score is 1.

curl -s $TESTWORLD_URL/v1/runs/$RUN_ID \
  -H "Authorization: Bearer $TESTWORLD_KEY"

200 OK
{
  "id": "run_5296fdfe-b495-4c5b-8e30-8f767d4c47dc",
  "status": "done",
  "ended_by": "finish",
  "error": null,
  "environment": {
    "id": "env_186c5910-e7d2-4c3c-bb5c-29da5672fbbe",
    "slug": "offsite-planning",
    "version": {
      "id": "ver_c8ab753a-2102-403f-9a20-502f3528935c",
      "number": 1
    }
  },
  "task": {
    "id": "task_schedule_q4_offsite",
    "text": "The leadership team wants a full-day Q4 offsite in Oslo. Read the 'Q4 offsite' mail thread for the proposed dates, check the six leaders' calendars for a working day between 12 and 14 November 2026 when all of them are free, confirm the date in #leadership on Slack, create an all-day event named 'Q4 offsite' on that day with the six leaders as attendees, and announce the date in #general. Do not send email: everything goes through Slack and Calendar."
  },
  "agent": {
    "name": "ops-assistant",
    "revision": "1"
  },
  "tools_url": "$TESTWORLD_URL/v1/runs/run_5296fdfe-b495-4c5b-8e30-8f767d4c47dc/tools",
  "mcp_url": "$TESTWORLD_URL/v1/runs/run_5296fdfe-b495-4c5b-8e30-8f767d4c47dc/tools/mcp",
  "tools": ...,
  "now": "2026-10-26T08:00:00.005Z",
  "calls": 5,
  "max_calls": 500,
  "idle_timeout_seconds": 600,
  "timeout_seconds": 3600,
  "created_at": "2026-11-11T21:10:27.220Z",
  "started_at": "2026-11-11T21:10:38.505Z",
  "expires_at": "2026-11-11T22:10:27.220Z",
  "finished_at": "2026-11-11T21:10:38.666Z",
  "scored_at": "2026-11-11T21:10:38.676Z",
  "metadata": {},
  "trace": {
    "calls": ...
  },
  "evaluation": {
    "score": 1,
    "passed": true,
    "checks": [
      {
        "id": "chk_event_exists",
        "kind": "record_exists",
        "description": "A calendar event named Q4 offsite exists",
        "source": "template",
        "passed": true,
        "explanation": "1 record in events matches {\"summary\":\"Q4 offsite\"}",
        "judge": null
      },
      {
        "id": "chk_event_on_free_day",
        "kind": "field_equals",
        "description": "The offsite is an all-day event on Friday 13 November, the only working day all six leaders are free",
        "source": "template",
        "passed": true,
        "explanation": "a record in events matching {\"summary\":\"Q4 offsite\"} has start.date equal to \"2026-11-13\"",
        "judge": null
      },
      {
        "id": "chk_leadership_confirmed",
        "kind": "record_exists",
        "description": "The date was confirmed in #leadership",
        "source": "template",
        "passed": true,
        "explanation": "1 record in slack_messages matches {\"channelId\":\"C_LEADERSHIP\",\"user\":\"U_ASSIST\"}",
        "judge": null
      },
      {
        "id": "chk_general_told",
        "kind": "field_contains",
        "description": "#general was told about the offsite",
        "source": "template",
        "passed": true,
        "explanation": "a record in slack_messages matching {\"channelId\":\"C_GENERAL\",\"user\":\"U_ASSIST\"} has text containing \"offsite\"",
        "judge": null
      },
      {
        "id": "chk_no_email",
        "kind": "no_writes",
        "description": "No email was sent and the inbox was left as it was",
        "source": "template",
        "passed": true,
        "explanation": "0 writes to messages",
        "judge": null
      }
    ],
    "state_diff": {
      "events": {
        "created": [
          "1hog5eicv4280"
        ],
        "updated": [],
        "removed": []
      },
      "slack_messages": {
        "created": [
          "1793001600.000000",
          "1793001600.001000",
          "1793001600.002000",
          "1793001600.004000"
        ],
        "updated": [],
        "removed": []
      }
    },
    "evaluated_at": "2026-11-11T21:10:38.676Z"
  }
}

The four objects.

Environment, Run, Trace and Evaluation. Every field in the tables is present in every answer; null means "not yet" or "does not apply". Times are RFC 3339 strings in UTC. Ids are opaque strings: never parse one.

Environment

A set of simulated tools seeded with data. Every run starts from the same saved state. The API shows the latest published version; a run keeps the version it started on.

field	type	description
`id`	string	Opaque id, `env_…`.
`slug`	string	URL-safe name. Accepted wherever an environment id is.
`name`	string	Display name.
`description`	string	One paragraph on the office, the people and the seeded data.
`version`	object	The latest published version: `id` (`ver_…`), `number` (integer from 1) and `created_at`.
`tools`	array	One row per provider and surface: `provider` (`google.gmail`, `slack`, `google.calendar`), `surface` (`rest`, `web-api` or `mcp`) and `path`, the suffix under a run's `tools_url`.
`tasks`	array	Ready-made tasks. Each has `id` (`task_…`), `title`, `instructions` (what the agent is told) and `checks`, each marked `"source": "template"`.
`seed_digest`	string	`sha256:` and the digest of the seeded start state. Equal digests mean equal starting data.
`created_at`	time	When the environment was first published.

One example, recorded: GET /v1/environments/offsite-planning. The answer cuts the task's checks after the second (the Checks section shows all five in the evaluation).

200 OK
{
  "id": "env_186c5910-e7d2-4c3c-bb5c-29da5672fbbe",
  "slug": "offsite-planning",
  "name": "Offsite planning",
  "description": "Nordlys Analytics AS is a 14-person software company in Oslo working in English. The agent is its ops assistant, with its own mailbox, Slack user and calendar. The seed holds twelve mails (the three-message \"Q4 offsite\" thread and nine unrelated ones), fifteen Slack users in four channels with about thirty messages of office chatter, and Google Calendar entries for the six leaders from 9 to 24 November 2026. Simulated now is Monday 26 October 2026, 08:00 UTC.",
  "version": {
    "id": "ver_c8ab753a-2102-403f-9a20-502f3528935c",
    "number": 1,
    "created_at": "2026-11-11T21:10:23.774Z"
  },
  "tools": [
    {
      "provider": "google.gmail",
      "surface": "rest",
      "path": "gmail.googleapis.com/gmail/v1"
    },
    {
      "provider": "google.gmail",
      "surface": "mcp",
      "path": "gmailmcp.googleapis.com/mcp/v1"
    },
    {
      "provider": "slack",
      "surface": "web-api",
      "path": "slack.com/api"
    },
    {
      "provider": "slack",
      "surface": "mcp",
      "path": "mcp.slack.com/mcp"
    },
    {
      "provider": "google.calendar",
      "surface": "rest",
      "path": "www.googleapis.com/calendar/v3"
    }
  ],
  "tasks": [
    {
      "id": "task_schedule_q4_offsite",
      "title": "Schedule the Q4 offsite",
      "instructions": "The leadership team wants a full-day Q4 offsite in Oslo. Read the 'Q4 offsite' mail thread for the proposed dates, check the six leaders' calendars for a working day between 12 and 14 November 2026 when all of them are free, confirm the date in #leadership on Slack, create an all-day event named 'Q4 offsite' on that day with the six leaders as attendees, and announce the date in #general. Do not send email: everything goes through Slack and Calendar.",
      "checks": [
        {
          "collection": "events",
          "description": "A calendar event named Q4 offsite exists",
          "id": "chk_event_exists",
          "kind": "record_exists",
          "where": {
            "summary": "Q4 offsite"
          },
          "source": "template"
        },
        {
          "collection": "events",
          "description": "The offsite is an all-day event on Friday 13 November, the only working day all six leaders are free",
          "field": "start.date",
          "id": "chk_event_on_free_day",
          "kind": "field_equals",
          "value": "2026-11-13",
          "where": {
            "summary": "Q4 offsite"
          },
          "source": "template"
        },
        ...
      ]
    }
  ],
  "seed_digest": "sha256:042017497c22d5f0d9a31be27373dddc5d721c5bdc7d2ee3044774f48faad71d",
  "created_at": "2026-11-11T21:10:23.764Z"
}

Run

One agent doing one task in one environment. It lives while the agent calls the tools, ends exactly once and is then evaluated. Every route that returns a run returns this object. token is only in the answer of POST /v1/runs and GET /v1/runs/{id}/tools.

Status

status	meaning
`created`	No call has reached the tools yet.
`running`	The first call has been answered. `started_at` is set. A run that never makes a call skips this state.
`evaluating`	The run has ended and the checks are being evaluated. The tools answer `409 run_ended`.
`done`	The evaluation is on the run and `scored_at` is set.
`failed`	Only when scoring itself breaks: `error` holds `type` `evaluation_failed` and a message. The trace is still there.

A rubric check without a judge model is not a failure. It is undecided: passed is null, judge is null and the explanation starts with undecided:. It counts as not passed, and the run still ends done.

field	type	description
`id`	string	`run_…`.
`status`	string	`created`, `running`, `evaluating`, `done` or `failed`.
`ended_by`	string or null	`finish`, `idle_timeout`, `timeout` or `max_calls`. `null` until the run ends.
`error`	object or null	Set only when `status` is `failed`: `type` and `message`.
`environment`	object	`id`, `slug` and `version` (`id`, `number`): the version the run is pinned to.
`task`	object	`id` (the template task's id, `null` for free text) and `text` (the template task's instructions, or the free text as given).
`agent`	object	`name` and `revision`, as given. Both `null` when left out.
`tools_url`	string	The agent's base address for the tools. See The tools under tools_url.
`mcp_url`	string	`{tools_url}/mcp`: one MCP address for the Gmail and Slack tools.
`tools`	array	One row per provider and surface: `provider`, `surface` and `url` (absolute).
`token`	string	The run's credential for the tools, `tw_run_…`. Valid until `expires_at`.
`now`	time	Simulated time: the environment's clock after the last call.
`calls`	integer	Calls so far. See how calls are counted.
`max_calls`	integer	The cap on `calls`.
`idle_timeout_seconds`	integer	The idle window.
`timeout_seconds`	integer	The hard cap, from `created_at`.
`created_at`	time	When the run was created.
`started_at`	time or null	When the first call was answered.
`expires_at`	time	`created_at` plus `timeout_seconds`.
`finished_at`	time or null	When the run ended.
`scored_at`	time or null	When the evaluation was stored.
`metadata`	object	Free-form, as given at creation, returned unchanged. At most 4 KB.
`trace`	object or null	`{"calls": [...]}` once the run has ended, `null` before.
`evaluation`	object or null	Set when `status` is `done`, `null` before.

One example, recorded: the finished quickstart run. The answer cuts tools, the trace's calls and the evaluation; both are shown below.

200 OK
{
  "id": "run_5296fdfe-b495-4c5b-8e30-8f767d4c47dc",
  "status": "done",
  "ended_by": "finish",
  "error": null,
  "environment": {
    "id": "env_186c5910-e7d2-4c3c-bb5c-29da5672fbbe",
    "slug": "offsite-planning",
    "version": {
      "id": "ver_c8ab753a-2102-403f-9a20-502f3528935c",
      "number": 1
    }
  },
  "task": {
    "id": "task_schedule_q4_offsite",
    "text": "The leadership team wants a full-day Q4 offsite in Oslo. Read the 'Q4 offsite' mail thread for the proposed dates, check the six leaders' calendars for a working day between 12 and 14 November 2026 when all of them are free, confirm the date in #leadership on Slack, create an all-day event named 'Q4 offsite' on that day with the six leaders as attendees, and announce the date in #general. Do not send email: everything goes through Slack and Calendar."
  },
  "agent": {
    "name": "ops-assistant",
    "revision": "1"
  },
  "tools_url": "$TESTWORLD_URL/v1/runs/run_5296fdfe-b495-4c5b-8e30-8f767d4c47dc/tools",
  "mcp_url": "$TESTWORLD_URL/v1/runs/run_5296fdfe-b495-4c5b-8e30-8f767d4c47dc/tools/mcp",
  "tools": ...,
  "now": "2026-10-26T08:00:00.005Z",
  "calls": 5,
  "max_calls": 500,
  "idle_timeout_seconds": 600,
  "timeout_seconds": 3600,
  "created_at": "2026-11-11T21:10:27.220Z",
  "started_at": "2026-11-11T21:10:38.505Z",
  "expires_at": "2026-11-11T22:10:27.220Z",
  "finished_at": "2026-11-11T21:10:38.666Z",
  "scored_at": "2026-11-11T21:10:38.676Z",
  "metadata": {},
  "trace": {
    "calls": ...
  },
  "evaluation": ...
}

Checks

A check is one assertion a run is scored on. A task's template checks always run, unless the request sets replace_template_checks to true. Add your own in checks on POST /v1/runs. A custom check is sent without id and source: the API assigns the id and sets source to custom. description is optional and defaults to a line built from the kind and its fields. At most one rubric check per run, counting the template's.

kind	required	optional	passes when
`record_exists`	collection, where		A record in the collection matches `where` at the end.
`record_absent`	collection, where		No record in the collection matches `where`.
`field_equals`	collection, field, value	where	A matching record has `field` exactly equal to `value`.
`field_contains`	collection, field, value	where	A matching record has `value` in `field`, case-insensitive.
`write_count`		collection, min, max	The number of records created, updated and removed, in that collection or in all, is within `min` and `max`.
`no_writes`		collection	That number is zero.
`action_called`	action		A call in the trace has that `operation`.
`action_not_called`	action		No call has that `operation`.
`output_contains`	value		The `output` sent to finish contains `value`, case-insensitive.
`output_equals`	value		The `output` equals `value` exactly.
`rubric`	text		A language model reads the task, the trace, the state diff and the output and answers yes or no to `text`. Use it only for what a fixed check cannot decide.

where is an object of field to value. A string value matches as a case-insensitive substring; any other value must be equal. field and the keys of where may be dotted paths such as start.date. min and max are integers from 0.

Collection and action names are the environment's own; a task's template checks show the names in use. For offsite-planning the collections are events, slack_messages, messages, drafts, labels, channels, users and calendars. A collection the environment does not have is refused with 400 invalid_request. Actions are the operation names of the trace, such as slack.chat.postMessage.

{
  "environment": "offsite-planning",
  "task": "Schedule the Q4 offsite",
  "checks": [
    {"kind": "field_contains", "collection": "slack_messages",
     "where": {"channelId": "C_GENERAL"}, "field": "text", "value": "13 November"},
    {"kind": "action_not_called", "action": "gmail.messages.send"}
  ]
}

That is a request body, not a recorded answer. Its kinds and field names are the ones the engine validates.

Trace

The ordered record of every call the tools answered during the run, reads included. It is set on the run once the run has ended. Nothing from inside the agent is in it.

field	type	description
`seq`	integer	Position in the run, from 1.
`at`	time	Simulated time when the call was answered.
`provider`	string	`google.gmail`, `slack` or `google.calendar`.
`surface`	string	`rest`, `web-api` or `mcp`.
`operation`	string	The operation name: `gmail.messages.list`, `slack.chat.postMessage`, `calendar.events.insert`. On the `mcp` surface it is the MCP tool's name, for example `gmail.search_messages` or `slack.read_channel`.
`arguments`	object	What the agent sent: path, query and body merged into one object for REST and Web API calls; the tool arguments for MCP.
`result`	any	The body the tool answered, as the agent saw it.
`status`	string	`ok` or `error`. `error` means the tool refused or failed the call.
`error`	object or null	`type` and `message` when `status` is `error`.
`mutated`	boolean	Whether the call changed the environment's state.
`state_digest`	string	`sha256:` digest of the whole state after the call, after any reactions.
`reactions`	array	Replies simulated colleagues posted because of this call: `{id, user, ts, text}`. `[]` when none.

One example, recorded: call 3 of the quickstart run, the post in #leadership. Erik and Mari answered it.

{
  "seq": 3,
  "at": "2026-10-26T08:00:00.001Z",
  "provider": "slack",
  "surface": "web-api",
  "operation": "slack.chat.postMessage",
  "arguments": {
    "channel": "C_LEADERSHIP",
    "text": "Proposal for the Q4 offsite: Friday 13 November, all day. It is the only working day all six of you are free."
  },
  "result": {
    "ok": true,
    "channel": "C_LEADERSHIP",
    "ts": "1793001600.000000",
    "message": {
      "type": "message",
      "user": "U_ASSIST",
      "text": "Proposal for the Q4 offsite: Friday 13 November, all day. It is the only working day all six of you are free.",
      "ts": "1793001600.000000"
    }
  },
  "status": "ok",
  "error": null,
  "mutated": true,
  "state_digest": "sha256:33e20882195ca32a450b6251d29257b3a7dc325cfee6a8d67cdd3e7ee32213f0",
  "reactions": [
    {
      "id": "rx_erik_date",
      "user": "U_ERIK",
      "ts": "1793001600.001000",
      "text": "Thu 12 is out for me, I'm at the partner day in Stockholm. Fri 13 works."
    },
    {
      "id": "rx_mari_go",
      "user": "U_MARI",
      "ts": "1793001600.002000",
      "text": "13th works for me too. Book it and tell #general."
    }
  ]
}

Evaluation

The end state and the agent's output compared with the run's checks: the task's template checks, then the custom ones. It is set when status is done.

field	type	description
`score`	number or null	Passed checks divided by all checks, 0 to 1, rounded to two places. A check that could not be decided counts as not passed. `null` when the run had no checks.
`passed`	boolean or null	`true` only when every check passed. `null` when the run had no checks.
`checks`	array	One result per check: `id`, `kind`, `description`, `source` (`template` or `custom`), `passed` (`true`, `false`, or `null` when undecided), `explanation` (one line on what was found) and `judge` (for `rubric` only: `{model}`; otherwise `null`).
`state_diff`	object	What changed, per collection: `{"created": [ids], "updated": [ids], "removed": [ids]}`. Collections with no change are left out. Colleagues' replies are in it.
`evaluated_at`	time	When the evaluation was stored.

One example, recorded: the evaluation of the quickstart run.

{
  "score": 1,
  "passed": true,
  "checks": [
    {
      "id": "chk_event_exists",
      "kind": "record_exists",
      "description": "A calendar event named Q4 offsite exists",
      "source": "template",
      "passed": true,
      "explanation": "1 record in events matches {\"summary\":\"Q4 offsite\"}",
      "judge": null
    },
    {
      "id": "chk_event_on_free_day",
      "kind": "field_equals",
      "description": "The offsite is an all-day event on Friday 13 November, the only working day all six leaders are free",
      "source": "template",
      "passed": true,
      "explanation": "a record in events matching {\"summary\":\"Q4 offsite\"} has start.date equal to \"2026-11-13\"",
      "judge": null
    },
    {
      "id": "chk_leadership_confirmed",
      "kind": "record_exists",
      "description": "The date was confirmed in #leadership",
      "source": "template",
      "passed": true,
      "explanation": "1 record in slack_messages matches {\"channelId\":\"C_LEADERSHIP\",\"user\":\"U_ASSIST\"}",
      "judge": null
    },
    {
      "id": "chk_general_told",
      "kind": "field_contains",
      "description": "#general was told about the offsite",
      "source": "template",
      "passed": true,
      "explanation": "a record in slack_messages matching {\"channelId\":\"C_GENERAL\",\"user\":\"U_ASSIST\"} has text containing \"offsite\"",
      "judge": null
    },
    {
      "id": "chk_no_email",
      "kind": "no_writes",
      "description": "No email was sent and the inbox was left as it was",
      "source": "template",
      "passed": true,
      "explanation": "0 writes to messages",
      "judge": null
    }
  ],
  "state_diff": {
    "events": {
      "created": [
        "1hog5eicv4280"
      ],
      "updated": [],
      "removed": []
    },
    "slack_messages": {
      "created": [
        "1793001600.000000",
        "1793001600.001000",
        "1793001600.002000",
        "1793001600.004000"
      ],
      "updated": [],
      "removed": []
    }
  },
  "evaluated_at": "2026-11-11T21:10:38.676Z"
}

The six routes.

All routes are under /v1/ and take Authorization: Bearer $TESTWORLD_KEY. Bodies and answers are JSON with snake_case names. A request field this page does not list is refused with 400 invalid_request. Errors are described in Errors.

GET /v1/environments

Lists the environments. Query: limit (1 to 200, default 50) and cursor (the next_cursor of the previous page). Answer 200: {"data": [Environment], "next_cursor": string or null}. Errors: 401. The example cuts tools and tasks.

curl -s $TESTWORLD_URL/v1/environments \
  -H "Authorization: Bearer $TESTWORLD_KEY"

200 OK
{
  "data": [
    {
      "id": "env_186c5910-e7d2-4c3c-bb5c-29da5672fbbe",
      "slug": "offsite-planning",
      "name": "Offsite planning",
      "description": "Nordlys Analytics AS is a 14-person software company in Oslo working in English. The agent is its ops assistant, with its own mailbox, Slack user and calendar. The seed holds twelve mails (the three-message \"Q4 offsite\" thread and nine unrelated ones), fifteen Slack users in four channels with about thirty messages of office chatter, and Google Calendar entries for the six leaders from 9 to 24 November 2026. Simulated now is Monday 26 October 2026, 08:00 UTC.",
      "version": {
        "id": "ver_c8ab753a-2102-403f-9a20-502f3528935c",
        "number": 1,
        "created_at": "2026-11-11T21:10:23.774Z"
      },
      "tools": ...,
      "tasks": ...,
      "seed_digest": "sha256:042017497c22d5f0d9a31be27373dddc5d721c5bdc7d2ee3044774f48faad71d",
      "created_at": "2026-11-11T21:10:23.764Z"
    }
  ],
  "next_cursor": null
}

GET /v1/environments/{id}

One environment, by id or slug. Answer 200: an Environment (the example is under Environment). Errors: 401, 404 not_found.

curl -s $TESTWORLD_URL/v1/environments/offsite-planning \
  -H "Authorization: Bearer $TESTWORLD_KEY"

POST /v1/runs

Starts a run. Answer 201: a Run in status created, with token. Errors: 400 invalid_request, 401, 404 not_found (no such environment), 409 idempotency_conflict. The quickstart's step 1 is the recorded example.

field	type	description
`environment`	string	Required. An environment id or slug.
`task`	string	Required, at most 8 KB. A task id (`task_…`), the title of one of the environment's tasks (case-insensitive), or free text. A template task brings its checks and its instructions; free text is the agent's task with no template checks. An unknown id that starts with `task_` is refused.
`checks`	array	Your own checks, in addition to the template's. See Checks.
`replace_template_checks`	boolean	`true` drops the task's template checks, so only `checks` are scored.
`agent`	object	`name` and `revision`, both strings. Returned on the run.
`max_calls`	integer	1 to 500, default 500. The run ends when this many calls have been answered.
`idle_timeout_seconds`	integer	30 to 3600, default 600.
`timeout_seconds`	integer	60 to 86400, default 3600.
`idempotency_key`	string	1 to 255 characters. The same key with the same body replays the first run (answer `201`, same run). The same key with a different body is `409 idempotency_conflict`. Keys are kept.
`metadata`	object	Free-form, at most 4 KB. Returned unchanged.

GET /v1/runs/{id}

Reads a run. Answer 200: a Run, without token. While the run is open trace and evaluation are null. Reading also ends a run that is due (idle or timed out). Errors: 401, 404 not_found. Recorded while the run was running, after the five calls (the answer cuts tools and task.text):

curl -s $TESTWORLD_URL/v1/runs/$RUN_ID \
  -H "Authorization: Bearer $TESTWORLD_KEY"

200 OK
{
  "id": "run_5296fdfe-b495-4c5b-8e30-8f767d4c47dc",
  "status": "running",
  "ended_by": null,
  "error": null,
  "environment": {
    "id": "env_186c5910-e7d2-4c3c-bb5c-29da5672fbbe",
    "slug": "offsite-planning",
    "version": {
      "id": "ver_c8ab753a-2102-403f-9a20-502f3528935c",
      "number": 1
    }
  },
  "task": {
    "id": "task_schedule_q4_offsite",
    "text": ...
  },
  "agent": {
    "name": "ops-assistant",
    "revision": "1"
  },
  "tools_url": "$TESTWORLD_URL/v1/runs/run_5296fdfe-b495-4c5b-8e30-8f767d4c47dc/tools",
  "mcp_url": "$TESTWORLD_URL/v1/runs/run_5296fdfe-b495-4c5b-8e30-8f767d4c47dc/tools/mcp",
  "tools": ...,
  "now": "2026-10-26T08:00:00.005Z",
  "calls": 5,
  "max_calls": 500,
  "idle_timeout_seconds": 600,
  "timeout_seconds": 3600,
  "created_at": "2026-11-11T21:10:27.220Z",
  "started_at": "2026-11-11T21:10:38.505Z",
  "expires_at": "2026-11-11T22:10:27.220Z",
  "finished_at": null,
  "scored_at": null,
  "metadata": {},
  "trace": null,
  "evaluation": null
}

Once the run is done the same call returns the answer of the quickstart's step 4.

POST /v1/runs/{id}/finish

Ends a run on purpose. Allowed while the run is created or running. Answer 200: the Run, status evaluating. Read the run until done. Errors: 400, 401, 404, 409 run_ended (a second finish, unless it repeats the same idempotency_key, which replays the first answer). The quickstart's step 3 is the recorded example.

field	type	description
`output`	string	The agent's final answer, at most 64 KB, default `""`. Read by the `output_contains` and `output_equals` checks and shown to the rubric judge.
`idempotency_key`	string	1 to 255 characters.

GET /v1/runs/{id}/tools

The tool addresses and the token again. Answer 200: run_id, tools_url, mcp_url, token, expires_at and tools. Errors: 401, 404, 409 run_ended once the run has ended. Recorded on the open run (the answer cuts tools):

curl -s $TESTWORLD_URL/v1/runs/$RUN_ID/tools \
  -H "Authorization: Bearer $TESTWORLD_KEY"

200 OK
{
  "run_id": "run_5296fdfe-b495-4c5b-8e30-8f767d4c47dc",
  "tools_url": "$TESTWORLD_URL/v1/runs/run_5296fdfe-b495-4c5b-8e30-8f767d4c47dc/tools",
  "mcp_url": "$TESTWORLD_URL/v1/runs/run_5296fdfe-b495-4c5b-8e30-8f767d4c47dc/tools/mcp",
  "token": "tw_run_6cdb03085eae616dbb91402d469952ef",
  "expires_at": "2026-11-11T22:10:27.220Z",
  "tools": ...
}

The same call after the run ended:

curl -s -i $TESTWORLD_URL/v1/runs/$RUN_ID/tools \
  -H "Authorization: Bearer $TESTWORLD_KEY"

409 Conflict
{
  "error": {
    "type": "run_ended",
    "message": "The run has ended; its tools no longer answer."
  }
}

The tools under tools_url

These are not routes of the control API. They are the simulated tools, on the real APIs' shape, and where the agent works.

Addresses

The pattern is {tools_url}/{real host}/{real path}. For offsite-planning:

{tools_url}/gmail.googleapis.com/gmail/v1/...: Gmail, as the Gmail REST API.
{tools_url}/slack.com/api/...: Slack, as the Slack Web API, for example chat.postMessage.
{tools_url}/www.googleapis.com/calendar/v3/...: Calendar, as the Calendar REST API. Calendar ids are emails or primary.
{tools_url}/gmailmcp.googleapis.com/mcp/v1 and {tools_url}/mcp.slack.com/mcp: the two per-provider MCP servers.

mcp_url ({tools_url}/mcp) is the one MCP address for an MCP client: it serves the Gmail and Slack tools together. Calendar is REST only. MCP tool names are the per-provider names the server lists, such as gmail.search_messages, gmail.get_message, slack.search and slack.read_channel.

The run token

Authenticate every tool call with the run token, Authorization: Bearer tw_run_..., or for Slack the token form field. It is valid for this run until expires_at. The control API's key does not work here, and the run token does not work on the routes above.

How calls are counted

calls counts every request a tool answered, reads and writes alike. A call the tool answered with its own error (for example Gmail's 404 for an unknown message) counts. A request refused before it reached a tool (401, 409 run_ended) does not. On MCP only tools/call counts. Each counted call is one entry of the trace.

Reactions

An environment can carry simulated colleagues who answer. In offsite-planning, a post in #leadership is answered by Erik and Mari, in the thread of the post. The replies appear in the call's reactions, in the state_diff and in Slack's own thread reads. They are not calls: calls does not count them.

How a run ends

A run ends one of four ways, recorded in ended_by:

finish: you called POST /v1/runs/{id}/finish.
idle_timeout: no call for idle_timeout_seconds, counted from created_at or the last call, whichever is later.
timeout: timeout_seconds have passed since created_at.
max_calls: calls reached max_calls. The call that reached it is answered, then the run ends.

A sweeper ends idle and timed-out runs every few seconds, and a read of the run (or a tool call) also ends a run that is due. Every ending is scored, including a run that never made a call (it simply fails its checks).

After the end

Every tool call on an ended run is refused with 409 and is not counted. Gmail and Calendar answer the error envelope; Slack answers its own shape. Recorded after the quickstart run ended:

curl -s -i "$TOOLS_URL/gmail.googleapis.com/gmail/v1/users/me/messages?q=offsite" \
  -H "Authorization: Bearer $RUN_TOKEN"

409 Conflict
{
  "error": {
    "type": "run_ended",
    "message": "Run run_5296fdfe-b495-4c5b-8e30-8f767d4c47dc ended (finish) at 2026-11-11T21:10:38.666Z; its tools no longer answer."
  }
}

curl -s -i -X POST $TOOLS_URL/slack.com/api/chat.postMessage \
  -H "Authorization: Bearer $RUN_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"channel":"C_GENERAL","text":"late"}'

409 Conflict
{
  "ok": false,
  "error": "run_ended"
}

Errors.

Every error of the control API has the same envelope:

{"error": {"type": "invalid_request", "message": "unknown field: foo"}}

type is stable and for programs; message is for people and can change.

status	type	when
`400`	invalid_request	The body is not valid JSON, has a field this page does not list, or a value outside its range; a check is invalid; `task` is an unknown `task_…` id.
`401`	unauthorized	No key, or a wrong one.
`404`	not_found	No such environment, run or route.
`409`	run_ended	The run has ended: `finish` a second time, `/tools`, or a tool call.
`409`	idempotency_conflict	The same `idempotency_key` with a different body.
`429`	rate_limited	Too many failed authentications for one key (ten within a minute). The answer carries `retry-after`, in seconds. Today it is sent only after repeated wrong keys.

The tools' own errors

The tools answer in the shape of the API they copy. Slack answers {"ok": false, "error": "..."} with the error name, for example invalid_auth for a wrong run token or run_ended after the end. Gmail and Calendar answer Google's {"error": {"code", "message", "status"}}. Only a request refused before it reaches a tool (401, 409 run_ended) gets the control API's envelope on Gmail and Calendar; see the recorded answers under After the end.

Recorded answers

No key:

curl -s -i -X POST $TESTWORLD_URL/v1/runs \
  -H "Content-Type: application/json" \
  -d '{"environment":"offsite-planning","task":"x"}'

401 Unauthorized
{
  "error": {
    "type": "unauthorized",
    "message": "A valid API key is required."
  }
}

A field the API does not know:

curl -s -i -X POST $TESTWORLD_URL/v1/runs \
  -H "Authorization: Bearer $TESTWORLD_KEY" \
  -H "Content-Type: application/json" \
  -d '{"environment":"offsite-planning","task":"x","foo":1}'

400 Bad Request
{
  "error": {
    "type": "invalid_request",
    "message": "unknown field: foo"
  }
}

An environment that does not exist:

curl -s -i $TESTWORLD_URL/v1/environments/nope \
  -H "Authorization: Bearer $TESTWORLD_KEY"

404 Not Found
{
  "error": {
    "type": "not_found",
    "message": "no environment \"nope\""
  }
}

A task id that does not exist:

curl -s -i -X POST $TESTWORLD_URL/v1/runs \
  -H "Authorization: Bearer $TESTWORLD_KEY" \
  -H "Content-Type: application/json" \
  -d '{"environment":"offsite-planning","task":"task_nope"}'

400 Bad Request
{
  "error": {
    "type": "invalid_request",
    "message": "unknown task id: task_nope"
  }
}

The key of step 1 with another body:

curl -s -i -X POST $TESTWORLD_URL/v1/runs \
  -H "Authorization: Bearer $TESTWORLD_KEY" \
  -H "Content-Type: application/json" \
  -d '{"environment":"offsite-planning","task":"Something else","idempotency_key":"docs-quickstart-1"}'

409 Conflict
{
  "error": {
    "type": "idempotency_conflict",
    "message": "this idempotency_key was used with a different request body"
  }
}

Changelog.

2026-11-12 · The public API

Six routes under /v1/: list and read environments, create, read and finish a run, and read a run's tool addresses.
One error envelope, {"error": {"type", "message"}}, with six types.
A tool gateway under each run's tools_url, on the real Gmail, Slack and Calendar shapes, with one MCP address per run.
Four ways a run ends: finish, idle_timeout, timeout and max_calls.
Scoring on fixed checks plus at most one model-judged rubric.
The first environment, offsite-planning: Gmail, Slack and Calendar, one task with five checks, and colleagues who answer in Slack.