Skip to content
RehearsalDocs

Guides

One task on each page, in the order you usually do them, from adding an application to comparing two versions of an agent.

Each guide does one task and shows it in the console, from a terminal, in Python and through an AI assistant. The guides follow the order of the work.

Applications and worlds

Add an applicationWhat your application must have, and how to pin its source and images.Build a practice worldThe nine phases, how long they take, and what to do when a build fails.

Agents

Evaluate an agentChoose jobs, splits and repeats, and know the episode count first.Read resultsWhat each outcome, ending and number means, and how to read a failed check.Improve an agentTurn failures into a better version, and check it on jobs it never saw.Compare agents and runsWhich runs can be compared, and why.

Your own agent

Bring your own agentTest agent code that you run, in any framework, over the API.

Automate

Use Rehearsal in CIFail a pull request when the held-out score drops or a rule is broken.

The usual path

  1. Add an application, once for each application.
  2. Build a practice world, once for each release of the application that you want to test against.
  3. Evaluate an agent on the dev jobs, and read the results.
  4. Improve the agent, by hand or with the improvement step.
  5. Evaluate the old and the new version on the holdout jobs, and compare them.
Checked against rehearsal-kit 0.1.2 on 11 October 2026.

Was this page helpful?

Edit this page

What costs money

Four actions spend model budget. This page says which, roughly how much, how to set a limit, and what happens when you reach it.

Add an application

Tell Rehearsal where your application's source and container images are, pinned to exact versions, so a world can be built from it.

On this page

Applications and worldsAgentsYour own agentAutomateThe usual path

Was this page helpful?

Edit this page