Evals: check the steps that matter on every change

A prompt tweak or a model swap can change what a step decides without anything failing. Evals catch it: each one replays a single step in a world you describe and checks the result.

What's new

• Evals. An eval is a short YAML document: the step to replay, the outputs of the steps before it, what its tools answer, and the checks. Check exact fields, phrases, patterns, a rubric a model grades against, or which tools an agent called. Every workflow has an Evals page.
• No side effects. A tool that posts, sends, writes or deletes is never called from an eval. It gets the answer the eval gives it, and every call is recorded so a check can ask what the step tried to do.
• Make an eval from a run. Any finished step on a run's page has Make an eval, which starts from that run's real inputs and output.
• Automatic Evals. Turn it on for a workflow and its evals run each time it changes. The same pass proposes steps worth an eval and points out evals that no longer fit their step. Team includes 3 workflows, Scale 25 and Enterprise is unlimited; evals by hand run on every plan.
• Needs a decision. The assistant drafts evals from real runs and suggests fixes when evals fail, replaying each fix against them first. Nothing counts or is saved until a person accepts it, and every decision is recorded.
• Over MCP. create_eval, run_eval, list_eval_decisions and the rest work for your own agents too.

Read more in Evals.