Build agents that learn from their mistakes.
CtrlB finds what went wrong in production, proposes a prompt fix, and tests it against the runs that failed. Your team decides what ships.
Works with your OpenTelemetry traces. Data stays in your bucket.
Requested $420 back on a $180 order.
The prompt never capped the refund amount.- Self-improving agents
- Traces & costs
- Evals
- Guard
Traces in. Findings and tested fixes back to your team.
- EvaluateScore production runs
- InvestigateFind the cause
- Compare promptsTest fixes on saved runs
- GuardCheck tool calls
Prompt changes are reviewed by your team before deployment.
A failed run is a test case for the next prompt.
A refund agent tries to pay out more than an order is worth. The trace shows the error. CtrlB investigates why it happened, suggests the missing instruction, and gives you the evidence to test the fix.
- Scheduled investigations group failures by cause, with a suggested prompt change and links to the runs.
- Save those runs as a test set, then compare the old and new prompts against the same cases.
- Review the results before you ship the change.
- InvestigateRead failed runs
- FindName the cause
- FreezeKeep the traces
- CompareOld prompt vs new
- What happened
- issue_refund was called with an amount above the order total and failed with ValidationError.
- Why
- The prompt asks for the customer's requested amount and never says to cap it at the order total.
- Suggested fix
- Add to the system prompt: call lookup_order first and never refund more than its total.
Define what a good run looks like.
Write what a good answer looks like. A judge model scores new traces as they arrive, on your own provider key. Each score is stored next to its trace.
- Filter, chart and alert on scores like any other column.
- Every eval run shows what the judge cost.
Put a check between intent and action.
Check tool calls before they execute and redact sensitive data from their results. Guard runs in your framework's tool hook, so policies can act on the call itself.
- A policy answers allow, deny, ask a human, or rewrite.
- Rules cover tool allow lists, PII in arguments, runaway loops, and calls that match an open finding.
- Works with Claude Code hooks, OpenAI Agents SDK, LangChain, Google ADK and FastMCP. Every check is recorded alongside the trace.
Follow the run. Find the expensive mistake.
Open a run to see the prompt, model responses and tool calls in order. Tokens, latency and cost sit alongside each call. A prompt hash ties the result back to the version that produced it.
- Sort runs by cost and open the one that spent the budget.
- Find the slowest tool and the errors it returns.
| Span | Timeline | Cost |
|---|---|---|
| agentrefund-agent | $0.0214 | |
| chatclaude-sonnet | $0.0087 | |
| toollookup_order | ||
| chatclaude-sonnet | $0.0087 | |
| toolissue_refund | ||
| chatclaude-sonnet | $0.0040 |
Keep the instrumentation you already have.
Point your OTLP exporter at CtrlB. Spans that follow the OpenTelemetry GenAI conventions show up as agents, model calls and tool calls, with no mapping step. Langfuse and other OTel-based SDKs export the same spans.
- Keep complete runs, including the calls that led to a failure.
- Your coding agent reads the same traces through the CtrlB MCP server: search_traces, fetch_trace and run_sql.
The evidence stays in your bucket.
Traces, scores and findings live as open Parquet in your S3, GCS or Azure Blob bucket. Query them with SQL or give your coding agent access through the CtrlB MCP server. The files stay readable without CtrlB.
Start with your existing OTLP exporter. Bring your own model key for evals and investigations, with the model cost visible on each run.
Give your agents a way to get better.
One plan for tracing, evaluations, prompt investigations and Guard. Start with the agents you run today, then expand as your workload grows.
- Production traces and model cost tracking
- Scheduled investigations and prompt comparisons
- Tool policies with Guard
- OpenTelemetry, your own bucket and your own model key
AI Observability
From $499 / month
Platform fee. Usage is quoted separately for your workload.
Discuss your workloadWe agree trace volume, evaluation usage and Guard checks with you before you start. Model and bucket costs are billed directly by your providers.