DeviAI
Agent Ops

Every agent shipswith the dashboard that watches it.

We promise monitoring dashboards with every agent we build. Here is one — running on the same real recorded runs the Playground replays. Every number is computed from the transcript, never entered by hand.

01The console
5
Recorded runs

live, replayable

329.1s
Agent runtime

wall-clock, all runs

15
Tool calls

9 distinct tools

22
Reasoning steps

plan · read · decide

2
Retries recovered

self-corrected, no crash

1
Honest refusals

declined vs hallucinated

Runtime by agent

Wall-clock per recorded run — how long the loop actually took.

SQL Analyst27.0s
Document Intelligence32.6s
Document Intelligence · refusal13.3s
Research & Recon97.3s
Recruitment Screening158.9s
Outcomes

What each run actually produced — read from its final payload.

  • SQL AnalystAnswered

    Answered with real figures and the SQL it ran.

  • Document IntelligenceAnswered · cited

    Answered from the corpus with 2 citations.

  • Document Intelligence · refusalRefused · honest

    Refused — answer not in the corpus, so it declined rather than invent one.

  • Research & ReconSourced brief

    Wrote a brief citing 2 sources it actually read.

  • Recruitment ScreeningShortlisted

    Ranked a 5-person shortlist, with deal-breaker rejections explained.

Reasoning trace

Every event of one real run on its true time axis. The gaps are the model thinking between actions — tools run in milliseconds; reasoning is where the wall-clock goes.

task · Which product category earns the most revenue from paid orders, and who are our top 3 customers by spend?

ReasoningTool callAnswer27.0s total
reasonact
0.0s6.8s13.5s20.3s27.0s

Hover or focus a mark to inspect the event — 8 in this run.

02Where these numbers come from

01 — One source of truth

The same real runs

Runtime, tool calls, reasoning steps, retries and outcomes are read straight off the recorded transcripts the Playground replays — derived in code, not typed into a slide.

02 — No invented metrics

What isn’t measured isn’t shown

These agents run through the Claude CLI with no API key and no per-call billing, so no token or cost figure exists — so we show none. A dashboard that invents a “$/run” is lying to you.

03 — Refusal is a feature

The honest-refusal count

One run declined to answer because the corpus didn’t hold it. That is tracked as a win, not an error — an agent that refuses cleanly is worth more than one that confidently makes things up.