AgentDebugX is local-first. Deterministic ingest and diagnosis can run without an external model. LLM-backed diagnosis, GUI root-cause analysis, and live reruns are enabled only when you configure the corresponding model or application-owned runner.

Start from your situation¶

The documented workflow¶

Installation¶

AgentDebugX supports Python 3.9 through 3.13. Install the base package with pip:

python -m pip install agentdebugx

Verify that the CLI and import are available:

agentdebug --help
python -c "import agentdebug; print(agentdebug.__version__)"

Optional features¶

The base install includes the portable schema, raw ingest, deterministic diagnosis, rerun planning, and the core GUI RCA package. Install extras only for the integrations you use.

Install from this repository¶

For local development:

git clone https://github.com/AgentDebugX/AgentDebugX.git
cd AgentDebugX
python -m pip install -e ".[ui]"

Run a quick health check:

agentdebug doctor
python -m pytest tests -q

The full GUI test matrix needs the three GUI-related extras:

python -m pip install -e ".[gui,gui-memory,gui-app]"
python -m pytest tests/gui -q

First diagnosis¶

In about five minutes, you will normalize the repository's sample trace, diagnose it without an API key, inspect the report, and prepare a rerun plan.

You will create: trace.json, report.json, and rerun-plan.json.

1. Install AgentDebugX¶

python -m pip install agentdebugx

If you are working from a clone of this repository, use an editable install instead:

python -m pip install -e .

2. Normalize the sample¶

From the repository root:

agentdebug ingest examples/sample_trace.json \
  --format auto \
  --out trace.json

trace.json is an AgentTrajectory: a framework-independent sequence of normalized events.

3. Run local diagnosis¶

agentdebug diagnose trace.json \
  --mode heuristic \
  --attributor heuristic \
  --recovery reflexion \
  --out report.json

This command runs the three Diagnose stages:

Detect → Attribute → Recover

• --mode heuristic selects deterministic rule-based detection.
• --attributor heuristic attaches local blame localization.
• --recovery reflexion adds a structured recovery proposal.
• --out report.json preserves the result for inspection or Rerun.

Likely root cause: Tool execution error in search at step 2.

4. Read the result¶

Open report.json and inspect these fields first:

The relevant part of this sample report looks like:

{
  "trace_id": "trace_sample",
  "root_cause_event_id": "evt_2",
  "root_cause_agent": "search",
  "root_cause_step_index": 2,
  "summary": "Likely root cause: Tool execution error in search at step 2."
}

You can also render a cascade-oriented terminal view:

agentdebug diagnose trace.json \
  --mode heuristic \
  --attributor heuristic \
  --recovery reflexion \
  --traceback

5. Prepare a rerun request¶

A normalized trajectory and diagnostic report do not contain a live agent environment. Start by producing a plan:

agentdebug rerun report.json \
  --trajectory trace.json \
  --plan-only \
  --out rerun-plan.json

The plan explains which runtime capabilities are available and which are still needed for real execution. Continue with Validate with Rerun when you have an application-owned runner.

How AgentDebugX works¶

AgentDebugX separates failure analysis from retry execution.

Your agent or exported trace
            │
            ▼
   AgentTrajectory schema
            │
            ▼
  Detect → Attribute → Recover
            │
            ▼
      DiagnosticReport
            │
            ▼
  plan / simulation / live rerun

Ingest: create a portable trajectory¶

Agent frameworks record different shapes: messages, callbacks, spans, tool events, screenshots, or benchmark JSONL. Ingest adapters convert these shapes into AgentTrajectory and AgentEvent objects.

Downstream diagnosis code therefore works with one contract instead of framework-specific objects.

Detect: identify visible failure signals¶

Detect produces FailureFinding objects from individual events or trajectory-level patterns. The deterministic analyzer loads manifest-backed rule packs; the CLI can also select LLM Judge, DeepDebug, or GUI RCA modes.

A finding records the failure mode, source event or step, evidence, optional confidence, and a suggested response.

Attribute: locate responsibility¶

The final error is not always the root cause. Attribute tests which earlier event or agent likely introduced the failure.

The CLI currently exposes heuristic, all-at-once, step-by-step, binary-search, and counterfactual attributors. DeepDebug owns its own multi-round attribution workflow.

Recover: package a proposed correction¶

Recover converts the diagnostic context into a structured fix proposal or retry directive. Available CLI strategies include DeepDebug, Reflexion, CRITIC, Self-Refine, AutoManual, and saga rollback.

Recovery remains suggest-only. It does not execute tools or mutate the target application.

Rerun: test the hypothesis¶

Rerun consumes a diagnostic report and, when available, the source trajectory. It can:

1. create an auditable plan,
2. export pending actor tasks,
3. generate an explicitly labeled simulation, or
4. dispatch to an application-owned live executor.

Only observed live execution can provide evidence about the real task outcome. A simulation executes no tools and is not proof that a fix worked.

Inspect and share¶

The optional local UI reads SQLite or JSONL trace stores. Error Hub bundles provide a scrubbed, portable unit for regression tests or opt-in sharing.

For detailed implementation boundaries, continue to Architecture.

DeepDebug: Unified Design (v1.0, 2026-07-08)¶

Single source of truth for what DeepDebug is, what is implemented today, what is missing, and the acceptance rules for changing it. Supersedes the scattered descriptions in module docstrings; the demo paper's Section 4 is the prose rendering of this document.

1. Positioning: the Deep Error Analysis stage of the Diagnostic Core¶

DeepDebug is the "Deep Error Analysis" box in the AgentDebugX Diagnostic Core (Figure 1, layer 2). It sits inside — not beside — the layer-2 flow:

Trace Capture → Normalized Schema → Error Taxonomy (+Auto Induction)
   → Detectors → LLM Judge → [ DEEP ERROR ANALYSIS ] → Who&When Attribution
   → Recovery Suggestions

Contract with its neighbors:

2. The harness contract¶

DeepDebug is one diagnostic agent harness: a multi-turn agent equipped with tools over (a) the captured trace and (b) the environment the trace came from, plus knowledge sources, with a bounded turn budget and a structured verdict.

Five clauses every channel must satisfy:

1. Multi-turn. The agent takes multiple investigative turns; a turn may invoke a tool or commit to a verdict. Turn cap enforced (text: 4 macro-turns
2. conditional arbitration; GUI: MAX_TURNS ReAct loop).
3. Tool-using. Tools are declared per channel from one registry (§4); the agent chooses among them (GUI channel) or the harness sequences them in the measured order (text channel, §3).
4. Env-diving. When the trace links to an on-disk / remote environment (OSWorld trajectory dir, GAIA attachments), the agent may open it — screenshots, files, task assets — not just re-read trace text.
5. Knowledge-augmented. Deep memory (retrieve + writeback), lessons, and (planned) Error Hub cases are injected as context, never as ground truth.
6. Auditable, typed verdict. Every turn is recorded; the final output is a DiagnosticReport whose first finding is the root cause with taxonomy label, quoted evidence, and one actionable fix suggestion.

Never re-executes the debugged agent's own tools. Env dive is read-only inspection; replay/counterfactual re-execution stays a separate, future attributor.

3. Text-trace channel: the measured macro-turn pipeline (default)¶

The default text channel runs a fixed, measured sequence (chosen by ablation — free-form exploration is not the default because the fixed order wins on Who&When; see §7):

Implementation: src/agentdebug/diagnose/profiles/deepdebug.py (DeepDebugAnalyzer) over src/agentdebug/diagnose/attribute/moe.py (aao_moe_attribute: all_at_once + _cascade/_bisect_refine + arbitration). Memory retrieval (step 0) feeds both readings and the final diagnosis turn. DeepDebugResult.rounds exposes the four table stages directly. AaoMoeAnalysis preserves both candidates, the cascade/bisection decisions and final window, and the adjudication verdict. Every candidate carries event_id + step_index + agent_name; a bare step is accepted only when unique. Final evidence is represented as an event-id quote, verified against trajectory input/output/error text before entering the report. Unverified model quotes are rejected and counted in report metadata.

DeepDebug lives under diagnose/profiles/ because it orchestrates a complete Diagnose workflow rather than implementing attribution alone. The former diagnose/attribute/deepdebug.py module remains a compatibility re-export. The registry ID attribute.deepdebug is also retained for existing plugin configuration, but its entrypoint resolves to the canonical profile module.

Design rationale (measured, not taste): replacing the structure-guided turn with a second global search costs −4.8 strict points (gpt-5.4-mini ablation); the dual design wins every Who&When metric on qwen3.5-9b and the strict metric on qwen3.6-27b; at the frontier a single reading suffices, so the extra turns are opt-in budget, not a tax (docs/benchmarks/).

4. Tool registry (unified across channels)¶

One registry, per-channel exposure. GUI names exist today in src/agentdebug/gui/tools/; text-channel "tools" are currently internal functions of moe.py — same capabilities, not yet declared as callable tools (G4 promotes them for the opt-in explore mode).

5. GUI / env channel (CUA · OSWorld)¶

GuiRcaAnalyzer (src/agentdebug/diagnose/gui_rca.py) satisfies the harness contract with a free ReAct loop (src/agentdebug/gui/agent.py: run_react_loop, turn-capped, old screenshots compressed to bound context):

• Ingest/env dive: IngestionResult.from_directory(osworld_root) — the agent inspects the actual trajectory directory (screenshots + step files), resolved from metadata['source_dir'] or screenshot artifact URIs.
• Tools: get_step_details (text + images), lesson tools, finish.
• Model routing: runtime/llm_channel.py presents an Anthropic-style .messages.create seam but executes through our OpenAICompatClient — one LLM stack for both channels, vision included.
• Verdict mapping: RCAResult → DiagnosticReport with the GUI taxonomy (runtime/gui_taxonomy.py); infeasible-task branch handled at ingestion.
• CLI: agentdebug diagnose --mode gui-rca --rule-pack gui (+ --format osworld ingest).

The main FastAPI dashboard also has a read-only CUA viewer. UI services derive a visual_capability from image artifacts that exist beneath the imported trajectory's metadata.source_dir; routes serve those files by trace/event/artifact identity and reject non-images, missing files, traversal, and symlink escapes. The shared trace page defaults to Visual when that capability is present, retains a per-trace Trace/Visual override in sessionStorage, and keeps timeline selection synchronized with screenshots and DiagnosticReport evidence. Streamlit remains a separate annotation and evaluation surface rather than a second runtime dashboard.

Visual comparison is event-centric: for selected event N, explicit before artifacts take precedence, otherwise N−1's result is the Before state; N's role-tagged images are the After state. Multiple images remain ordered by their artifact indices, and a missing adjacent state is shown as missing rather than substituted from an older event.

The dashboard discussion channel is format-neutral. It snapshots an AgentTrajectory and selected DiagnosticReport, exposes only bounded read-only event/report tools, and stores versioned sessions separately in .agentdebug/discussions.sqlite. Assistant citations use canonical event IDs. Structured report-revision proposals are drafts for explicit export, never implicit report mutations.

6. Knowledge: memory, lessons, Error Hub¶

7. Evaluation contract & current evidence¶

Acceptance rule (unchanged): any redesign must not regress the completed round on any reported cell; if it does, analyze, iterate, or revert — never ship a worse number for narrative reasons. Every design change lands with a measured before/after (the judge-seeding → dual-reading history is the precedent: both replacements were adopted only after beating the incumbent).

8. Gap plan (what to build, in order)¶

Order: G2 → G5 (cheap, knowledge-side) → G1 (facade) → G3 → G4 (needs eval budget). GAIA rerun does not block on G3/G4.

9. What DeepDebug is not¶

• Not a judge replacement: the one-call judge stays the cheap default; DeepDebug is the escalation tier (~6 calls) for when strict localization matters.
• Not a replayer: no re-execution of the debugged agent's tools.
• Not a committee of agents: one agent, multiple turns, complementary readings.

Ingest traces¶

Use Ingest when the run was produced outside AgentDebugX or is still in a framework-specific export format.

Convert one file¶

Auto-detect a JSON or JSONL export:

agentdebug ingest raw_trace.json \
  --format auto \
  --out trace.json

When detection is ambiguous, select a format explicitly:

agentdebug ingest messages.json \
  --format messages \
  --task-id checkout-42 \
  --goal "Complete checkout" \
  --framework my-agent \
  --out trace.json

Supported CLI format names¶

The current CLI accepts:

agenttrajectory
messages
message_list
conversations
event_list
webshop_pages
openai_agents_spans
crewai_events
langgraph_callbacks
openclaw
claude_code
hermes
osworld

auto asks the importer to infer a supported format from the payload.

Framework integrations may require the corresponding optional extra. See Installation.

Process a JSONL collection¶

Each non-empty line is treated as an independent record:

agentdebug batch ingest dataset.jsonl \
  --format auto \
  --out-dir normalized

Batch diagnosis performs normalization and diagnosis in one command:

agentdebug batch diagnose dataset.jsonl \
  --format auto \
  --mode heuristic \
  --attributor heuristic \
  --recovery reflexion \
  --out-dir runs

Every batch writes batch-summary.json. Invalid inputs are isolated from successful records. A partially failed CLI batch exits with status 3.

Import an OSWorld trajectory directory¶

OSWorld input is a directory containing its trajectory JSONL, result metadata, and screenshots:

agentdebug ingest path/to/osworld-task \
  --format osworld \
  --out osworld-trace.json

The adapter records the resolved source directory in trajectory metadata and attaches screenshot paths as image artifacts. GUI RCA uses that on-disk source directory to inspect the original evidence.

Validate the normalized output¶

A converted trajectory should contain:

• a stable trace_id,
• optional task, goal, and framework metadata,
• an ordered events list,
• event types from the canonical enum,
• source-specific details under metadata, and
• artifact URI references for files or screenshots.

See the Trace schema for the full contract.

Diagnose failures¶

Diagnose runs three ordered stages over one normalized trajectory:

Detect → Attribute → Recover

Start with the local baseline¶

The deterministic path is the fastest way to validate a trace and requires no API key:

agentdebug diagnose trace.json \
  --mode heuristic \
  --attributor heuristic \
  --recovery reflexion \
  --out report.json

The heuristic detector loads rule packs. By default, auto selects core rules and any benchmark-specific pack inferred from trajectory metadata. Override it with one or more --rule-pack options:

agentdebug diagnose trace.json \
  --mode heuristic \
  --rule-pack core \
  --rule-pack gui \
  --out report.json

Configure an OpenAI-compatible model endpoint¶

Save the endpoint locally:

agentdebug config set-llm \
  --base-url "https://your-host.example/v1" \
  --api-key "your-key" \
  --model "your-model"

Inspect the masked configuration and test connectivity:

agentdebug config show
agentdebug config doctor

Environment variables are also supported:

export AGENTDEBUG_LLM_BASE_URL="https://your-host.example/v1"
export AGENTDEBUG_LLM_API_KEY="your-key"
export AGENTDEBUG_LLM_MODEL="your-model"

Choose a diagnosis mode¶

The CLI also accepts compatibility aliases shown by agentdebug diagnose --help.

Choose attribution and recovery explicitly¶

Regular diagnosis modes let you combine stages:

agentdebug diagnose trace.json \
  --mode judge \
  --attributor all-at-once \
  --recovery self-refine \
  --out report.json

Attributors exposed by the CLI include:

• heuristic
• all-at-once
• step-by-step
• binary-search
• counterfactual
• none

Recovery strategies include:

• deepdebug
• reflexion
• critic
• self-refine
• auto-manual
• saga-rollback
• none

DeepDebug is a full profile rather than a fourth Diagnose stage:

agentdebug diagnose trace.json --mode deepdebug --out report.json

It runs deterministic Detect first, treats those findings as fallible prior signals, performs its multi-round attribution, and packages its final correction as a retry directive. Explicit --recovery none disables the standard recovery payload.

Render a traceback view¶

For terminal inspection:

agentdebug diagnose trace.json \
  --mode heuristic \
  --attributor heuristic \
  --traceback

Add --no-color for logs or environments that should not receive ANSI color.

Validate with Rerun¶

Rerun tests a recovery proposal after Diagnose. It keeps planning, simulation, and real execution separate.

Mode 1: build a plan¶

This is the safe default for a trajectory-only workflow:

agentdebug rerun report.json \
  --trajectory trace.json \
  --plan-only \
  --out rerun-plan.json

The plan includes the retry directive, checkpoint policy, approval metadata, and a capability assessment. A trace alone is normally missing the framework runner, tool runtime, and environment state needed for real execution.

Export the request for a separate actor pipeline:

agentdebug rerun report.json \
  --trajectory trace.json \
  --plan-only \
  --actor-task-format jsonl \
  --out rerun-tasks.jsonl

These rows contain pending actor inputs and provenance. They are not responses, verified outcomes, or training labels. Parquet output additionally requires pyarrow.

Mode 2: generate a labeled simulation¶

agentdebug rerun report.json \
  --trajectory trace.json \
  --simulate \
  --out rerun.simulated.json

Simulation asks the configured LLM to produce a hypothetical continuation. It executes no tools. Its output is marked simulated, and any evaluation is scoped to the simulated trajectory only.

Mode 3: use a persistent live runner¶

The target application must provide a callback that owns its real model, tools, credentials, environment, and trajectory recorder. Start a runner service:

agentdebug runner serve my_project.runner:run_agent \
  --name my-agent \
  --framework langgraph \
  --host 0.0.0.0 \
  --port 8765 \
  --token-env MY_RUNNER_TOKEN

Save and verify the runner:

agentdebug config set-runner my-agent \
  --url http://127.0.0.1:8765 \
  --token-env MY_RUNNER_TOKEN \
  --default

agentdebug config doctor-runner my-agent

Then dispatch the rerun:

agentdebug rerun report.json \
  --trajectory trace.json \
  --out rerun.live.json

Select a named runner with --runner NAME when it is not the configured default.

Branch from an event¶

--start-event uses a 1-based event position and resolves it to the event's stable ID:

agentdebug rerun report.json \
  --trajectory trace.json \
  --start-event 4 \
  --out rerun.from-event.json

The selected runner must advertise support for restoring or continuing from event checkpoints.

Process compatibility transport¶

Local scripts and CI can use an application-owned command:

agentdebug rerun report.json \
  --trajectory trace.json \
  --runner-command "python path/to/project_rerun_runner.py" \
  --out rerun.json

The repository includes callback examples in examples/http_agent_runner.py and examples/live_rerun_runner.py.

Execution proof¶

AgentDebugX accepts a live result only when the executor declares live execution and confirms that the returned trajectory was observed. Rerun then compares the source and returned branches with its local proxy evaluator.

GUI / OSWorld root-cause analysis¶

The GUI RCA path analyzes computer-use trajectories containing actions, screenshots, reward, completion state, and execution errors.

What the analyzer does¶

For a failed trajectory, the RCA agent starts at the terminal failure step F and walks backwards:

terminal failure F
      ↓ inspect
step F, F-1, F-2, ...
      ↓
earliest step that introduced a new causal mistake
      ↓
root_error_step N

For each inspected step, the RCA tools expose textual action details plus input and result screenshots. The final result contains:

• one root_error_step,
• a GUI taxonomy tag,
• grounded evidence,
• a proposed correction,
• model-reported confidence,
• concise summaries for inspected steps, and
• the retained thinking trace.

Infeasible OSWorld tasks use a separate prompt branch that checks whether the agent recognized that the task could not be completed.

Install the needed layer¶

The GUI RCA package ships with the core install. Add Pillow when screenshots need decoding:

python -m pip install "agentdebugx[gui]"

The heavier lesson-memory and batch application are separate:

python -m pip install "agentdebugx[gui,gui-memory,gui-app]"

Normalize the OSWorld directory¶

agentdebug ingest path/to/osworld-task \
  --format osworld \
  --out osworld-trace.json

Keep the source directory and screenshots in place. The normalized trajectory stores a resolved metadata.source_dir and screenshot artifact URI references so the RCA tools can reopen the evidence.

Run standard GUI RCA¶

Configure an OpenAI-compatible backend that supports both tool calling and vision, then run:

agentdebug diagnose osworld-trace.json \
  --mode gui-rca \
  --model "your-model" \
  --out gui-report.json

The standard analyzer maps the GUI-specific RCAResult into the common DiagnosticReport format. The primary finding preserves the taxonomy tag, evidence, correction, step index, event ID when found, and inspected-step summaries.

GUI batch and annotation application¶

The older batch application is available through:

python -m agentdebug.gui --help

It requires the GUI memory and application extras. Its configuration, result layout, annotation UI, and accuracy commands are documented in the GUI command reference.

Local inspection UI¶

The optional FastAPI application provides a local browser surface for stored traces and diagnostic reports.

Install and launch¶

python -m pip install "agentdebugx[ui]"

Use a SQLite store:

agentdebug serve \
  --store-sqlite .agentdebug/traces.sqlite \
  --host 127.0.0.1 \
  --port 7777

Or a JSONL store:

agentdebug serve \
  --store-jsonl .agentdebug/traces.jsonl \
  --host 127.0.0.1 \
  --port 7777

Open http://127.0.0.1:7777.

Import native JSON files¶

Place normalized trajectory and diagnostic-report JSON files under .agentdebug/imports/, then select Sync imports in the workspace. To use a different server-owned directory, set AGENTDEBUG_IMPORT_DIR before starting the server.

Inspect GUI evidence¶

OSWorld trajectories with locally available screenshot artifacts open in the read-only Visual view. The Trace / Visual control switches representations without changing the selected event.

Visual compares:

• the selected action's explicit input image, or the preceding event result, and
• all result images attached to the selected event.

Screenshot files are served only through trace and event artifact IDs, and only when the resolved path remains inside the trajectory's recorded source directory.

Discuss a report¶

Discuss with Debugger works with every normalized trace format. Discussions are local, reference canonical event IDs, and remain pinned to a report snapshot. They may create an exportable report-revision draft but never overwrite the stored diagnostic report.

Prepare reruns¶

Rerun Composer opens from the selected event and uses it as the checkpoint. Configure runner details on the server:

export AGENTDEBUG_RUNNER_URL="http://127.0.0.1:8765"

The process compatibility fallback is:

export AGENTDEBUG_RERUN_COMMAND="python path/to/project_rerun_runner.py"

The browser does not accept or persist runner commands or bearer tokens.

CLI reference¶

The installed command is agentdebug.

agentdebug --help
agentdebug <command> --help

Primary commands¶

Compatibility commands remain available:

• agentdebug analyze is the heuristic-compatible diagnosis entry point.
• agentdebug convert aliases agentdebug ingest.
• agentdebug act contains compatibility namespaces for advanced actions.

ingest¶

agentdebug ingest INPUT
  [--out PATH]
  [--format FORMAT]
  [--trace-id ID]
  [--task-id ID]
  [--goal TEXT]
  [--framework NAME]

Use Ingest traces for supported format names and examples.

diagnose¶

agentdebug diagnose TRAJECTORY
  [--mode MODE]
  [--attributor [ATTRIBUTOR]]
  [--recovery RECOVERY]
  [--model MODEL]
  [--base-url URL]
  [--api-key KEY]
  [--embedding-model MODEL]
  [--embedding MODEL]
  [--rule-pack PACK]
  [--out PATH]
  [--traceback]
  [--no-color]

--store-sqlite and --store-jsonl let TRAJECTORY refer to a stored trace ID. The two store options are mutually exclusive.

Use Diagnose failures for mode and component semantics.

rerun¶

agentdebug rerun DIAGNOSTIC_REPORT
  [--trajectory TRAJECTORY]
  [--start-event N]
  [--runner NAME]
  [--runner-command COMMAND]
  [--runner-cwd PATH]
  [--runner-timeout SECONDS]
  [--simulate]
  [--plan-only]
  [--actor-task-format jsonl|parquet]
  [--out PATH]

--start-event is 1-based. Planning, simulation, and live execution are intentionally distinct. See Validate with Rerun.

serve¶

Exactly one store is required:

agentdebug serve
  (--store-sqlite PATH | --store-jsonl PATH)
  [--host HOST]
  [--port PORT]

The UI dependencies come from agentdebugx[ui].

Configuration safety¶

Prefer saved configuration or environment variables over repeating API keys on the command line. agentdebug config show masks stored secrets. Use agentdebug config --help for the current configuration subcommands.

Python API¶

The distribution is named agentdebugx; the Python package is agentdebug.

Record and diagnose a run¶

AgentDebug is the high-level embedded entry point:

from agentdebug import AgentDebug, EventType

debugger = AgentDebug()

with debugger.trace(
    goal="Book a refundable NYC to SFO flight",
    framework="my-agent",
) as trace:
    trace.record(
        EventType.PLAN,
        agent_name="planner",
        output="Search for the cheapest fares.",
    )
    trace.record(
        EventType.TOOL_RESULT,
        agent_name="browser",
        error="Checkout failed: refund_policy is required.",
    )

    report = trace.analyze()

print(report.summary)

When step_index is omitted, TraceSession.record() assigns increasing step numbers. Explicit values are preserved.

The context manager records a successful run end when the block exits normally. If an exception leaves the block, it records a failed terminal event and does not suppress the exception.

Construct a portable trajectory¶

Use schema models when converting a custom framework:

from agentdebug import AgentEvent, AgentTrajectory, EventType

trajectory = AgentTrajectory(
    task_id="task-42",
    goal="Complete checkout",
    framework="my-framework",
)

trajectory.add_event(
    AgentEvent(
        trace_id=trajectory.trace_id,
        agent_name="browser",
        event_type=EventType.TOOL_CALL,
        step_index=1,
        input={"tool": "checkout", "refund_policy": None},
    )
)

See the Trace schema before adding framework-specific metadata.

Run the local Diagnose pipeline¶

from agentdebug import DiagnosePipeline

pipeline = DiagnosePipeline.local_default()
result = pipeline.run(trajectory)

print(result.report.summary)
print(result.attribution)

DiagnosePipeline accepts custom detector, attributor, and recoverer implementations. Passing None for an attributor or recoverer disables that sub-stage.

Build a rerun plan¶

from agentdebug.rerun import RerunWorkflow

workflow = RerunWorkflow.suggest_only()
rerun_result = workflow.run(
    report=result.report,
    trajectory=trajectory,
    execute=False,
)

print(rerun_result.plan.capability.reason)

Execution requires a configured RerunExecutor and an explicit execute=True. Simulation executors are rejected unless the workflow was created with simulation explicitly allowed.

Serialize models¶

Use AgentDebugX helpers for Pydantic 1 and 2 compatibility:

from agentdebug.schema import model_to_json, trajectory_from_json

payload = model_to_json(trajectory, indent=2)
restored = trajectory_from_json(payload)

For detailed fields, read Trace schema and Diagnostic report.

Diagnostic report¶

Every analyzer returns the same top-level DiagnosticReport model, even when the internal analysis method differs.

Top-level fields¶

Failure findings¶

Each FailureFinding contains:

• a FailureMode,
• optional event, agent, and step localization,
• optional confidence,
• a list of evidence strings,
• an optional suggestion, and
• metadata describing how and why the finding was produced.

The deterministic analyzer records its rule pack, rule ID, trigger scope, and confidence basis under finding metadata.

Attribution payload¶

When an attributor is enabled, the pipeline stores:

{
  "method": "...",
  "elapsed_ms": 0,
  "hypotheses": [],
  "primary": null,
  "raw": null
}

primary is the first ranked hypothesis when any hypotheses exist.

Recovery payload¶

When a recoverer is enabled:

{
  "proposal_count": 1,
  "proposals": []
}

The top-level suggestions list is updated from the proposal text.

GUI RCA metadata¶

The standard GUI RCA analyzer adds:

• analyzer: "gui_rca",
• source: "gui_rca",
• the model name,
• per_step_summaries, and
• thinking_trace.

Each inspected-step summary contains step_num, intent_summary, outcome_summary, and summary_source.

Confidence behavior¶

Confidence is optional in the schema. Public serialization omits uncalibrated confidence from reports produced by the deterministic HeuristicAnalyzer and the DeepDebugAnalyzer. Other analyzers may retain their own confidence value and provenance.

Use reports as hypotheses¶

A diagnostic report is evidence-bearing analysis, not automatic ground truth. Preserve the source trajectory and report provenance when reviewing, sharing, or using a report to prepare reruns.

Canonical Trace Schema¶

This document describes the schema implemented by the current AgentDebugX codebase. It is a runtime contract, not a roadmap.

The source of truth is src/agentdebug/schema/models.py. Public imports live under agentdebug.schema; legacy imports under agentdebug.core remain available as compatibility shims.

1. Scope¶

AgentDebugX uses one framework-independent intermediate representation for an agent execution:

AgentTrajectory
└── events: list[AgentEvent]
    └── artifacts: list[Artifact]

Diagnose produces a separate report linked by trace_id:

DiagnosticReport
├── findings: list[FailureFinding]
├── attribution: dict | null
├── recovery: dict | null
└── audit: list[DiagnosticAuditEntry]

The repository does not currently define Run, Session, Trace, or Span Pydantic models. OpenTelemetry support is an optional export adapter that maps an AgentTrajectory to spans; the canonical stored representation remains the trajectory/event model documented here.

2. AgentTrajectory¶

AgentTrajectory represents one recorded agent run.

add_event(event) appends an event. prefix(n) returns a trajectory copy with only the first n events and is used by attribution algorithms.

3. AgentEvent¶

AgentEvent is the smallest normalized execution record.

Event identity should use event_id. A step_index may repeat across agents or across thought, action, and observation events. Attribution and rerun code therefore resolve events with event_id first and use step/agent information as supporting context.

EventType values¶

run.start       run.end          agent.step      llm.call
llm.response    tool.call        tool.result     memory.read
memory.write    reflection       plan            handoff
guardrail       observation      error           human.feedback

4. Artifact¶

Artifact links non-inline evidence to an event.

AgentDebugX does not embed or copy artifact bytes as part of this model.

5. DiagnosticReport¶

DiagnosticReport is the standard output of Diagnose.

DiagnoseContext is the in-process contract between Detect, Attribute, and Recover. It preserves detector findings while promoting the primary attribution to the recovery target. It is not serialized as a top-level schema object; bounded context needed by later stages is placed in report or rerun metadata.

6. Failure Models¶

FailureMode¶

FailureMode describes a taxonomy node:

• mode_id
• name
• family
• description
• signals
• suggestion_templates
• source

The built-in taxonomy is exposed as SEED_FAILURE_MODES.

FailureFinding¶

FailureFinding links one failure mode to a trajectory location:

• generated finding_id
• failure_mode
• optional event_id, agent_name, and step_index
• optional confidence
• evidence strings
• optional recovery suggestion
• metadata

Confidence is retained internally for ranking and compatibility. Public serialization removes confidence recursively for Heuristic and DeepDebug reports; LLM Judge reports retain the model-reported value.

7. Diagnostic Audit¶

DiagnosticAuditEntry records one stage used to produce a report:

DeepDebug writes four ordered entries:

1. global_read
2. structure_probe
3. cross_examine
4. diagnose_and_suggest

The CLI serializes DiagnosticReport.audit directly. DeepDebug reports therefore retain candidate localization, cascade or bisection decisions, cross-examination verdicts, final evidence, and per-stage duration. The older metadata.deepdebug_stages list remains as a compact compatibility summary.

8. Serialization¶

Use the public helpers instead of calling Pydantic methods directly:

from agentdebug.schema import (
    model_to_dict,
    model_to_json,
    report_from_json,
    trajectory_from_json,
)

• model_to_json(model, indent=None) supports Pydantic v1 and v2.
• model_to_dict(model) applies report-specific public-output filtering.
• trajectory_from_json(payload) parses an AgentTrajectory.
• report_from_json(payload) parses a DiagnosticReport.

Datetimes are serialized as ISO 8601 strings. Enums are serialized by value. Unknown producer formats should go through agentdebug ingest or a registered adapter instead of being passed directly to trajectory_from_json.

9. Storage¶

Two local storage backends implement TraceStore:

JsonlTraceStore¶

• Append-only JSONL.
• One complete serialized AgentTrajectory per line.
• Loading a duplicated trace_id returns the last matching entry.
• Also recognizes supported AgentErrorBench JSONL rows during reads.

SQLiteTraceStore¶

• Stores complete trajectory JSON in trajectories.payload_json.
• Stores complete report JSON in diagnostic_reports.payload_json.
• Index columns contain identifiers and timestamps used for lookup and ordering.
• It is an embedded local store, not a distributed database contract.

DuckDB, Parquet, msgpack, and native OTLP persistence are not implemented storage formats in the current repository.

10. OpenTelemetry¶

agentdebug.ingest.adapters.otel.export_trajectory() is an optional best-effort exporter. It creates one root span and one child span per event with selected gen_ai.* and agentdebug.* attributes.

This exporter does not make the AgentDebugX JSON schema wire-compatible with OTLP, and it does not define native Span models. When OpenTelemetry packages are absent, the adapter reports itself unavailable and emits nothing.

11. Error Hub Bundles and Privacy¶

Error Hub bundles contain:

• BundleManifest
• one AgentTrajectory
• an optional DiagnosticReport
• optional artifact paths

BundleManifest currently has schema_version = "1.0.0". This version covers the bundle manifest and directory layout, not every trajectory, event, finding, or report record.

Hub push scrubs common secrets and PII by default. The scrubber operates over the trajectory goal and metadata plus event input, output, error, and metadata. It does not inspect artifact file contents or rewrite Artifact fields. Local trace storage is not automatically scrubbed.

12. Schema Evolution Status¶

Current facts:

• AgentTrajectory, AgentEvent, and DiagnosticReport do not contain a schema_version field.
• There is no trajectory/report migration registry.
• Read helpers rely on Pydantic validation and field defaults.
• Additive fields with defaults, such as DiagnosticReport.audit, remain readable from older payloads because the missing value is defaulted.
• Breaking field renames or type changes are not automatically migrated.

Until record-level versioning is implemented, schema changes should remain additive where possible, preserve compatibility import paths, and include JSON round-trip tests for old and current payload shapes.

13. Validation and Tests¶

The canonical validation entry points are Pydantic construction and the JSON read helpers. There is no separate agentdebug.schema.validate() function.

The test suite currently checks:

• trajectory JSON round trips
• diagnostic report and audit round trips
• Pydantic v1 and v2 compatibility in CI
• adapter conversion into AgentTrajectory
• JSONL and SQLite persistence
• public confidence filtering
• DeepDebug audit visibility in serialized CLI reports

When adding a schema field, add a default unless the change intentionally breaks compatibility, export the public type from agentdebug.schema, and add round-trip coverage before changing adapters or stores.

GUI / CUA Debugger Commands¶

The GUI RCA pipeline ships inside the agentdebug package as agentdebug.gui. It expects OSWorld-style trajectory logs; it does not include OSWorld runtime or agent runner code.

Everything below resolves paths against your current working directory, so run these commands from the directory that holds your results/ tree.

Install¶

The RCA main path (agentdebug.gui.rca, .ingester, .taxonomy, .tagger) needs nothing beyond a core install. The pipeline, provider adapters and annotation UI are optional:

pip install "agentdebugx[gui]"                       # + screenshot decoding
pip install "agentdebugx[gui,gui-app]"               # + providers, pipeline, UI
pip install "agentdebugx[gui,gui-memory,gui-app]"    # + lesson/episodic memory

Configuration¶

Configuration is resolved in this order, first hit wins:

1. $AGENTDEBUG_GUI_CONFIG
2. ~/.agentdebug/gui.json
3. ./debugger/config/debugger.json
4. the built-in defaults

A copy of the defaults ships with the package as a starting point:

python -c "from pathlib import Path; import agentdebug.gui.config as c; \
print(Path(c.__file__).parent / 'config' / 'debugger.example.json')"

API keys must be set through environment variables, never committed in JSON files:

Run RCA¶

python -m agentdebug.gui \
  --trajectory-dir results/input_trajectory/claude-sonnet-4-5-20250929_50steps \
  --output-dir results/debugger_results \
  --trial-name claude-sonnet-4-5-20250929_50steps \
  --provider openai \
  --model gpt-4o-mini

Result Layout¶

results/debugger_results/
  <trial_name>/
    annotations/
    <debugger_model>/
      rca/
      summary.json
      episodic.json

Annotation UI¶

streamlit run needs a file path, so ask the package where it installed the app:

streamlit run "$(python -c 'from agentdebug.gui.vis import app_path; print(app_path())')"

For read-only inspection, use the main FastAPI dashboard instead:

pip install "agentdebugx[ui]"
agentdebug serve --store-sqlite .agentdebug/traces.sqlite

An imported OSWorld trace with screenshots beneath its recorded source_dir opens in Visual mode automatically. The shared timeline controls the screenshot comparison, step metadata, click marker, and RCA evidence; Trace / Visual switches views without rerunning diagnosis. Before/After panes preserve every role-tagged image for the selected event.

Discuss with Debugger in the FastAPI dashboard works for OSWorld and all other normalized trace formats. It uses the shared LLM settings, persists report-pinned sessions locally, and can export a report-revision draft without changing the stored report. Streamlit is still required for annotation writes, reviewer assignment, and accuracy tooling.

Accuracy¶

Pass the debugger subdirectory, not the agent-level directory:

from agentdebug.gui.eval import quick_acc, compute_accuracy
quick_acc("results/debugger_results/<trial>/<debugger>")
compute_accuracy("results/debugger_results/<trial>/<debugger>")

Other entry points¶

python -m agentdebug.gui.scripts.download_input_trajectory --help
python -m agentdebug.gui.vis.generate_assignments results/debugger_results/<trial>

Troubleshooting¶

agentdebug is not found¶

Confirm that the package was installed into the active Python environment:

python -m pip show agentdebugx
python -m pip install agentdebugx

The install name is agentdebugx; the command is agentdebug.

An optional integration cannot be imported¶

Run:

agentdebug doctor

Then install the extra that owns the integration. Examples:

python -m pip install "agentdebugx[ui]"
python -m pip install "agentdebugx[crewai]"
python -m pip install "agentdebugx[openai-agents]"

python -m agentdebug.gui reports a missing package¶

The GUI batch application needs the memory and application layers:

python -m pip install "agentdebugx[gui,gui-memory,gui-app]"

Core GUI RCA and import agentdebug.gui intentionally have a smaller dependency boundary.

LLM diagnosis cannot connect¶

Inspect masked configuration and test the endpoint:

agentdebug config show
agentdebug config doctor

Check that the base URL is an OpenAI-compatible API root expected by the configured client, the model exists on that endpoint, and the API key is available.

GUI RCA cannot find screenshots¶

The OSWorld adapter stores screenshot paths and the resolved source directory. Check that:

1. the trajectory was imported from the original OSWorld directory,
2. metadata.source_dir still points to that directory,
3. screenshot files have not been moved, and
4. Pillow is installed when decoding is required.

python -m pip install "agentdebugx[gui]"

A rerun plan says execution is unavailable¶

This is expected for a trajectory-only input. A real rerun needs an application-owned runner with the framework, model, tools, credentials, and environment state.

Create a plan to inspect the missing capabilities:

agentdebug rerun report.json \
  --trajectory trace.json \
  --plan-only \
  --out rerun-plan.json

Then configure a persistent HTTP runner or a trusted process runner as described in Validate with Rerun.

The local UI is reachable from other machines¶

The documentation examples bind to 127.0.0.1. If you intentionally bind to 0.0.0.0, place the UI behind suitable authentication and transport security. The UI itself is designed as a local surface.

A batch exits with status 3¶

Status 3 means the batch partially failed. Successful records are retained. Inspect batch-summary.json and the per-record outputs to identify isolated invalid inputs.

Get exact version-specific flags¶

agentdebug --help
agentdebug ingest --help
agentdebug diagnose --help
agentdebug rerun --help

Build these docs¶

The documentation site is built from Markdown under docs/ with MkDocs Material.

Install documentation dependencies¶

From the repository root:

python -m pip install -r requirements-docs.txt

Preview locally¶

mkdocs serve

Open http://127.0.0.1:8000. MkDocs reloads the preview when a source file changes.

Run the same strict build as CI¶

mkdocs build --strict

The generated site/ directory is a build artifact and should not be committed.

Source-of-truth rules¶

When updating a tutorial:

1. verify CLI flags against agentdebug <command> --help,
2. verify Python behavior against the implementation and focused tests,
3. distinguish deterministic results, model-produced analysis, simulation, and observed live execution,
4. do not copy experiment metrics into usage documentation without a versioned source, and
5. keep optional dependency boundaries aligned with pyproject.toml.

GitHub Pages deployment¶

.github/workflows/docs.yml builds documentation for pull requests and deploys the main branch through GitHub Pages. In the repository settings, set Pages → Build and deployment → Source to GitHub Actions.

The configured site URL is:

https://agentdebugx.github.io/AgentDebugX/

Deployment is separate from the GitHub Wiki. The Markdown sources remain versioned with the code in this repository.