← Blog

Engineering · · 4 min read

How to debug an agent without copying every secret into its traces

Capture enough identity, timing, schema, and result evidence to reproduce failures while limiting sensitive payload retention.

Deixic

A failed account lookup is difficult to reproduce, so someone enables full request and response logging. The trace now contains customer notes, an access token from an error message, and a private document excerpt. Debugging has created another place the company must protect and eventually clean up.

Keep the evidence ordinary operations need, and capture other payloads through a restricted path. An MCP audit-context proposal raises both traceability and the sensitivity of user intent. Treat proposed metadata as an idea to evaluate: it is neither authenticated proof nor a universal standard.

Start with the failure you need to distinguish#

  • A tool-routing error needs the presented tool definitions, selected name, arguments after safe transformation, and result classification.
  • A timeout needs the request timing, attempt identity, provider identifier, and last known operation state.

Neither needs the complete document body. A retrieval-quality problem can need selected excerpts. Capture them through a restricted diagnostic path with an owner and retention period, and keep a reference in ordinary telemetry so operators can locate the evidence without exposing it to every dashboard viewer.

OpenTelemetry's sensitive-data guidance discusses avoiding, filtering, and transforming sensitive telemetry. Apply them before export instead of relying on downstream consumers.

Build a useful minimal trace#

A failed support brief records the task identifier, authenticated agent identity, connector operation, schema version, target type, elapsed time, status, and provider request identifier. It can record that the response lacked renewal_date without logging customer fields.

Keep logical operation and attempt identifiers separate, so an engineer can see whether one operation retried three times or three operations targeted similar objects. Record the retry reason and scheduling component.

Do not use raw tokens as correlation identifiers. Hashing does not make a sensitive value anonymous, because small or predictable value spaces can be guessed. When cross-system correlation is needed, use purpose-built opaque identifiers with an access-controlled mapping.

Diagnostic question Usually useful evidence Payload requiring extra care
Which tool was selected? Tool identity and definition version Full user prompt
Why did validation fail? Field name, expected type, error class Rejected raw value
Did the provider accept it? Provider request or object ID Authentication headers
Which document supported the answer? Source ID and version Full document text
Why was access denied? Trusted decision code and identity reference Tokens and private policy inputs

Separate statements from decisions#

A model-generated reason records what the model said it intended and does not prove user authorization. A client-supplied model name is useful metadata but does not establish which service executed the request.

Record authoritative decisions at the component that makes them, and keep the policy result and trusted identity reference separate from generated rationale. The agent audit-row guide explains why action history needs more than a plausible narrative. The separation helps debugging: for a denied tool call, an engineer can tell the model's requested action from the server's decision without reading a transcript.

Reproduce with a smaller fixture#

Once you understand the failure, reduce the payload to a synthetic or sanitized fixture that preserves the causal shape. A missing field, malformed date, or duplicated tool-call identifier rarely needs the customer's name and notes.

Confirm the reduced fixture still fails before deleting or restricting the original capture.

For content-dependent failures, keep only the minimum permitted excerpt, document why it is necessary, restrict who can retrieve it, and make sure exports and screenshots do not bypass the same protection.

Test redaction as behavior#

Send a fake credential marker through headers, query parameters, tool arguments, exception messages, and nested result fields in a controlled test, and check where it appears after every exporter and processor. A rule covering normal responses can miss an exception stack or a retry log.

Also test that redaction leaves enough evidence to diagnose the original problem: removing all arguments can hide that the wrong target was selected, so replace sensitive values with stable diagnostic references.

Keep the diagnostic fixture with the fix and the capture policy with the telemetry configuration. Deixic's evidence and activity guide and agent identity documentation describe inspection points.