← Agent Frameworks 🕐 9 min read
Agent Frameworks

Agent Trace Format Crosswalk 2026

State of AI · June 2026

> **Status: crosswalk appendix.** Use this page for source-by-source evidence

See also (wiki): wiki/agent-telemetry-format-standardization.md · wiki/agent-operations-observability.md · wiki/semantic-observability.md

Source status: primary documentation crosswalk plus local schema observations and conversion experiment; retrieved and updated 2026-07-23.

Evidence Boundaries: Native formats and vendor exports remain version-sensitive; local conversion results are not vendor conformance claims.

Status: crosswalk appendix. Use this page for source-by-source evidence mapping. Use Agent Observability Standardization Playbook for the current synthesis and Agent Observability Format Standard 2026 for current config/schema guidance.

July 2026 Publication Update: Interchange Standards

This page is now the standards crosswalk for agent trajectory and evaluation evidence. The formats are complementary, not competing replacements:

Layer Canonical question Recommended role
Native records What did the provider, harness, or framework emit? Preserve unchanged with version and hash
OpenTelemetry GenAI What executed, when, and with which parent/child relationships? Runtime transport and correlation envelope
OpenInference How should LLM, agent, tool, retriever, chain, and evaluator spans be named and attributed? Span semantic projection on OTel
ATIF What was the complete agent trajectory of a run? Trajectory interchange and replay/eval input
Agent Data Protocol How can trajectories, actions, and observations move between datasets and training pipelines? Dataset/training interchange
Harbor How is a controlled environment/task executed and rewarded? Eval harness and artifact producer
eval_result_v1 What result can be safely rolled up across repos and teams? Governance and scorecard contract

ATIF status

The local ATIF work previously demonstrated only a redacted Codex JSONL conversion into an ATIF-like trajectory and OpenInference-like spans. It did not establish full ATIF conformance. The current public NVIDIA NeMo Agent Toolkit model documents ATIF versions through ATIF-v1.7 and uses Pydantic validation. That is now recorded as the version-sensitive primary boundary: pin the ATIF version and library commit in every adapter.

Required conformance tests

Before a format adapter is treated as eligible for production review, it must pass:

  1. Native fixture to normalized trace conversion without losing parent/child IDs, timestamps, tool calls, retrieval evidence, errors, or artifacts.
  2. Normalized trace to ATIF and back, with an explicit loss report.
  3. ATIF to ADP trajectory conversion, preserving action/observation order and task/session identity.
  4. Harbor trial artifacts to trajectory plus eval_result_v1, separating target failure from environment/verifier failure.
  5. Redaction tests showing that prompts, tool arguments, tool outputs, file paths, secrets, and retrieved documents follow their privacy class.
  6. Cross-harness fixtures for Codex CLI, Claude Code, Strands, LangGraph, and Google ADK recordings.

Firm recommendation

Do not select ATIF, ADP, or OpenInference as the sole enterprise record. Keep native evidence in a protected artifact store, use OTel/OpenInference for runtime correlation, use ATIF/ADP as explicit projections for trajectory and training workflows, and publish only normalized result facts plus immutable artifact references to the firm rollup.

Primary references: NVIDIA ATIF model · OpenTelemetry semantic conventions · OpenTelemetry GenAI convention changes · Agent Data Protocol

Executive Summary

Agent observability only becomes useful when each record is treated as evidence for a specific claim. A local transcript proves what the assistant/user session saw. An OTel trace proves timing, nesting, and correlation. A Bedrock invocation log proves a supported model-runtime call occurred. AG-UI and A2A events prove protocol-level UI and delegation activity. None of them alone proves that the agent completed a task correctly.

The practical contract is a crosswalk: preserve native records, normalize them into OpenTelemetry/OpenInference-style spans where possible, and attach labels that distinguish audit, replay, evaluation, skill mining, KB mining, and training-data candidates.

Canonical Record Types

Surface Native record shape Best normalized unit Content risk Main use
Claude Code local sessions Project JSONL rows under ~/.claude/projects/... with event-like user, assistant, attachment, queue, prompt-title, and operation records Session event plus optional span High: prompts, outputs, attachments Replay, skill mining, local task corpus
Claude Code OTel Metrics, logs/events, optional traces; content gates for prompts and tool details/content OTel log/event and trace span Config-dependent Fleet metrics, tool reliability, task timing
Claude Agent SDK Application spans plus Claude Code child-process telemetry linked by W3C trace context Linked trace tree Config-dependent End-to-end app plus agent-loop tracing
Claude Cowork OTel Admin-configured OTLP events keyed by prompt.id, session, org, user/account, workspace, tool result metadata OTel log/event and trace-like prompt group High: prompt content, user email, file paths Enterprise activity audit and prompt-level correlation
Codex CLI OTel otel.* config for prompt logging, metrics, traces, OTLP endpoint/header/protocol/TLS OTel log/event/span Config-dependent CLI agent metrics and traces
Codex local state ~/.codex config, session index JSONL, SQLite logs/state, shell snapshots, skills, memories, plugin cache Session ledger plus local operation rows High: prompts, command logs, filenames Replay, debugging, skill mining
Codex Desktop App-server plus global/project config behavior; exact OTel/schema still needs live validation TBD; likely OTel plus local session metadata High Desktop agent audit and replay once contract is verified
Microsoft Scout Preview desktop app session history, memory, skills, Heartbeat, command/browser/M365/WorkIQ activity; public export contract unclear TBD High: local files, commands, M365 data, labels Enterprise desktop-agent audit and KB mining
Bedrock invocation logging CloudWatch/S3 invocation log records for supported bedrock-runtime operations Cloud model invocation event High: model input/output if enabled Model-call audit, cloud request correlation
Bedrock AgentCore Observability CloudWatch GenAI observability data from ADOT/OTel instrumentation, including traces, sessions, logs, metrics OTel trace/log/metric Config-dependent Production agent monitoring and session tracing
Strands Agents Framework OTel spans for agent runs, model calls, tools, and significant actions OTel trace tree Config-dependent Agent runtime debugging and CloudWatch/Phoenix/Langfuse integration
AG-UI Streaming events: run/step lifecycle, text message chunks, tool-call events, state snapshot/delta, custom/raw events Protocol event attached to span Medium to high UI replay and frontend/agent boundary tracing
A2A JSON-RPC methods and task/message/status/artifact objects; streaming via SSE in supported flows Protocol task/message/artifact event attached to span Medium to high Agent-to-agent delegation audit
Phoenix/OpenInference OTel-compatible spans with OpenInference semantic conventions for LLM/tool/retriever/chain/etc. Span tree Config-dependent Trace inspection, evaluation, dataset extraction
ATIF Agent trajectory objects imported by Phoenix and converted into OTel-compatible span trees Trajectory High Agent trajectory evaluation and replay

Join Keys To Preserve

Key Why it matters
trace_id, span_id, parent_span_id Reconstruct causality and timing across agent, model, and tool calls
session_id / agent.session_id Group many runs or prompts into one human-agent session
prompt.id Correlate Claude Cowork events triggered by the same user prompt
thread_id Join local Codex logs/session index to user-visible threads
tool_call_id Join tool request, streamed arguments, result, error, and downstream edits
protocol.run_id Join AG-UI lifecycle and UI stream events
a2a.task_id / contextId Join delegated agent work, status updates, artifacts, and history
aws.request_id Join application spans to Bedrock invocation logs and CloudTrail-style audit
aws.account_id, aws.region Prevent false joins across accounts or Regions
model.provider, model.id Attribute cost, latency, failures, and tuning effects
repo, git.commit, workspace.path_hash Reproduce coding-agent context without exposing full paths broadly
artifact_id Link generated files, structured outputs, eval results, and training candidates
privacy.redaction_version Know which redaction policy transformed a record

Evidence Boundaries

Claim Strongest evidence Weak evidence
“The model was invoked” Bedrock invocation log or provider request ID joined to app trace Local transcript text alone
“The agent ran a shell command” Local agent tool event, shell snapshot, terminal capture, or Scout command log Model output that says it will run a command
“The workflow completed correctly” Deterministic validation, test output, human approval, or benchmark score linked to trace RUN_FINISHED or assistant success wording
“A KB answer was hard to find” High retrieval churn, many query rewrites, late successful citation, repeated similar user questions One failed search
“A prompt got better” Same evaluation set before/after optimization with pass/fail and cost/latency Anecdotal side-by-side transcript
“Training data is safe to use” Redaction, license/retention review, human label, source provenance, dedupe, train/test split Raw chat log export

Local Format Notes

Local inspection in this workspace found:

  • Claude project session JSONL rows under ~/.claude/projects/-Users-pierre-consulting-stateofai/ with keys such as type, content, operation, sessionId, and timestamp.
  • Global Claude settings exporting OTel metrics/logs to http://localhost:4318, without the inspected prompt/tool-content gates enabled.
  • No project .codex directory in this repo.
  • Global Codex config in ~/.codex/config.toml, with this workspace marked trusted.
  • Codex ~/.codex/logs_2.sqlite with a logs table containing timestamp, severity, target, log body, module/file/line, thread ID, process UUID, and estimated byte size.
  • Codex ~/.codex/session_index.jsonl containing thread IDs, thread names, and update timestamps.

These are local observations, not vendor contracts. They should be sampled with schema-only or redacted extraction before any shared reporting.

Normalization Pattern

Use a two-table minimum viable lake:

raw_events

Field Purpose
raw_event_id Stable local ID
source_system claude_code_local, claude_code_otel, codex_local, bedrock_invocation, ag_ui, a2a, etc.
native_schema_version Version or observed extractor version
observed_at Collector timestamp
native_timestamp Event’s own timestamp
join_keys JSON object containing trace/session/thread/tool/request IDs
payload_redacted Redacted native payload
payload_hash Hash of full raw payload when raw storage is allowed
redaction_version Policy version

agent_spans

Field Purpose
trace_id, span_id, parent_span_id OTel-style tree
span_kind OpenInference-style LLM, TOOL, CHAIN, RETRIEVER, AGENT, PROTOCOL, EVAL
name Human-readable action name
start_time, end_time Timing
status Success, error, timeout, aborted, user-denied
input_ref, output_ref References into redacted/raw vault
model, tool, protocol Structured attributes
eval_label Optional pass/fail/failure-mode label

Do not flatten everything into chat messages. Tool calls, file edits, retrieval, approvals, UI events, and cloud invocations need their own rows.

Format Conversion Recipes

Local Transcript To Skill Candidate

  1. Group events by session/thread.
  2. Split into task episodes using user prompts, stop events, repo edits, or validation commands.
  3. Keep only episodes with a validation artifact: passing build/test, accepted generated article, human approval, or successful retrieval citation.
  4. Extract trigger, repeated command/search sequence, recovery moves, and final validation.
  5. Promote to a skill only after at least three similar successful episodes and at least one known counterexample.

Trace To KB Defect

  1. Find episodes with repeated searches, many query rewrites, or high elapsed retrieval time.
  2. Identify the eventual answer source, if any.
  3. Label the defect: missing alias, stale title, bad access control, no owner, missing canonical page, no citation anchor, or no answer exists.
  4. Add a synthetic eval query that should find the source in the future.

Trace To YAML Optimization Dataset

  1. Save the raw user request, schema version, generated YAML, validator output, and final corrected YAML.
  2. Convert deterministic validator failures into textual feedback.
  3. Build a small adversarial evaluation set before tuning: missing required fields, wrong enum, invalid nesting, ambiguous domain terminology, unsafe defaults, and semantically invalid but syntactically valid YAML.
  4. Use 30-100 curated examples for prompt optimization/GEPA-style instruction refinement.
  5. Move toward 100-300 examples when the issue is coverage across variants, not just prompt wording.
  6. Consider SFT only after constrained decoding, schema validation, retrieval, and prompt optimization fail the reliability/cost/latency target.

Open Questions

  • Does the current Codex Desktop build honor project-scoped .codex/config.toml for OTel, or only user-level ~/.codex/config.toml?
  • What exact local database/session files does Codex Desktop use across macOS, Windows, and remote-control flows?
  • What export, admin audit, retention, and OTel surfaces does Microsoft Scout expose beyond preview session-history and memory docs?
  • Can Claude/Codex local JSONL/SQLite be converted to ATIF without Harbor while preserving enough tool/action semantics for Phoenix?
  • Which OpenInference attributes should represent protocol-native AG-UI and A2A events: custom span kinds, EVENT, TOOL, or explicit protocol spans?
  • For domain YAML generation, where is the break-even between validator-driven prompt optimization and LoRA/SFT for the specific schema families in this repo?

Sources

State of AI | brandon@brandonsneider.com June 2026