Logging Tool Calls, Inputs, and Outputs for Debuggable Agents

The Logging Gap in AI Agent Systems

Most engineering teams have sophisticated logging for their traditional application code. Requests are logged with timestamps, status codes, and latency. Database queries are logged with execution times. But when they add an AI agent layer, logging discipline often disappears. The agent 'just works' in development, and its internal operations, which LLM calls it made, which tools it invoked, what inputs and outputs flowed through each, are not captured anywhere.

This creates a specific and painful class of production incident: the agent produces a wrong or harmful output, a user reports it, and the engineering team cannot reconstruct what happened. They cannot see what prompt the agent sent to the model, what the model returned, which tools were called with what arguments, or which step in the reasoning chain went wrong. Without this information, the investigation is guesswork.

The solution is comprehensive structured logging of every operation the agent performs, designed from the start to answer the questions you will need to answer during an incident: What did the agent receive? What did it decide to do? What happened when it tried to do it? What did it ultimately return?

What to Log: The Complete Field Specification

Every tool call, every LLM invocation, and every agent decision should produce a structured log record. The record should be rich enough to reproduce the full context of the operation without requiring you to dig through other systems.

Required Fields: The Non-Negotiables

• trace_id: The distributed trace identifier linking this log to all other operations in the same user request. Without this, log records are isolated islands.
• span_id: The specific span within the trace. Allows exact lookup of the corresponding trace span for latency correlation.
• timestamp: ISO 8601 with millisecond precision. Always UTC. Never relative timestamps in log records.
• agent_id: A unique identifier for the specific agent instance that produced this log. Enables per-agent filtering and performance comparison.
• operation: The type of operation, 'llm_call', 'tool_call', 'agent_decision', 'agent_spawn'. Enables fast filtering.
• tool_name or model: The specific tool or model involved. Enables per-tool and per-model analysis.
• input: The exact input that was sent. For LLM calls, the full message array. For tool calls, the exact arguments dict.
• output: The exact output received. For LLM calls, the full response text. For tool calls, the result payload.
• duration_ms: Wall clock time from invocation to completion. The most important performance metric.
• status: 'success', 'error', or 'timeout'. Never log without this.

Optional Enrichment Fields

• input_tokens and output_tokens: Token counts for LLM calls. Required for cost tracking and budget enforcement.
• cost_usd: Calculated cost for this specific call. Useful for per-request cost attribution.
• model_version: The full model identifier including version string (e.g., claude-sonnet-4-5-20251022). Model behavior changes between versions.
• session_id: Groups all operations for a single user conversation. Enables replay of an entire session.
• user_id: Links agent operations to specific users for per-user debugging and SLA tracking.
• retry_attempt: Which retry this is if the operation was retried. Helps identify flaky tools.
• cached: Whether the response came from a cache. Important for debugging consistency issues.

Structured Logging Over Free-Text

The most important logging decision you can make is committing to structured JSON logs rather than free-text log messages. Free-text logs like 'Tool web_search called with query: how to optimize LLM costs' are human-readable but machine-hostile. You cannot efficiently filter them, aggregate them, or build dashboards over them without brittle string parsing.

Structured logs emit every field as a separate key-value pair in JSON. The log aggregation backend indexes each field independently, letting you query for all tool calls with duration_ms greater than 1000, or all llm_call records for the claude-opus model, with index-speed lookups across billions of records.

import structlog
import time
from typing import Any

logger = structlog.get_logger()

async def logged_tool_call(
    tool_name: str,
    arguments: dict,
    tool_fn,
    agent_id: str,
    trace_id: str,
    span_id: str,
) -> Any:
    start = time.monotonic()
    log = logger.bind(
        trace_id=trace_id,
        span_id=span_id,
        agent_id=agent_id,
        operation="tool_call",
        tool_name=tool_name,
        input=arguments,  # Log the exact input
    )

    try:
        result = await tool_fn(**arguments)
        duration_ms = (time.monotonic() - start) * 1000

        log.info(
            "tool_call.success",
            output=result,
            duration_ms=round(duration_ms, 2),
            status="success",
            output_size_bytes=len(str(result).encode()),
        )
        return result

    except Exception as e:
        duration_ms = (time.monotonic() - start) * 1000
        log.error(
            "tool_call.error",
            error_type=type(e).__name__,
            error_message=str(e),
            duration_ms=round(duration_ms, 2),
            status="error",
        )
        raise

Log Levels for Agent Operations

Log levels for agent systems should be defined by the operational impact of the event, not by developer convention. The following levels work well for production agent logging.

• DEBUG: Full prompt content, full tool arguments, full LLM response text. Extremely verbose, enable only when actively debugging a specific issue in a non-production environment.
• INFO: Tool call start and completion, LLM call start and completion with token counts and latency. The standard production log level.
• WARNING: Tool call retries, fallback model activation, context window approaching limit, cost per request exceeding threshold.
• ERROR: Tool call failure after all retries exhausted, LLM call failure, timeout, unexpected exception in agent logic.
• CRITICAL: Agent produces output that trips a safety filter, security-relevant events like suspected prompt injection, service degradation affecting all requests.

Logging LLM Inputs Safely

Logging the full input to every LLM call, the system prompt plus the conversation history plus any retrieved documents, is invaluable for debugging. But it creates two risks: storage cost and data sensitivity. A single LLM input can be 50,000 tokens. At production scale, storing full inputs for every call is expensive. And if user messages or retrieved documents contain PII or confidential information, storing them in a log system with broad access is a compliance risk.

Selective Input Logging Strategy

The practical solution is selective input logging: always log the structure (which messages are present, their roles, their token counts) but selectively log content based on a sampling rate or a debug flag. For error cases, always log the full input. That's when you need it most. For successful calls, log a configurable percentage (e.g., 5%) for ongoing quality monitoring.

import random
from dataclasses import dataclass

@dataclass
class LLMCallLogConfig:
    log_full_input_on_error: bool = True
    log_full_input_sample_rate: float = 0.05  # 5% of successful calls
    log_full_output_on_error: bool = True
    log_full_output_sample_rate: float = 0.05
    max_input_log_chars: int = 2000  # Truncate for sampled logs
    redact_patterns: list[str] = None  # Regex patterns for PII

def should_log_full_content(
    status: str,
    config: LLMCallLogConfig,
) -> bool:
    if status == "error" and config.log_full_input_on_error:
        return True
    return random.random() < config.log_full_input_sample_rate

async def logged_llm_call(
    messages: list[dict],
    model: str,
    config: LLMCallLogConfig,
    **kwargs,
) -> str:
    # Always log structural metadata
    input_meta = {
        "message_count": len(messages),
        "roles": [m["role"] for m in messages],
        "estimated_tokens": sum(len(m["content"].split()) * 1.3 for m in messages),
    }

    response_text, status = None, "success"
    try:
        response_text = await call_llm(model, messages, **kwargs)
        return response_text
    except Exception as e:
        status = "error"
        raise
    finally:
        log_data = {"operation": "llm_call", "model": model, "input_meta": input_meta, "status": status}

        if should_log_full_content(status, config):
            log_data["input_sample"] = str(messages)[:config.max_input_log_chars]
            if response_text:
                log_data["output_sample"] = response_text[:config.max_input_log_chars]

        logger.info("llm_call.complete", **log_data)

PII Redaction and Sensitive Data Handling

Agent systems that handle user data frequently have LLM inputs containing PII: names, email addresses, phone numbers, account numbers. Logging this data without redaction violates GDPR, CCPA, and similar regulations, creates compliance liability, and expands your breach surface if the log system is compromised.

import re
from typing import Any

PII_PATTERNS = [
    (r'\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Z|a-z]{2,}\b', '[EMAIL]'),
    (r'\b\d{3}[-.]?\d{3}[-.]?\d{4}\b', '[PHONE]'),
    (r'\b\d{4}[\s-]?\d{4}[\s-]?\d{4}[\s-]?\d{4}\b', '[CARD]'),
    (r'\b(?:\d{3}-\d{2}-\d{4}|\d{9})\b', '[SSN]'),
    (r'\bsk-[a-zA-Z0-9]{40,}\b', '[API_KEY]'),  # OpenAI key pattern
    (r'\bsk-ant-[a-zA-Z0-9]{90,}\b', '[API_KEY]'),  # Anthropic key pattern
]

def redact_pii(text: str) -> str:
    for pattern, replacement in PII_PATTERNS:
        text = re.sub(pattern, replacement, text)
    return text

def redact_log_value(value: Any) -> Any:
    if isinstance(value, str):
        return redact_pii(value)
    if isinstance(value, dict):
        return {k: redact_log_value(v) for k, v in value.items()}
    if isinstance(value, list):
        return [redact_log_value(item) for item in value]
    return value

Log Aggregation Architecture

Individual log records from each agent process need to flow into a centralized aggregation system where they can be queried, alerted on, and retained according to your policies. The standard architecture for production agent systems is: agent processes emit JSON to stdout, a log shipper (Fluentd, Vector, Filebeat) collects and routes them, a stream processor enriches or filters at high throughput, and a log storage backend indexes and retains them.

Choosing a Log Storage Backend

• OpenSearch / Elasticsearch: Best full-text search capabilities. Excellent for ad-hoc debugging queries. Higher storage cost for large volumes due to index overhead.
• ClickHouse: Columnar storage optimized for analytical queries over large datasets. 10x to 50x more cost-efficient than Elasticsearch at scale. Use for high-volume agent systems.
• Loki (Grafana): Lightweight, label-based indexing. Lowest storage cost but limited query expressiveness. Good for high-volume low-cardinality log streams where you filter by known labels.
• Datadog Logs: Managed service with tight APM integration. Best for teams already on Datadog. Premium pricing that scales linearly with volume.

Querying Logs for Debugging

The value of structured logs is fully realized when you can query them efficiently during an incident. The following queries cover the most common debugging scenarios for agent systems.

-- Find all operations for a specific user session
SELECT timestamp, operation, tool_name, status, duration_ms, error_message
FROM agent_logs
WHERE session_id = 'sess_abc123'
ORDER BY timestamp ASC;

-- Find the slowest tool calls in the last 24 hours
SELECT tool_name, agent_id,
       COUNT(*) AS call_count,
       PERCENTILE_CONT(0.95) WITHIN GROUP (ORDER BY duration_ms) AS p95_ms,
       AVG(duration_ms) AS avg_ms,
       SUM(CASE WHEN status='error' THEN 1 ELSE 0 END) AS errors
FROM agent_logs
WHERE operation = 'tool_call'
  AND timestamp > NOW() - INTERVAL '24 hours'
GROUP BY tool_name, agent_id
ORDER BY p95_ms DESC;

-- Find all requests where the agent fell back to a secondary model
SELECT trace_id, session_id, timestamp, model
FROM agent_logs
WHERE operation = 'llm_call'
  AND model NOT IN ('claude-sonnet-4-5', 'gpt-4o')  -- Not the primary models
  AND timestamp > NOW() - INTERVAL '1 hour'
ORDER BY timestamp DESC;

Retention, Cost, and Compliance

Log retention is a balance between operational need, storage cost, and compliance requirement. For most agent systems, a tiered retention strategy works well: hot storage for recent logs (instant query response, expensive), warm storage for medium-term logs (seconds to query, cheaper), and cold archival for compliance retention (minutes to access, cheapest).

• Hot tier (0-7 days): All logs at full fidelity. Used for active incident investigation and daily monitoring.
• Warm tier (7-90 days): Compressed and downsampled. Tool call logs kept at 100%. LLM input/output content sampled at 10%.
• Cold tier (90-365+ days): Structured metadata only (no content). Required for compliance audit trails. Stored in object storage at minimal cost.