Secrets Management in Agent Runtimes: Keeping Credentials Out of the Context Window

Every agentic system that does anything useful needs credentials. An API key to query a language model, a connection string to read from a database, a webhook secret to send notifications, OAuth tokens to act on behalf of a user. Managing these credentials correctly in an agent runtime is significantly harder than in a traditional service, for one specific reason: agents process text from untrusted sources, and that text ends up in the same place as your secrets.

A traditional web service keeps its database password in an environment variable. The only ways that password leaks are if the variable gets logged, the process gets inspected, or the secret store is compromised. An agentic system has an additional leak vector that traditional services simply don't have: the context window. If your system prompt or tool instructions contain a credential value, that credential is now inside the model's context. It can be reflected back in outputs, logged by your observability system, or extracted by a prompt injection attack.

Why Secrets Are Different for Agents

Traditional secret management advice focuses on keeping secrets out of source code, environment variable plaintext, and log files. All of that still applies to agents. But agents introduce three additional threat vectors that traditional services simply don't have.

• Context window exposure: if a secret value appears anywhere in the model's input or output, it's at risk. The model may reflect it back, your observability system may log it, or a prompt injection attack may exfiltrate it.
• Tool call parameter leakage: when an agent calls a tool that requires authentication, the credentials used for that call may appear in tool call parameters that get logged as part of the agent trace.
• Prompt injection extraction: an attacker who successfully injects instructions into the agent's context may be able to instruct it to output its credentials. An agent that has credentials in its system prompt is a credential exfiltration risk by design.

The Surface Area Problem

The core principle for secrets in agent systems: secret values should never appear in any text the agent processes. Not in the system prompt. Not in tool descriptions. Not in tool call parameters. Not in tool results. The agent should work with references to secrets, and the actual credential values should be resolved by the execution layer below the agent, invisibly.

What Never to Do

Before covering correct patterns, it's worth being explicit about what to avoid. These mistakes are common, and the consequences are severe.

# ❌ Never hardcode credentials in agent configuration
system_prompt = f"""
You are a research agent. Use the OpenAI API with key sk-live-abc123xyz
to call the embeddings endpoint when needed.
"""

# ❌ Never pass credentials as tool arguments visible to the model
tools = [{
    "name": "call_database",
    "description": "Query the database",
    "parameters": {
        "connection_string": {
            "type": "string",
            "description": "Use postgres://user:PASSWORD123@host/db"
            # ^ this appears in the model's context
        }
    }
}]

# ❌ Never format credentials into messages
user_msg = f"Search the API. Your key is {os.environ['API_KEY']}"
# ^ now in the LLM's input, logged by your observability system

# ❌ Never log tool call parameters without redaction
logger.info(f"Tool called: {tool_name}, params: {json.dumps(params)}")
# ^ if params contains Authorization headers, they're now in your logs

Vault-Based Architecture

The correct architecture puts a secrets vault between the agent and its credentials. The agent never sees credential values. It sees vault references. When the execution layer needs to make an authenticated call, it fetches the credential from the vault, uses it, and discards it, all without the value ever appearing in the agent's context.

HashiCorp Vault, AWS Secrets Manager, Google Secret Manager, and Azure Key Vault all support this pattern. The core capability you need: fetch a secret value programmatically at runtime using a service account that has been granted access to that specific secret and no others.

import boto3
from functools import lru_cache
import json

class SecretsManager:
    def __init__(self, region: str = "us-east-1"):
        self.client = boto3.client("secretsmanager", region_name=region)
        self._cache: dict = {}

    def get(self, secret_name: str) -> str:
        """Fetch secret value, never log or return to agent context."""
        if secret_name not in self._cache:
            response = self.client.get_secret_value(SecretId=secret_name)
            # Store in-process cache (never in agent context or logs)
            self._cache[secret_name] = response["SecretString"]
        return self._cache[secret_name]

    def clear_cache(self):
        """Call after task completion to flush credentials from memory."""
        self._cache.clear()

secrets = SecretsManager()

# Tool implementation, credential is fetched below the agent layer
async def call_external_api(endpoint: str, payload: dict) -> dict:
    """The agent calls this function. It never sees the API key."""
    api_key = secrets.get("myapp/openai-api-key")  # fetched at runtime
    response = await httpx.post(
        endpoint,
        headers={"Authorization": f"Bearer {api_key}"},
        json=payload,
    )
    # api_key is a local variable here, never returned to agent
    return response.json()

# What the agent sees in its tool definition:
# "call_external_api(endpoint, payload), calls the API with appropriate auth"
# No mention of the key, its value, or how auth is handled

Secret Injection Patterns

There are several ways to get credentials into the execution layer without touching the agent's context. Each has different tradeoffs on security, operational complexity, and rotation support.

For production agentic systems, the vault SDK pattern is the standard. Yes, setting up a secrets manager takes time. But the security guarantees it provides, especially auto-rotation, eliminate the most common source of long-lived credential compromise. The operational overhead pays for itself the first time you avoid an incident.

Short-Lived Credentials and Auto-Rotation

One of the most valuable features of a well-configured secrets management system is automatic credential rotation. Instead of managing long-lived API keys that remain valid until manually revoked, short-lived credentials are generated on demand and expire automatically. AWS IAM roles, Vault dynamic secrets, and Google service account key rotation all support this. And the benefit is real: a leaked credential that expires in an hour is a much smaller incident than one that never expires.

# Dynamic credentials via Vault, generated per-agent, auto-expire
import hvac
from datetime import datetime, timedelta

class DynamicCredentialProvider:
    def __init__(self, vault_addr: str, vault_token: str):
        self.vault = hvac.Client(url=vault_addr, token=vault_token)

    def get_database_credentials(self, role: str = "research-agent") -> dict:
        """
        Generate short-lived DB credentials for this agent run.
        Vault creates a dedicated DB user that auto-expires after TTL.
        """
        response = self.vault.secrets.database.generate_credentials(
            name=role,
            mount_point="database",
        )
        creds = response["data"]
        # username and password are unique to this execution
        # they expire automatically after the TTL (e.g. 1 hour)
        # even if leaked, they stop working at expiry
        return {
            "username": creds["username"],
            "password": creds["password"],
            "expires_at": datetime.utcnow() + timedelta(hours=1),
        }

    def get_api_token(self, service: str) -> str:
        """Fetch a rotating API token, vault handles refresh."""
        secret = self.vault.secrets.kv.v2.read_secret_version(
            path=f"agents/{service}/api-token",
            mount_point="secret",
        )
        return secret["data"]["data"]["token"]

Preventing Context Window Exposure

Even with a vault architecture, credentials can still leak into the context window if your execution layer isn't careful. The most common way this happens is through tool result formatting. If a tool's result contains an Authorization header, a connection string, or an API response that echoes back authentication parameters, those values end up in the model's input on the next step.

import re
from typing import Any

# Patterns that should never appear in agent context
SENSITIVE_PATTERNS = [
    re.compile(r"(?i)(api[_-]?key|apikey)s*[:=]s*['"]?[a-z0-9_\-]{20,}"),
    re.compile(r"(?i)(password|passwd|pwd)s*[:=]s*['"]?\S{8,}"),
    re.compile(r"(?i)bearer\s+[a-zA-Z0-9_\-\.]{20,}"),
    re.compile(r"postgres(?:ql)?://[^@]+:[^@]+@"),
    re.compile(r"sk-(?:live|test)-[a-zA-Z0-9]{20,}"),
]

def redact_sensitive_values(text: str) -> str:
    """Sanitize tool results before they enter the agent's context."""
    for pattern in SENSITIVE_PATTERNS:
        text = pattern.sub("[REDACTED]", text)
    return text

def safe_tool_result(raw_result: Any) -> str:
    """
    Convert a tool result to a string safe for the agent's context.
    Always run before adding tool results to the message history.
    """
    result_text = json.dumps(raw_result) if not isinstance(raw_result, str) else raw_result
    return redact_sensitive_values(result_text)

# Usage: always sanitize before adding to context
tool_result = await execute_tool(tool_name, tool_params)
safe_result = safe_tool_result(tool_result)
messages.append({"role": "tool", "content": safe_result})

Logging Safely Around Secrets

Your observability system is one of the most common places credentials accidentally end up. Every tool call parameter that gets logged, every HTTP request traced with full headers, every context window snapshot saved for debugging is a potential credential exposure. Getting this right isn't optional.

• Never log raw HTTP request headers. Log the request URL and method. Log whether the Authorization header was present, not its value.
• Apply redaction middleware to all structured logs before they leave your process. Pattern matching against known secret formats as a defense-in-depth measure.
• Treat LLM context window snapshots as sensitive. They should be encrypted at rest and access-controlled as tightly as your most sensitive business data.
• Audit access to your observability system. If your logs contain redacted credentials, the observability system is itself a target.

Conclusion

Secrets management in agent runtimes is harder than in traditional services because the attack surface includes the model's context window, not just environment variables and source code. The architecture that handles this correctly has one rule: credential values never appear in any text the agent processes. All credential access happens below the agent layer, through a vault that enforces access control, logs every fetch, and rotates values automatically.

This costs some operational setup but eliminates an entire class of vulnerabilities that would otherwise be present in every agent you ship. That's a trade worth making every time.