Environment Configuration for AI Agents: Managing Secrets, Models, and Feature Flags

Why Configuration Management Matters for AI Agents

AI agents have more configuration surface area than typical web services. Beyond the usual database URLs and API keys, you manage model names, temperature settings, system prompts, token limits, tool configurations, and feature flags that control agent behavior. Hardcoding any of these means rebuilding and redeploying for every change — unacceptable when you need to swap models, adjust prompts, or disable a misbehaving tool in minutes.

Good configuration management follows the twelve-factor app principle: store config in the environment, separate it from code, and make it easy to change without deployments.

Pydantic Settings for Typed Configuration

Pydantic Settings gives you validated, typed configuration that reads from environment variables automatically:

flowchart LR
    INPUT(["User intent"])
    PARSE["Parse plus<br/>classify"]
    PLAN["Plan and tool<br/>selection"]
    AGENT["Agent loop<br/>LLM plus tools"]
    GUARD{"Guardrails<br/>and policy"}
    EXEC["Execute and<br/>verify result"]
    OBS[("Trace and metrics")]
    OUT(["Outcome plus<br/>next action"])
    INPUT --> PARSE --> PLAN --> AGENT --> GUARD
    GUARD -->|Pass| EXEC --> OUT
    GUARD -->|Fail| AGENT
    AGENT --> OBS
    style AGENT fill:#4f46e5,stroke:#4338ca,color:#fff
    style GUARD fill:#f59e0b,stroke:#d97706,color:#1f2937
    style OBS fill:#ede9fe,stroke:#7c3aed,color:#1e1b4b
    style OUT fill:#059669,stroke:#047857,color:#fff

# app/config.py
from pydantic_settings import BaseSettings
from pydantic import Field, field_validator
from typing import Optional

class AgentSettings(BaseSettings):
    # API Keys — required, no defaults
    openai_api_key: str
    anthropic_api_key: Optional[str] = None

    # Model configuration
    default_model: str = "gpt-4o"
    fallback_model: str = "gpt-4o-mini"
    temperature: float = Field(0.7, ge=0.0, le=2.0)
    max_tokens: int = Field(4096, ge=1, le=128000)

    # Service configuration
    agent_timeout_seconds: int = 60
    max_retries: int = 3
    redis_url: str = "redis://localhost:6379/0"
    log_level: str = "info"

    # Feature flags
    enable_tool_use: bool = True
    enable_streaming: bool = True
    enable_memory: bool = False

    @field_validator("log_level")
    @classmethod
    def validate_log_level(cls, v):
        allowed = {"debug", "info", "warning", "error", "critical"}
        if v.lower() not in allowed:
            raise ValueError(f"log_level must be one of {allowed}")
        return v.lower()

    class Config:
        env_file = ".env"
        env_file_encoding = "utf-8"
        case_sensitive = False

settings = AgentSettings()

Every field maps to an environment variable automatically. default_model reads from DEFAULT_MODEL, openai_api_key reads from OPENAI_API_KEY. If a required field is missing, the application fails immediately at startup with a clear error.

The .env File for Local Development

Create a .env file for local development that never gets committed:

# .env (add to .gitignore)
OPENAI_API_KEY=sk-proj-local-dev-key
DEFAULT_MODEL=gpt-4o-mini
TEMPERATURE=0.5
LOG_LEVEL=debug
ENABLE_MEMORY=true
REDIS_URL=redis://localhost:6379/0

Add .env to your .gitignore and provide a .env.example template:

# .env.example (committed to repo)
OPENAI_API_KEY=your-key-here
DEFAULT_MODEL=gpt-4o
TEMPERATURE=0.7
LOG_LEVEL=info
REDIS_URL=redis://localhost:6379/0

Environment-Specific Configuration

Use different .env files per environment, loaded based on an APP_ENV variable:

# app/config.py
import os

class AgentSettings(BaseSettings):
    app_env: str = "development"

    class Config:
        env_file = f".env.{os.getenv('APP_ENV', 'development')}"

Now APP_ENV=staging loads .env.staging, and APP_ENV=production loads .env.production.

Secret Management in Production

Never store production secrets in .env files on disk. Use a dedicated secret manager:

# app/secrets.py
import boto3
import json
from functools import lru_cache

@lru_cache
def get_secret(secret_name: str, region: str = "us-east-1") -> dict:
    client = boto3.client("secretsmanager", region_name=region)
    response = client.get_secret_value(SecretId=secret_name)
    return json.loads(response["SecretString"])

# Usage at startup
def load_production_secrets():
    secrets = get_secret("ai-agent/production")
    os.environ["OPENAI_API_KEY"] = secrets["openai_api_key"]
    os.environ["ANTHROPIC_API_KEY"] = secrets["anthropic_api_key"]

Call load_production_secrets() before instantiating your Pydantic settings.

Runtime Feature Flags

Feature flags let you change agent behavior without redeploying. For simple cases, environment variables work. For dynamic flags that change at runtime, use Redis:

# app/feature_flags.py
import redis.asyncio as redis

class FeatureFlags:
    def __init__(self, redis_url: str):
        self.redis = redis.from_url(redis_url)
        self.prefix = "feature:"

    async def is_enabled(self, flag: str, default: bool = False) -> bool:
        val = await self.redis.get(f"{self.prefix}{flag}")
        if val is None:
            return default
        return val.decode() == "true"

    async def set_flag(self, flag: str, enabled: bool):
        await self.redis.set(f"{self.prefix}{flag}", str(enabled).lower())

# Usage in agent logic
flags = FeatureFlags(settings.redis_url)

async def run_agent(message: str):
    use_tools = await flags.is_enabled("tool_use", default=True)
    use_memory = await flags.is_enabled("agent_memory", default=False)

    agent = Agent(
        name="assistant",
        instructions="You are a helpful assistant.",
        tools=tool_list if use_tools else [],
    )
    # ...

Set flags without deployment:

redis-cli SET feature:tool_use false
redis-cli SET feature:agent_memory true

FAQ

How do I handle configuration changes that require an agent restart versus those that take effect immediately?

Classify your configuration into two tiers. Static config (API keys, model names, service URLs) is read once at startup — changes require a restart. Dynamic config (feature flags, temperature, max tokens) is read per-request from Redis or a config service. Design your agent to pull dynamic values before each invocation rather than caching them in instance variables.

Should I use a .env file or Kubernetes ConfigMaps for non-secret configuration?

Use both. During local development, .env files are the most convenient. In Kubernetes, ConfigMaps and Secrets are the standard approach — they integrate with pod lifecycle, support rolling updates, and can be managed by GitOps tools. Your Pydantic settings class reads from environment variables regardless of how they are injected.

How do I safely rotate API keys for a running agent service?

Store the new key alongside the old key in your secret manager. Update the environment variable in your deployment and perform a rolling restart. During the rollout, old pods use the old key and new pods use the new key — both work simultaneously. After all pods are updated, revoke the old key. Never rotate keys by editing a .env file on a running server.

---

#Configuration #AIAgents #SecretsManagement #FeatureFlags #Python #AgenticAI #LearnAI #AIEngineering