xml-pipeline/docs/wiki/reference/Configuration.md
dullfig 515c738abb Add wiki documentation for xml-pipeline.org
Comprehensive documentation set for XWiki:
- Home, Installation, Quick Start guides
- Writing Handlers and LLM Router guides
- Architecture docs (Overview, Message Pump, Thread Registry, Shared Backend)
- Reference docs (Configuration, Handler Contract, CLI)
- Hello World tutorial
- Why XML rationale
- Pandoc conversion scripts (bash + PowerShell)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-20 20:40:47 -08:00

6.7 KiB

Configuration Reference

Organisms are configured via YAML files. The default location is config/organism.yaml.

Minimal Configuration

organism:
  name: my-organism

listeners:
  - name: greeter
    payload_class: handlers.hello.Greeting
    handler: handlers.hello.handle_greeting
    description: Greeting handler

Full Configuration Reference

# ============================================================
# ORGANISM SECTION
# Core identity and network settings
# ============================================================
organism:
  name: "my-organism"              # Human-readable name (required)
  port: 8765                       # WebSocket port (optional)
  identity: "config/identity.key"  # Ed25519 private key path (optional)
  tls:                             # TLS settings (optional)
    cert: "certs/fullchain.pem"
    key: "certs/privkey.pem"

# ============================================================
# LLM SECTION
# Language model routing configuration
# ============================================================
llm:
  strategy: failover               # failover | round-robin | least-loaded
  retries: 3                       # Max retry attempts
  retry_base_delay: 1.0            # Base delay for exponential backoff
  retry_max_delay: 60.0            # Maximum delay between retries

  backends:
    - provider: xai                # xai | anthropic | openai | ollama
      api_key_env: XAI_API_KEY     # Environment variable name
      priority: 1                  # Lower = preferred (for failover)
      rate_limit_tpm: 100000       # Tokens per minute limit
      max_concurrent: 20           # Max concurrent requests

    - provider: anthropic
      api_key_env: ANTHROPIC_API_KEY
      priority: 2

    - provider: ollama
      base_url: http://localhost:11434
      supported_models: [llama3, mistral]

# ============================================================
# BACKEND SECTION (Optional)
# Shared state for multiprocess deployments
# ============================================================
backend:
  type: memory                     # memory | manager | redis
  # Redis-specific settings (when type: redis)
  redis_url: "redis://localhost:6379"
  redis_prefix: "xp:"              # Key prefix for multi-tenancy
  redis_ttl: 86400                 # TTL in seconds (24 hours)

# ============================================================
# PROCESS POOL SECTION (Optional)
# Worker processes for CPU-bound handlers
# ============================================================
process_pool:
  workers: 4                       # Number of worker processes
  max_tasks_per_child: 100         # Restart workers after N tasks

# ============================================================
# LISTENERS SECTION
# Message handlers (tools and agents)
# ============================================================
listeners:
  # Simple tool (non-agent)
  - name: calculator.add
    payload_class: handlers.calc.AddPayload
    handler: handlers.calc.add_handler
    description: "Adds two numbers"

  # LLM Agent
  - name: researcher
    payload_class: handlers.research.ResearchQuery
    handler: handlers.research.research_handler
    description: "Research agent that searches and synthesizes"
    agent: true                    # Marks as LLM agent
    peers:                         # Allowed call targets
      - calculator.add
      - web_search
    prompt: |                      # System prompt for LLM
      You are a research assistant.
      Use tools to find information.

  # CPU-bound handler (runs in process pool)
  - name: librarian
    payload_class: handlers.librarian.Query
    handler: handlers.librarian.handle_query
    description: "Document analysis with heavy computation"
    cpu_bound: true                # Dispatch to ProcessPoolExecutor

# ============================================================
# GATEWAYS SECTION (Optional)
# Federation with remote organisms
# ============================================================
gateways:
  - name: remote_search
    remote_url: "wss://search.example.org"
    trusted_identity: "keys/search_node.pub"
    description: "Federated search gateway"

Section Details

organism

Field Type Required Description
name string Yes Human-readable organism name
port int No WebSocket server port
identity path No Ed25519 private key for signing
tls.cert path No TLS certificate path
tls.key path No TLS private key path

llm.backends[]

Field Type Required Description
provider string Yes xai, anthropic, openai, ollama
api_key_env string Depends Env var containing API key
base_url string No Override API endpoint
priority int No Lower = preferred (default: 1)
rate_limit_tpm int No Tokens per minute limit
max_concurrent int No Max concurrent requests
supported_models list No Models this backend serves (ollama)

listeners[]

Field Type Required Description
name string Yes Unique listener name
payload_class string Yes Import path to @xmlify dataclass
handler string Yes Import path to handler function
description string Yes Short description (used in prompts)
agent bool No Is this an LLM agent? (default: false)
peers list No Allowed call targets for agents
prompt string No System prompt for LLM agents
cpu_bound bool No Run in ProcessPoolExecutor (default: false)
broadcast bool No Allow shared root tag (default: false)

backend

Field Type Required Description
type string No memory, manager, redis (default: memory)
redis_url string If redis Redis connection URL
redis_prefix string No Key prefix (default: xp:)
redis_ttl int No Key TTL in seconds

process_pool

Field Type Required Description
workers int No Number of worker processes (default: CPU count)
max_tasks_per_child int No Tasks before worker restart

Environment Variables

API keys should be stored in environment variables, referenced via api_key_env:

# .env file
XAI_API_KEY=xai-abc123...
ANTHROPIC_API_KEY=sk-ant-...
OPENAI_API_KEY=sk-...

Validation

Validate your configuration without running:

xml-pipeline check config/organism.yaml

See Also