xml-pipeline/docs/configuration.md

AgentServer v2.1 — Organism Configuration

This file is the canonical reference for organism.yaml format in v2.1.
The old configuration.md is hereby obsolete and superseded.

The entire organism is declared in a single YAML file (default: config/organism.yaml).
It is the single source of truth for initial composition, loaded at bootstrap.
Runtime structural changes (add/remove/replace listeners) are performed exclusively via privileged OOB commands (hot-reload).

Full Example (ResearchSwarm-01)

organism:
  name: "ResearchSwarm-01"
  identity: "config/identity/private.ed25519"   # Ed25519 private key path
  port: 8765                                    # Main WSS message bus
  tls:
    cert: "certs/fullchain.pem"
    key: "certs/privkey.pem"

oob:                                            # Out-of-band privileged channel
  enabled: true
  bind: "127.0.0.1"                             # Localhost-only by default (GUI safe)
  port: 8766                                    # Separate WSS port from main bus
  # unix_socket: "/tmp/organism.sock"           # Alternative binding

thread_scheduling: "breadth-first"              # or "depth-first"

meta:
  enabled: true
  allow_list_capabilities: true
  allow_schema_requests: "admin"                # "admin" | "authenticated" | "none"
  allow_example_requests: "admin"
  allow_prompt_requests: "admin"
  allow_remote: false                           # Federation peers may query meta

listeners:
  - name: calculator.add
    payload_class: examples.calculator.AddPayload
    handler: examples.calculator.add_handler
    description: "Adds two integers and returns their sum."

  - name: calculator.multiply
    payload_class: examples.calculator.MultiplyPayload
    handler: examples.calculator.multiply_handler
    description: "Multiplies two integers and returns their product."

  - name: local_summarizer
    payload_class: agents.summarizer.SummarizePayload
    handler: agents.summarizer.summarize_handler
    description: "Summarizes text via local LLM."

  - name: researcher
    payload_class: agents.researcher.ResearchPayload
    handler: agents.researcher.research_handler
    description: "Primary research agent that reasons and coordinates tools."
    agent: true                                 # LLM agent → unique root tag, own_name exposed
    peers:                                      # Allowed call targets
      - calculator.add
      - calculator.multiply
      - local_summarizer
      - web_search                              # gateway group, defined below

  - name: search.google
    payload_class: gateways.google.SearchPayload
    handler: gateways.google.search_handler
    description: "Google search gateway."
    broadcast: true                             # Shares root tag with other search.* listeners

  - name: search.bing
    payload_class: gateways.google.SearchPayload      # Identical dataclass required
    handler: gateways.bing.search_handler
    description: "Bing search gateway."
    broadcast: true

gateways:
  - name: web_search
    remote_url: "wss://trusted-search-node.example.org"
    trusted_identity: "pubkeys/search_node.ed25519.pub"
    description: "Federated web search gateway group."

llm:
  strategy: failover                      # failover | round-robin | least-loaded
  retries: 3                              # Max retry attempts per request
  retry_base_delay: 1.0                   # Base delay for exponential backoff
  retry_max_delay: 60.0                   # Maximum delay between retries

  backends:
    - provider: xai
      api_key_env: XAI_API_KEY            # Read from environment
      priority: 1                         # Lower = preferred for failover
      rate_limit_tpm: 100000              # Tokens per minute
      max_concurrent: 20                  # Max concurrent requests

    - provider: anthropic
      api_key_env: ANTHROPIC_API_KEY
      priority: 2

    - provider: ollama
      base_url: http://localhost:11434
      supported_models: [llama3, mistral]

Sections Explained

organism

Core identity and main bus.

• name: Human identifier, used in logs and discovery.
• identity: Path to Ed25519 private key (signing, federation, OOB auth).
• port / tls: Main encrypted message bus.

oob

Privileged local control channel (GUI/hot-reload ready).

• Disabled → fully static configuration (restart required for changes).
• Bound to localhost by default for security.

thread_scheduling

Subthread execution policy across the organism.

• "breadth-first" (default): fair round-robin, prevents deep branch starvation.
• "depth-first": aggressive dive into branches.

meta

Introspection controls (https://xml-pipeline.org/ns/meta/v1 namespace).

• Flags control who may request capability lists, schemas, examples, prompts.

listeners

All bounded capabilities (tools and agents).

• name: Unique registered name (dots allowed for hierarchy). Becomes prefix of derived root tag.
• payload_class: Full import path to @xmlify dataclass.
• handler: Full import path to async handler function.
• description: Mandatory short blurb — leads auto-generated tool prompts.
• agent: true: Designates LLM-driven listener → enforces unique root tag, exposes own_name in HandlerMetadata.
• peers:: List of registered names (or gateway groups) this listener is allowed to address. Enforced by pump for agents.
• broadcast: true: Opt-in flag allowing multiple listeners to share the exact same derived root tag (used for parallel gateways).

gateways

Federation peers (trusted remote organisms).

• Declared separately for clarity.
• Referenced in agent peers: lists by their registered name.

llm

LLM router configuration for agents. See llm-router-v2.1.md for complete specification.

• strategy: Backend selection strategy.
  • failover (default): Try backends in priority order, fail over on error.
  • round-robin: Distribute requests evenly across backends.
  • least-loaded: Route to backend with lowest current load.
• retries: Max retry attempts per request.
• backends: List of provider configurations.
  • provider: Provider type (xai, anthropic, openai, ollama).
  • api_key_env: Environment variable name containing the API key.
  • priority: Lower = preferred (for failover strategy).
  • rate_limit_tpm: Tokens per minute limit.
  • max_concurrent: Max concurrent requests to this backend.
  • base_url: Override default API endpoint (required for Ollama).
  • supported_models: Model names this backend handles (Ollama only).

Environment Variables (.env)

API keys and secrets should never be stored in YAML. Use environment variables instead.

The bootstrap process automatically loads .env from the project root via python-dotenv:

# .env (add to .gitignore!)
XAI_API_KEY=xai-abc123...
ANTHROPIC_API_KEY=sk-ant-...
OPENAI_API_KEY=sk-...

Reference in organism.yaml via api_key_env:

llm:
  backends:
    - provider: xai
      api_key_env: XAI_API_KEY    # Reads from environment

Key Invariants (v2.1)

• Root tag = {lowercase_name}.{lowercase_dataclass_name} — fully derived, never written manually.
• Registered names must be unique across the organism.
• Normal listeners have globally unique root tags.
• Broadcast listeners may share root tags intentionally (same dataclass required).
• Agents always have unique root tags (enforced automatically).
• All structural changes after bootstrap require privileged OOB hot-reload.

This YAML is the organism’s DNA — precise, auditable, minimal, and fully aligned with listener-class-v2.1.md.