xml-pipeline/docs/wiki/Writing-Handlers.md

Writing Handlers

Handlers are async Python functions that process messages. This guide covers everything you need to know to write effective handlers.

Basic Handler Structure

Every handler follows this pattern:

from dataclasses import dataclass
from third_party.xmlable import xmlify
from xml_pipeline.message_bus.message_state import HandlerMetadata, HandlerResponse

# 1. Define your payload (input)
@xmlify
@dataclass
class MyPayload:
    """Description of what this payload represents."""
    field1: str
    field2: int

# 2. Define your response (output)
@xmlify
@dataclass
class MyResponse:
    """Description of the response."""
    result: str

# 3. Write the handler
async def my_handler(payload: MyPayload, metadata: HandlerMetadata) -> HandlerResponse:
    """Process the payload and return a response."""
    result = f"Processed {payload.field1} with {payload.field2}"

    return HandlerResponse(
        payload=MyResponse(result=result),
        to="next-listener",  # Where to send the response
    )

The @xmlify Decorator

The @xmlify decorator enables automatic XML serialization and XSD generation:

from dataclasses import dataclass
from third_party.xmlable import xmlify

@xmlify
@dataclass
class Greeting:
    name: str                    # Required field
    formal: bool = False         # Optional with default
    count: int = 1               # Optional with default

This generates XML like:

<greeting>
  <name>Alice</name>
  <formal>true</formal>
  <count>3</count>
</greeting>

Supported Types

Python Type	XML Representation
str	Text content
int	Integer text
float	Decimal text
bool	true / false
list[T]	Repeated elements
Optional[T]	Optional element
@xmlify class	Nested element

Nested Payloads

@xmlify
@dataclass
class Address:
    street: str
    city: str

@xmlify
@dataclass
class Person:
    name: str
    address: Address  # Nested payload

HandlerMetadata

The metadata parameter provides context about the message:

@dataclass
class HandlerMetadata:
    thread_id: str              # Opaque thread UUID
    from_id: str                # Who sent this message
    own_name: str | None        # This listener's name (agents only)
    is_self_call: bool          # True if message is from self
    usage_instructions: str     # Peer schemas for LLM prompts
    todo_nudge: str             # System reminders about pending todos

Usage Examples

async def my_handler(payload: MyPayload, metadata: HandlerMetadata) -> HandlerResponse:
    # Log who sent the message
    print(f"Received from: {metadata.from_id}")

    # Check if this is a self-call (agent iterating)
    if metadata.is_self_call:
        print("This is a self-call")

    # For agents: use peer schemas in LLM prompts
    if metadata.usage_instructions:
        system_prompt = metadata.usage_instructions + "\n\nYour custom instructions..."

Response Types

Forward to Target

Send the response to a specific listener:

return HandlerResponse(
    payload=MyResponse(result="done"),
    to="next-listener",
)

Respond to Caller

Return to whoever sent the message:

return HandlerResponse.respond(
    payload=ResultPayload(value=42)
)

This uses the thread registry to route back up the call chain.

Terminate Chain

End processing with no response:

return None

Use this for terminal handlers (logging, display, etc.).

Handler Patterns

Simple Tool

A stateless transformation:

@xmlify
@dataclass
class AddPayload:
    a: int
    b: int

@xmlify
@dataclass
class AddResult:
    sum: int

async def add_handler(payload: AddPayload, metadata: HandlerMetadata) -> HandlerResponse:
    result = payload.a + payload.b
    return HandlerResponse.respond(payload=AddResult(sum=result))

LLM Agent

An agent that uses language models:

async def research_handler(payload: ResearchQuery, metadata: HandlerMetadata) -> HandlerResponse:
    from xml_pipeline.platform.llm_api import complete

    # Build prompt with peer awareness
    system_prompt = f"""
{metadata.usage_instructions}

You are a research agent. Answer the query using available tools.
"""

    response = await complete(
        model="grok-4.1",
        messages=[
            {"role": "system", "content": system_prompt},
            {"role": "user", "content": payload.query},
        ],
    )

    return HandlerResponse(
        payload=ResearchResult(answer=response.content),
        to="summarizer",
    )

Terminal Handler

A handler that displays output and ends the chain:

async def console_output(payload: TextOutput, metadata: HandlerMetadata) -> None:
    print(f"[{payload.source}] {payload.text}")
    return None  # Chain ends here

Self-Iterating Agent

An agent that calls itself to continue reasoning:

async def thinking_agent(payload: ThinkPayload, metadata: HandlerMetadata) -> HandlerResponse:
    # Check if we should continue thinking
    if payload.iteration >= 5:
        return HandlerResponse(
            payload=FinalAnswer(answer=payload.current_answer),
            to="output",
        )

    # Continue thinking by calling self
    return HandlerResponse(
        payload=ThinkPayload(
            iteration=payload.iteration + 1,
            current_answer=f"Refined: {payload.current_answer}",
        ),
        to=metadata.own_name,  # Self-call
    )

Error Handling

Handlers should handle errors gracefully:

async def safe_handler(payload: MyPayload, metadata: HandlerMetadata) -> HandlerResponse:
    try:
        result = await risky_operation(payload)
        return HandlerResponse.respond(payload=SuccessResult(data=result))
    except ValidationError as e:
        return HandlerResponse.respond(payload=ErrorResult(error=str(e)))
    except Exception as e:
        # Log and return generic error
        logger.exception("Handler failed")
        return HandlerResponse.respond(payload=ErrorResult(error="Internal error"))

Registration

Register handlers in organism.yaml:

listeners:
  - name: calculator.add
    payload_class: handlers.calc.AddPayload
    handler: handlers.calc.add_handler
    description: "Adds two numbers and returns the sum"

The description is important—it's used in auto-generated tool prompts for LLM agents.

CPU-Bound Handlers

For computationally expensive handlers, mark them as cpu_bound:

listeners:
  - name: analyzer
    payload_class: handlers.analyze.AnalyzePayload
    handler: handlers.analyze.analyze_handler
    description: "Heavy document analysis"
    cpu_bound: true

These run in a separate process pool to avoid blocking the event loop.

Security Notes

Handlers are untrusted code. The system enforces:

1. Identity injection — <from> is always set by the pump, never by handlers
2. Thread isolation — Handlers see only opaque UUIDs
3. Peer constraints — Agents can only send to declared peers

Even compromised handlers cannot:

• Forge sender identity
• Access other threads
• Discover organism topology
• Route to undeclared peers

See Also

• Handler Contract — Complete handler specification
• Configuration — Registering handlers
• LLM Router — Using language models