Update README and pyproject.toml for PyPI release
README.md: - Rebrand from AgentServer to xml-pipeline - Library-focused introduction with pip install - Quick start guide with code examples - Console example documentation - Concise feature overview pyproject.toml: - Update authors to "xml-pipeline contributors" - Update URLs to xml-pipeline.org Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
dullfig
parent
3cae862d7a
commit
0553ec294f
2 changed files with 181 additions and 114 deletions
286
README.md
286
README.md
|
|
@ -1,152 +1,220 @@
|
||||||
# AgentServer — The Living Substrate (v2.1)
|
# xml-pipeline
|
||||||
***"It just works... safely."***
|
|
||||||
|
|
||||||
**January 06, 2026**
|
**Schema-driven XML message bus for multi-agent systems.**
|
||||||
**Architecture: Autonomous Schema-Driven, Turing-Complete Multi-Agent Organism**
|
|
||||||
|
|
||||||
## The Rant
|
[](https://www.python.org/downloads/)
|
||||||
**Why XML?**
|
[](LICENSE)
|
||||||
[Why not JSON?](docs/why-not-json.md)
|
|
||||||
|
|
||||||
XML is the sovereign wire format — standards-based, self-describing, attack-resistant, and evolvable without drift. JSON was a quick hack that escaped into the wild and became the default for everything, including AI tool calling, where its brittleness causes endless prompt surgery and validation headaches.
|
xml-pipeline is a Python library for building multi-agent systems with validated XML message passing. Agents communicate through typed payloads, validated against auto-generated XSD schemas, with built-in LLM routing and conversation memory.
|
||||||
|
|
||||||
This project chooses XML deliberately. The organism enforces contracts exactly (XSD validation, no transcription bugs), tolerates dirty streams (repair + dummy extraction), and keeps reasoning visible. No fragile conventions. No escaping hell. Just bounded, auditable computation.
|
## Why XML?
|
||||||
|
|
||||||
Read the full rant [here](docs/why-not-json.md) for the history, pitfalls, and why XML wins permanently.
|
JSON was a quick hack that became the default for AI tool calling, where its brittleness causes endless prompt surgery and validation headaches. xml-pipeline chooses XML deliberately:
|
||||||
|
|
||||||
## What It Is
|
- **Exact contracts** — XSD validation catches malformed messages before they cause problems
|
||||||
AgentServer is a production-ready substrate for the `xml-pipeline` nervous system. Version 2.1 evolves the design around parallel per-listener pipelines, true concurrent broadcast, opaque UUID threading for privacy, and blind agent self-iteration—all while preserving strict validation and handler purity.
|
- **Tolerant parsing** — Repair mode recovers from LLM output quirks
|
||||||
|
- **Self-describing** — Namespaces prevent collision, schemas are discoverable
|
||||||
|
- **No escaping hell** — Mixed content, nested structures, all handled cleanly
|
||||||
|
|
||||||
See [Core Architectural Principles](docs/core-principles-v2.1.md) for the single canonical source of truth.
|
Read the [full rationale](docs/why-not-json.md).
|
||||||
|
|
||||||
## Core Philosophy
|
## Installation
|
||||||
- **Autonomous DNA:** Listeners declare their contract via `@xmlify` dataclasses; the organism auto-generates XSDs, examples, and tool prompts.
|
|
||||||
- **Schema-Locked Intelligence:** Payloads validated directly against XSD (lxml) → deserialized to typed instances → pure handlers.
|
|
||||||
- **Multi-Response Tolerance:** Handlers return `HandlerResponse` dataclasses; bus extracts payloads and routes them (perfect for parallel tool calls or multi-step workflows).
|
|
||||||
- **Computational Sovereignty:** Turing-complete via blind self-calls, subthreading primitives, concurrent broadcast, and visible reasoning — all bounded by private thread hierarchy and local-only control.
|
|
||||||
|
|
||||||
## Developer Experience — Create a Listener in 12 Lines
|
```bash
|
||||||
**No manual schemas. No brittle JSON conventions. No hand-written prompts.**
|
pip install xml-pipeline
|
||||||
Just declare a dataclass contract and a one-line human description. The organism handles validation, XSD, examples, and tool prompts automatically.
|
|
||||||
|
# With LLM provider support
|
||||||
|
pip install xml-pipeline[anthropic] # Anthropic Claude
|
||||||
|
pip install xml-pipeline[openai] # OpenAI GPT
|
||||||
|
|
||||||
|
# With all features
|
||||||
|
pip install xml-pipeline[all]
|
||||||
|
```
|
||||||
|
|
||||||
|
## Quick Start
|
||||||
|
|
||||||
|
### 1. Define a payload
|
||||||
|
|
||||||
```python
|
```python
|
||||||
from dataclasses import dataclass
|
from dataclasses import dataclass
|
||||||
from third_party.xmlable import xmlify
|
from third_party.xmlable import xmlify
|
||||||
|
|
||||||
|
@xmlify
|
||||||
|
@dataclass
|
||||||
|
class Greeting:
|
||||||
|
name: str
|
||||||
|
```
|
||||||
|
|
||||||
|
### 2. Write a handler
|
||||||
|
|
||||||
|
```python
|
||||||
from xml_pipeline.message_bus.message_state import HandlerMetadata, HandlerResponse
|
from xml_pipeline.message_bus.message_state import HandlerMetadata, HandlerResponse
|
||||||
|
|
||||||
@xmlify
|
@xmlify
|
||||||
@dataclass
|
@dataclass
|
||||||
class AddPayload:
|
class GreetingReply:
|
||||||
a: int
|
message: str
|
||||||
b: int
|
|
||||||
|
|
||||||
@xmlify
|
async def handle_greeting(payload: Greeting, metadata: HandlerMetadata) -> HandlerResponse:
|
||||||
@dataclass
|
return HandlerResponse(
|
||||||
class ResultPayload:
|
payload=GreetingReply(message=f"Hello, {payload.name}!"),
|
||||||
value: int
|
to="output",
|
||||||
|
)
|
||||||
async def add_handler(payload: AddPayload, metadata: HandlerMetadata) -> HandlerResponse:
|
|
||||||
"""Handlers MUST be async and return HandlerResponse."""
|
|
||||||
result = payload.a + payload.b
|
|
||||||
return HandlerResponse.respond(payload=ResultPayload(value=result))
|
|
||||||
|
|
||||||
# In organism.yaml:
|
|
||||||
# listeners:
|
|
||||||
# - name: calculator.add
|
|
||||||
# payload_class: mymodule.AddPayload
|
|
||||||
# handler: mymodule.add_handler
|
|
||||||
# description: "Adds two integers and returns their sum."
|
|
||||||
```
|
```
|
||||||
|
|
||||||
The organism now speaks `<add>` — fully validated, typed, and discoverable.<br/>
|
### 3. Configure the organism
|
||||||
Unlike rigid platforms requiring custom mappings or fragile item structures, this is pure Python — typed, testable, and sovereign.
|
|
||||||
|
|
||||||
## Security Model
|
```yaml
|
||||||
|
# organism.yaml
|
||||||
|
organism:
|
||||||
|
name: hello-world
|
||||||
|
|
||||||
AgentServer's security is **architectural**, not bolted-on:
|
listeners:
|
||||||
|
- name: greeter
|
||||||
|
payload_class: myapp.Greeting
|
||||||
|
handler: myapp.handle_greeting
|
||||||
|
description: Greets users by name
|
||||||
|
|
||||||
### Two Completely Isolated Channels
|
- name: output
|
||||||
- **Main Bus**: Standard `<message>` envelope, all traffic undergoes identical validation pipeline regardless of source
|
payload_class: myapp.GreetingReply
|
||||||
- **OOB Channel**: Privileged commands only, different schema, localhost-bound, used for structural changes
|
handler: myapp.print_output
|
||||||
|
description: Prints output
|
||||||
|
```
|
||||||
|
|
||||||
### Handler Isolation & Trust Boundary
|
### 4. Run it
|
||||||
**Handlers are untrusted code.** Even compromised handlers cannot:
|
|
||||||
- Forge their identity (sender name captured in coroutine scope before execution)
|
|
||||||
- Escape thread context (thread UUID captured in coroutine, not handler output)
|
|
||||||
- Route to arbitrary targets (routing computed from peers list, not handler claims)
|
|
||||||
- Access other threads' data (opaque UUIDs, private path registry)
|
|
||||||
- Discover topology (only declared peers visible)
|
|
||||||
|
|
||||||
The message pump maintains authoritative metadata in coroutine scope and **never trusts handler output** for security-critical properties.
|
```python
|
||||||
|
import asyncio
|
||||||
|
from xml_pipeline.message_bus import bootstrap
|
||||||
|
|
||||||
### Closed-Loop Validation
|
async def main():
|
||||||
ALL messages on the main bus undergo identical security processing:
|
pump = await bootstrap("organism.yaml")
|
||||||
- External ingress: WSS → pipeline → validation
|
await pump.run()
|
||||||
- Handler outputs: bytes → pipeline → validation (same steps!)
|
|
||||||
- Error messages: generated → pipeline → validation
|
|
||||||
- System notifications: generated → pipeline → validation
|
|
||||||
|
|
||||||
No fast-path bypasses. No "trusted internal" messages. Everything validates.
|
asyncio.run(main())
|
||||||
|
```
|
||||||
|
|
||||||
### Topology Privacy
|
## Console Example
|
||||||
- Agents see only opaque thread UUIDs, never hierarchical paths
|
|
||||||
- Private path registry (UUID → `agent.tool.subtool`) maintained by system
|
|
||||||
- Peers list enforces capability boundaries (no ambient authority)
|
|
||||||
- Federation gateways are opaque abstractions
|
|
||||||
|
|
||||||
### Anti-Paperclip Architecture
|
Try the interactive console example:
|
||||||
- Threads are ephemeral (complete audit trail, then deleted)
|
|
||||||
- No persistent cross-thread memory primitives
|
|
||||||
- Token budgets enforce computational bounds
|
|
||||||
- Thread pruning prevents state accumulation
|
|
||||||
- All reasoning visible in message history
|
|
||||||
|
|
||||||
This architecture ensures:<br>
|
```bash
|
||||||
✅ No privilege escalation (handlers can't forge privileged commands)<br>
|
pip install xml-pipeline[console]
|
||||||
✅ No fast-path bypasses (even system-generated messages validate)<br>
|
python -m examples.console
|
||||||
✅ Physical separation (privileged and regular traffic cannot mix)<br>
|
```
|
||||||
✅ Capability-safe handlers (compromised code still bounded by peers list)<br>
|
|
||||||
✅ Complete auditability (thread history is ground truth)
|
|
||||||
|
|
||||||
|
```
|
||||||
|
> @greeter Alice
|
||||||
|
[greeter] Hello, Alice! Welcome to xml-pipeline.
|
||||||
|
|
||||||
|
> @echo Hello world
|
||||||
|
[echo] Hello world
|
||||||
|
|
||||||
|
> /quit
|
||||||
|
```
|
||||||
|
|
||||||
|
See [examples/console/](examples/console/) for the full source.
|
||||||
|
|
||||||
## Key Features
|
## Key Features
|
||||||
### 1. The Autonomous Schema Layer
|
|
||||||
- Dataclass → cached XSD + example + rich tool prompt (mandatory description + field docs).
|
|
||||||
- Namespaces: `https://xml-pipeline.org/ns/<category>/<name>/v1` (served live via domain for discoverability).
|
|
||||||
- Multiple listeners per root tag supported (broadcast parallelism).
|
|
||||||
|
|
||||||
### 2. Thread-Based Lifecycle & Reasoning
|
### Typed Message Passing
|
||||||
- Opaque `<thread/>` UUIDs with private hierarchical path registry for reliable subthreading, audit trails, and topology privacy.
|
|
||||||
- LLM agents use unique root tags for blind self-iteration (no name knowledge or `<to/>` needed).
|
|
||||||
- Agents reason via open self-calls, multi-payload parallelism, and optional `<todo-until/>` scaffolding in visible text.
|
|
||||||
- All thought steps visible as messages — no hidden state.
|
|
||||||
|
|
||||||
### 3. Message Pump
|
Payloads are Python dataclasses with automatic XSD generation:
|
||||||
- Parallel preprocessing pipelines (one per listener) with central async pump orchestration.
|
|
||||||
- True concurrency: pipeline tasks parallel, broadcast handlers via asyncio.gather.
|
|
||||||
- Single linear flow per pipeline with repair, C14N, XSD validation, deserialization, handler execution, and multi-payload extraction.
|
|
||||||
- Supports clean tools, forgiving LLM streams, and natural broadcast alike.
|
|
||||||
- Thread-based message queue with bounded memory and fair scheduling.
|
|
||||||
|
|
||||||
### 4. Structural Control
|
```python
|
||||||
- Bootstrap from `organism.yaml` (including unique root enforcement for agents).
|
@xmlify
|
||||||
- Runtime changes (hot-reload, add/remove listeners) via local-only OOB channel (localhost WSS or Unix socket — GUI-ready).
|
@dataclass
|
||||||
- Main bus oblivious to privileged ops.
|
class Calculate:
|
||||||
|
expression: str
|
||||||
|
precision: int = 2
|
||||||
|
```
|
||||||
|
|
||||||
### 5. Federation & Introspection
|
The library auto-generates:
|
||||||
- YAML-declared gateways with trusted keys.
|
- XSD schema for validation
|
||||||
- Controlled meta queries (schema/example/prompt/capability list).
|
- Example XML for documentation
|
||||||
|
- Usage instructions for LLM prompts
|
||||||
|
|
||||||
## Technical Stack
|
### LLM Router
|
||||||
- **Validation & Parsing:** lxml (XSD, C14N, repair) + xmlable (round-trip).
|
|
||||||
- **Protocol:** Mandatory WSS (TLS) + TOTP on main port.
|
|
||||||
- **Identity:** Ed25519 (signing, federation, privileged).
|
|
||||||
- **Format:** Exclusive C14N XML (wire sovereign).
|
|
||||||
|
|
||||||
## Why This Matters
|
Multi-backend LLM support with failover:
|
||||||
AgentServer v2.1 is a bounded, auditable, owner-controlled organism where the **XSD is the security**, the **private thread registry is the memory**, and the **OOB channel is the sovereignty**.
|
|
||||||
|
|
||||||
One port. Many bounded minds. Autonomous yet obedient evolution. 🚀
|
```yaml
|
||||||
|
llm:
|
||||||
|
strategy: failover
|
||||||
|
backends:
|
||||||
|
- provider: anthropic
|
||||||
|
api_key_env: ANTHROPIC_API_KEY
|
||||||
|
- provider: openai
|
||||||
|
api_key_env: OPENAI_API_KEY
|
||||||
|
```
|
||||||
|
|
||||||
|
```python
|
||||||
|
from xml_pipeline.llm import complete
|
||||||
|
|
||||||
|
response = await complete(
|
||||||
|
model="claude-sonnet-4",
|
||||||
|
messages=[{"role": "user", "content": "Hello!"}],
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||
|
### Handler Security
|
||||||
|
|
||||||
|
Handlers are sandboxed. They cannot:
|
||||||
|
- Forge sender identity (injected by pump)
|
||||||
|
- Escape thread context (managed by registry)
|
||||||
|
- Route to undeclared peers (validated against config)
|
||||||
|
- Access other threads (opaque UUIDs)
|
||||||
|
|
||||||
|
### Conversation Memory
|
||||||
|
|
||||||
|
Thread-scoped context buffer tracks message history:
|
||||||
|
|
||||||
|
```python
|
||||||
|
from xml_pipeline.memory import get_context_buffer
|
||||||
|
|
||||||
|
buffer = get_context_buffer()
|
||||||
|
history = buffer.get_thread(metadata.thread_id)
|
||||||
|
```
|
||||||
|
|
||||||
|
## Architecture
|
||||||
|
|
||||||
|
```
|
||||||
|
┌─────────────────────────────────────────────────────────────────┐
|
||||||
|
│ StreamPump │
|
||||||
|
│ • Parallel pipelines per listener │
|
||||||
|
│ • Repair → C14N → Validate → Deserialize → Route → Dispatch │
|
||||||
|
└─────────────────────────────────────────────────────────────────┘
|
||||||
|
↓
|
||||||
|
┌─────────────────────────────────────────────────────────────────┐
|
||||||
|
│ Handlers │
|
||||||
|
│ • Receive typed payload + metadata │
|
||||||
|
│ • Return HandlerResponse or None │
|
||||||
|
│ • Cannot forge identity or escape thread │
|
||||||
|
└─────────────────────────────────────────────────────────────────┘
|
||||||
|
```
|
||||||
|
|
||||||
|
See [docs/core-principles-v2.1.md](docs/core-principles-v2.1.md) for the full architecture.
|
||||||
|
|
||||||
|
## Documentation
|
||||||
|
|
||||||
|
| Document | Description |
|
||||||
|
|----------|-------------|
|
||||||
|
| [Core Principles](docs/core-principles-v2.1.md) | Architecture overview |
|
||||||
|
| [Handler Contract](docs/handler-contract-v2.1.md) | How to write handlers |
|
||||||
|
| [Message Pump](docs/message-pump-v2.1.md) | Pipeline processing |
|
||||||
|
| [LLM Router](docs/llm-router-v2.1.md) | Multi-backend LLM support |
|
||||||
|
| [Configuration](docs/configuration.md) | organism.yaml reference |
|
||||||
|
| [Why Not JSON?](docs/why-not-json.md) | Design rationale |
|
||||||
|
|
||||||
|
## Requirements
|
||||||
|
|
||||||
|
- Python 3.11+
|
||||||
|
- Dependencies: lxml, aiostream, pyyaml, httpx, cryptography
|
||||||
|
|
||||||
|
## License
|
||||||
|
|
||||||
|
MIT License. See [LICENSE](LICENSE).
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
*XML wins. Safely. Permanently.*
|
*XML wins. Safely. Permanently.*
|
||||||
|
|
|
||||||
|
|
@ -15,7 +15,7 @@ readme = "README.md"
|
||||||
requires-python = ">=3.11"
|
requires-python = ">=3.11"
|
||||||
license = {text = "MIT"}
|
license = {text = "MIT"}
|
||||||
authors = [
|
authors = [
|
||||||
{name = "Your Name", email = "you@example.com"},
|
{name = "xml-pipeline contributors"},
|
||||||
]
|
]
|
||||||
keywords = [
|
keywords = [
|
||||||
"xml",
|
"xml",
|
||||||
|
|
@ -115,11 +115,10 @@ xp = "xml_pipeline.cli:main" # Short alias
|
||||||
# PROJECT URLS
|
# PROJECT URLS
|
||||||
# =============================================================================
|
# =============================================================================
|
||||||
[project.urls]
|
[project.urls]
|
||||||
Homepage = "https://github.com/yourorg/xml-pipeline"
|
Homepage = "https://xml-pipeline.org"
|
||||||
Documentation = "https://xml-pipeline.org/docs"
|
Documentation = "https://xml-pipeline.org/docs"
|
||||||
Repository = "https://github.com/yourorg/xml-pipeline"
|
Repository = "https://git.xml-pipeline.org/dullfig/xml-pipeline"
|
||||||
Issues = "https://github.com/yourorg/xml-pipeline/issues"
|
Issues = "https://git.xml-pipeline.org/dullfig/xml-pipeline/issues"
|
||||||
Changelog = "https://github.com/yourorg/xml-pipeline/blob/main/CHANGELOG.md"
|
|
||||||
|
|
||||||
# =============================================================================
|
# =============================================================================
|
||||||
# PACKAGE DISCOVERY
|
# PACKAGE DISCOVERY
|
||||||
|
|
|
||||||
Loading…
Reference in a new issue