Three workstreams implemented:
W1 (Repo Split): Remove proprietary BloxServer files and docs, update
pyproject.toml URLs to public GitHub, clean doc references, add CI
workflow (.github/workflows/ci.yml) and CONTRIBUTING.md.
W2 (Message Journal): Add DispatchHook protocol for dispatch lifecycle
events, SQLite-backed MessageJournal with WAL mode for certified-mail
delivery guarantees (PENDING→DISPATCHED→ACKED/FAILED), integrate hooks
into StreamPump._dispatch_to_handlers(), add journal REST endpoints,
and aiosqlite dependency.
W3 (Hot Deployment): Add RestartOrchestrator for graceful restart with
queue drain and journal stats collection, SIGHUP signal handler in CLI,
POST /organism/restart endpoint, restart-aware app lifespan with journal
recovery on boot, and os.execv/subprocess re-exec for Unix/Windows.
All 439 tests pass (37 new tests for W2/W3).
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- UsageStore with async SQLite persistence via aiosqlite
- Background batch writer for non-blocking event persistence
- Auto-subscribes to UsageTracker for transparent capture
- Query methods: query(), get_billing_summary(), get_daily_usage()
- REST API endpoints: /usage/history, /usage/billing, /usage/daily
- Filtering by org_id, agent_id, model, time range
- 18 new tests for persistence layer
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Create BudgetWarning primitive payload (75%, 90%, 95% thresholds)
- Add threshold tracking to ThreadBudget with triggered_thresholds set
- Change consume() to return (budget, crossed_thresholds) tuple
- Wire warning injection in LLM router when thresholds crossed
- Add 15 new tests for threshold detection and warning injection
Agents now receive BudgetWarning messages when approaching their token limit,
allowing them to design contingencies (summarize, escalate, save state).
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
When threads terminate (handler returns None or chain exhausted),
the pump now calls budget_registry.cleanup_thread() to:
- Free memory for completed threads
- Return final budget for logging/billing
- Log token usage at debug level
This ensures budgets don't accumulate for completed conversations.
Also adds:
- has_budget() method to check if thread exists without creating
- Tests for cleanup behavior
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Endpoints:
- GET /api/v1/usage - Overview with totals, per-agent, per-model breakdown
- GET /api/v1/usage/threads - List all thread budgets sorted by usage
- GET /api/v1/usage/threads/{id} - Single thread budget details
- GET /api/v1/usage/agents/{id} - Usage totals for specific agent
- GET /api/v1/usage/models/{model} - Usage totals for specific model
- POST /api/v1/usage/reset - Reset all usage tracking
Models:
- UsageTotals, UsageOverview, UsageResponse
- ThreadBudgetInfo, ThreadBudgetListResponse
- AgentUsageInfo, ModelUsageInfo
Also adds has_budget() method to ThreadBudgetRegistry for checking
if a thread exists without auto-creating it.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Adds operator-only endpoints for discovering organism capabilities:
- GET /api/v1/capabilities - list all listeners
- GET /api/v1/capabilities/{name} - detailed info with schema/example
These are REST-only for operators. Agents cannot access them -
they only know their declared peers (peer constraint isolation).
10 new tests for introspection functionality.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Implements runtime configuration reload via POST /api/v1/organism/reload:
- StreamPump.reload_config() re-reads organism.yaml
- Adds new listeners, removes old ones, updates changed ones
- System listeners (system.*) are protected from removal
- ReloadEvent emitted to notify WebSocket subscribers
- ServerState.reload_config() refreshes agent runtime state
14 new tests covering add/remove/update scenarios.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Implements the AgentServer API from docs/agentserver_api_spec.md:
REST API (/api/v1):
- Organism info and config endpoints
- Agent listing, details, config, schema
- Thread and message history with filtering
- Control endpoints (inject, pause, resume, kill, stop)
WebSocket:
- /ws: Main control channel with state snapshot + real-time events
- /ws/messages: Dedicated message stream with filtering
Infrastructure:
- Pydantic models with camelCase serialization
- ServerState bridges StreamPump to API
- Pump event hooks for real-time updates
- CLI 'serve' command: xml-pipeline serve [config] --port 8080
35 new tests for models, state, REST, and WebSocket.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Implement two virtual node patterns for message flow orchestration:
- Sequence: Chains listeners in order (A→B→C), feeding each step's
output as input to the next. Uses ephemeral listeners to intercept
step results without modifying core pump behavior.
- Buffer: Fan-out to parallel worker threads with optional result
collection. Supports fire-and-forget mode (collect=False) for
non-blocking dispatch.
New files:
- sequence_registry.py / buffer_registry.py: State tracking
- sequence.py / buffer.py: Payloads and handlers
- test_sequence.py / test_buffer.py: 52 new tests
Pump additions:
- register_generic_listener(): Accept any payload type
- unregister_listener(): Cleanup ephemeral listeners
- Global singleton accessors for pump instance
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Move lsp-integration.md and secure-console-v3.md to docs/archive-obsolete/
(these features are now in the Nextra SaaS product)
- Update CLAUDE.md with current project state
- Simplify run_organism.py
- Fix test fixtures for shared backend compatibility
- Minor handler and llm_api cleanups
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Introduces SharedBackend Protocol for cross-process state sharing:
- InMemoryBackend: default single-process storage
- ManagerBackend: multiprocessing.Manager for local multi-process
- RedisBackend: distributed deployments with TTL auto-GC
Adds ProcessPoolExecutor support for CPU-bound handlers:
- worker.py: worker process entry point
- stream_pump.py: cpu_bound handler dispatch
- Config: backend and process_pool sections in organism.yaml
ContextBuffer and ThreadRegistry now accept optional backend
parameter while maintaining full backward compatibility.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
OSS restructuring for open-core model:
- Rename package from agentserver/ to xml_pipeline/
- Update all imports (44 Python files, 31 docs/configs)
- Update pyproject.toml for OSS distribution (v0.3.0)
- Move prompt_toolkit from core to optional [console] extra
- Remove auth/server/lsp from core optional deps (-> Nextra)
New console example in examples/console/:
- Self-contained demo with handlers and config
- Uses prompt_toolkit (optional, falls back to input())
- No password auth, no TUI, no LSP — just the basics
- Shows how to use xml-pipeline as a library
Import changes:
- from agentserver.* -> from xml_pipeline.*
- CLI entry points updated: xml_pipeline.cli:main
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- SecureConsole with Argon2id password hashing
- Password stored in ~/.xml-pipeline/console.key
- Protected commands require password re-entry
- Attach/detach model with idle timeout
- Fallback to simple input for incompatible terminals
- @listener message injection into pump
- Boot handler no longer sends to old console
- Response handler prints to stdout with prompt refresh
Dependencies: argon2-cffi, prompt_toolkit
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Implements an observer pattern where agents can register watchers
for conditions on their thread. When the condition is met, the
agent gets "nagged" on subsequent invocations until it explicitly
closes the todo.
Key components:
- TodoRegistry: thread-scoped watcher tracking with eyebrow state
- TodoUntil/TodoComplete payloads and system handlers
- HandlerMetadata.todo_nudge for delivering raised eyebrow notices
- Integration in StreamPump dispatch to check and nudge
Greeter now demonstrates the pattern:
1. Registers watcher for ShoutedResponse from shouter
2. On next invocation, sees nudge and closes completed todos
3. Includes nudge in LLM prompt for awareness
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Demonstrates message chaining: user -> greeter -> shouter -> user
- Add ShoutedResponse payload class (ALL CAPS response)
- Add handle_shout handler that forwards to original sender
- Update GreetingResponse to track original_sender
- Fix wrap_in_envelope to add xmlns="" to payloads
- Update organism.yaml with shouter listener
- Update tests for 2-listener configuration
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Major refactor of the message pump architecture:
- Replace bus.py with stream_pump.py using aiostream for composable
stream processing with natural fan-out via flatmap
- Add to_id field to MessageState for explicit routing
- Fix routing to use to_id.class format (e.g., "greeter.greeting")
- Generate XSD schemas from xmlified payload classes
- Fix xmlable imports (absolute -> relative) and parse_element ctx
New features:
- handlers/hello.py: Sample Greeting/GreetingResponse handler
- config/organism.yaml: Sample organism configuration
- 41 tests (31 unit + 10 integration) all passing
Schema changes:
- envelope.xsd: Allow any namespace payloads (##other -> ##any)
Dependencies added to pyproject.toml:
- aiostream>=0.5 (core dependency)
- pyhumps, termcolor (for xmlable)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
