Three workstreams implemented:
W1 (Repo Split): Remove proprietary BloxServer files and docs, update
pyproject.toml URLs to public GitHub, clean doc references, add CI
workflow (.github/workflows/ci.yml) and CONTRIBUTING.md.
W2 (Message Journal): Add DispatchHook protocol for dispatch lifecycle
events, SQLite-backed MessageJournal with WAL mode for certified-mail
delivery guarantees (PENDING→DISPATCHED→ACKED/FAILED), integrate hooks
into StreamPump._dispatch_to_handlers(), add journal REST endpoints,
and aiosqlite dependency.
W3 (Hot Deployment): Add RestartOrchestrator for graceful restart with
queue drain and journal stats collection, SIGHUP signal handler in CLI,
POST /organism/restart endpoint, restart-aware app lifespan with journal
recovery on boot, and os.execv/subprocess re-exec for Unix/Windows.
All 439 tests pass (37 new tests for W2/W3).
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
When threads terminate (handler returns None or chain exhausted),
the pump now calls budget_registry.cleanup_thread() to:
- Free memory for completed threads
- Return final budget for logging/billing
- Log token usage at debug level
This ensures budgets don't accumulate for completed conversations.
Also adds:
- has_budget() method to check if thread exists without creating
- Tests for cleanup behavior
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Implements runtime configuration reload via POST /api/v1/organism/reload:
- StreamPump.reload_config() re-reads organism.yaml
- Adds new listeners, removes old ones, updates changed ones
- System listeners (system.*) are protected from removal
- ReloadEvent emitted to notify WebSocket subscribers
- ServerState.reload_config() refreshes agent runtime state
14 new tests covering add/remove/update scenarios.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Implements the AgentServer API from docs/agentserver_api_spec.md:
REST API (/api/v1):
- Organism info and config endpoints
- Agent listing, details, config, schema
- Thread and message history with filtering
- Control endpoints (inject, pause, resume, kill, stop)
WebSocket:
- /ws: Main control channel with state snapshot + real-time events
- /ws/messages: Dedicated message stream with filtering
Infrastructure:
- Pydantic models with camelCase serialization
- ServerState bridges StreamPump to API
- Pump event hooks for real-time updates
- CLI 'serve' command: xml-pipeline serve [config] --port 8080
35 new tests for models, state, REST, and WebSocket.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Implement two virtual node patterns for message flow orchestration:
- Sequence: Chains listeners in order (A→B→C), feeding each step's
output as input to the next. Uses ephemeral listeners to intercept
step results without modifying core pump behavior.
- Buffer: Fan-out to parallel worker threads with optional result
collection. Supports fire-and-forget mode (collect=False) for
non-blocking dispatch.
New files:
- sequence_registry.py / buffer_registry.py: State tracking
- sequence.py / buffer.py: Payloads and handlers
- test_sequence.py / test_buffer.py: 52 new tests
Pump additions:
- register_generic_listener(): Accept any payload type
- unregister_listener(): Cleanup ephemeral listeners
- Global singleton accessors for pump instance
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Introduces SharedBackend Protocol for cross-process state sharing:
- InMemoryBackend: default single-process storage
- ManagerBackend: multiprocessing.Manager for local multi-process
- RedisBackend: distributed deployments with TTL auto-GC
Adds ProcessPoolExecutor support for CPU-bound handlers:
- worker.py: worker process entry point
- stream_pump.py: cpu_bound handler dispatch
- Config: backend and process_pool sections in organism.yaml
ContextBuffer and ThreadRegistry now accept optional backend
parameter while maintaining full backward compatibility.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
OSS restructuring for open-core model:
- Rename package from agentserver/ to xml_pipeline/
- Update all imports (44 Python files, 31 docs/configs)
- Update pyproject.toml for OSS distribution (v0.3.0)
- Move prompt_toolkit from core to optional [console] extra
- Remove auth/server/lsp from core optional deps (-> Nextra)
New console example in examples/console/:
- Self-contained demo with handlers and config
- Uses prompt_toolkit (optional, falls back to input())
- No password auth, no TUI, no LSP — just the basics
- Shows how to use xml-pipeline as a library
Import changes:
- from agentserver.* -> from xml_pipeline.*
- CLI entry points updated: xml_pipeline.cli:main
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-19 21:41:19 -08:00
Renamed from agentserver/message_bus/stream_pump.py (Browse further)
