From 67c77d568edbe5d5594942a76c612767c1609123 Mon Sep 17 00:00:00 2001 From: dullfig Date: Sun, 4 Jan 2026 22:22:17 -0800 Subject: [PATCH] added rant --- docs/core-principles-v2.0.md | 16 ++++++ docs/message-pump.md | 67 ++++++++++++++++++------- docs/thread-management.md | 94 +++++++++++++++--------------------- 3 files changed, 104 insertions(+), 73 deletions(-) diff --git a/docs/core-principles-v2.0.md b/docs/core-principles-v2.0.md index 6eb57c5..e7b6b65 100644 --- a/docs/core-principles-v2.0.md +++ b/docs/core-principles-v2.0.md @@ -11,6 +11,16 @@ These principles are the single canonical source of truth for the project. All d - Exclusive C14N on ingress and egress. - Malformed XML repaired on ingress; repairs logged in `` metadata. +## Identity Injection & Handler Purity +- Handlers are pure, stateless functions with no knowledge of routing, thread context, or their own registered name. +- On ingress (external or gateway messages): is provided and authenticated by the client/gateway (enforced by envelope validation). +- On response generation (after handler execution and multi-payload extraction): + - The dispatcher injects using the executing listener's registered name (e.g., "calculator.add" or "researcher"). + - For meta/primitive responses: is injected as "core". +- is inherited from the incoming message (or assigned/updated for primitives like spawn-thread). +- remains optional and rarely used. +- This ensures every enveloped message has a trustworthy, auditable without handler involvement, preventing spoofing and keeping capability code minimal/testable. + ## Configuration & Composition - YAML file (`organism.yaml`) is the bootstrap source of truth, loaded at startup. - Defines initial listeners, agents, gateways, meta privileges, and OOB channel configuration. @@ -46,6 +56,7 @@ These principles are the single canonical source of truth for the project. All d - Subthreading natively supported via hierarchical thread IDs and primitives (e.g., reserved payload to spawn "parent.sub1"). - Optional structured constructs like `` for visible planning. - No hidden loops or state machines; all reasoning steps are visible messages. +- Thread management follows the dynamic call tracing model (see thread-management.md). Paths are built by appending target listener names on emission, with automatic popping on responses. Agents remain oblivious, enabling natural delegation and parallelism. ## Security & Sovereignty - Privileged messages (per `privileged-msg.xsd`) handled exclusively on dedicated OOB channel. @@ -73,4 +84,9 @@ These principles are the single canonical source of truth for the project. All d - Single process, async non-blocking. - XML is the sovereign wire format; everything else is implementation detail. +## Scheduled Computation +- Timers and delays implemented as normal listeners using async sleeps. +- Caller idles naturally; wakeup messages bubble back via standard tracing. +- Enables recurrent tasks (e.g., periodic monitoring) without blocking or external schedulers. + These principles are now locked. All existing docs will be updated to match this file exactly. Future changes require explicit discussion and amendment here first. \ No newline at end of file diff --git a/docs/message-pump.md b/docs/message-pump.md index dd02567..4d4bb83 100644 --- a/docs/message-pump.md +++ b/docs/message-pump.md @@ -4,25 +4,56 @@ The AgentServer message pump processes individual messages through a single, lin ```mermaid flowchart TD - A["WebSocket Ingress\n(enqueue to thread buffer)"] --> B["Dispatcher Loop:\nSelect next message\n(per thread_scheduling strategy)"] - B --> C["Repair + Exclusive C14N"] - C --> D["Envelope Validation (lxml)"] - D --> E["Extract Payload Tree"] - E --> F{"Payload Namespace?"} - F -->|meta/v1| G["Core Meta Handler\n(introspection & reserved primitives)"] - F -->|capability| H["Route by (namespace, root)"] - H --> I["Validate Payload vs Listener XSD (lxml)"] - I --> J["Deserialize to Dataclass Instance (xmlable)"] - J --> K["Call handler(instance) → raw bytes"] - K --> L["Wrap bytes in "] - L --> M["Repair/Parse → Extract all top-level payloads"] - M --> N["Wrap each payload in separate envelope\n(enqueue to target thread buffers)"] - G --> N - N --> O["Exclusive C14N + Sign"] - O --> P["WebSocket Egress\n(sequential per connection)"] - P --> B["Continue dispatcher loop if buffers non-empty"] + subgraph MessagePump + subgraph Init + start([Start]) + raw[/Optional
Raw Bytes/] + wrapstart["Wrap
<start>{...}</start>"] + end + enq1([QUEUE 1]) + rawwaiting{Raw
Msg
Waiting?} + waitRaw([Wait]) + subgraph Process + extract["Extract
Tree"] + split["Split
Tree"] + subgraph foreach [For Each Message] + getmsg[Get Msg] + badTo{<To> Missing?} + endnoto([Discard]) + addfrom["Add .from"] + repair[Repair + C14N] + validate[Validate] + invalidMsg{Bad
Message?} + badmsg([Discard]) + more{More?} + end + enqueue([QUEUE 2]) + xmlWaiting{XML
Waiting?} + waitXml([Wait]) + subgraph Async + lookup[Lookup Listener] + route[Route] + wait[await Response] + wrap["Wrap
<from>{...}</from>"] + end + end + end + + start --> raw --> wrapstart --> enq1 --> rawwaiting + rawwaiting --> |NO| waitRaw + rawwaiting ---> |YES| extract + extract --> split --> foreach + getmsg --> badTo + badTo --> |YES| endnoto + badTo --> |NO| addfrom --> repair --> validate --> invalidMsg + invalidMsg ---> |NO| more --> |Yes| getmsg + invalidMsg --> |YES| badmsg + more --> |NO| enqueue + enqueue --> xmlWaiting + xmlWaiting --> |NO| waitXml + xmlWaiting ---> |YES| lookup --> route --> wait --> wrap --> enq1 + ``` - ## Detailed Stages (Per-Message) 1. **Ingress/Enqueue**: Raw bytes → repair → preliminary tree → enqueue to target thread buffer. diff --git a/docs/thread-management.md b/docs/thread-management.md index a1a8954..19a68de 100644 --- a/docs/thread-management.md +++ b/docs/thread-management.md @@ -1,65 +1,49 @@ # Thread Management in AgentServer v2.0 -**January 03, 2026** +**January 04, 2026** -This document clarifies the thread ID system, subthreading mechanics, and internals. It supplements the Core Architectural Principles — hierarchical dot notation examples there reflect the wire format. +## Overview +Thread IDs are dynamic hierarchical paths that trace the exact call chain through the organism. The message pump builds and maintains them automatically. +Agents, tools, and handlers are **completely oblivious** to thread IDs — they never read, copy, or emit them. -## Wire Format: Hierarchical String IDs -- Mandatory `` contains a **server-assigned** hierarchical string (dot notation, e.g., "root", "root.research", "root.research.images"). -- Root IDs: Short, opaque, server-generated (e.g., "sess-abcd1234"). -- Sub-IDs: Relative extensions for readability. -- Benefits: LLM/human-friendly copying, natural tree structure for logs/GUI. +No explicit spawn primitives are required. Topology emerges solely from the shape (single vs multiple) and targets of emitted payloads. -## Server Assignment Only -The organism assigns all final IDs — agents never invent them. +## Wire Format +- Mandatory `` contains a readable dot-notation string (e.g., `sess-abcd1234.researcher.search.calc`). +- Root segment is an opaque server-generated session identifier. +- Subsequent segments are registered listener short names appended during routing. -- **Root initiation**: Client suggests or server auto-generates on first message; uniqueness enforced. -- **Subthread spawning**: Explicit reserved payload for intent clarity: - ```xml - - - - - - ``` - Core handler: - - Appends label (or auto-short if omitted). - - Resolves uniqueness conflicts (append "-1" etc.). - - Creates queue + seeds bootstrap. - - **Always responds** in current thread: - ```xml - - ``` +## Dynamic Call Tracing Rules +1. **Emission** (from current path `parent.path`): + - After handler execution and multi-payload extraction: + - For each payload, determine target listener name. + - Append that name → new path = `parent.path.target_name`. + - Single payload → sequential delegation (one deepened path). + - Multiple payloads → parallel forks (one deepened path per target, each with its own queue). -## Error Handling (No Silent Failure) -- Unknown `` ID → no implicit creation. -- **Always inject** system error into parent thread (or root): - ```xml - - ``` -- LLM sees error immediately, retries without hanging. -- Logs warning for monitoring. +2. **Response Bubbling**: + - On emission from path `parent.path.listener_name`: + - Pump removes the last segment. + - Routes all response payloads to `parent.path`. + - Injects `` as the responding listener’s registered name. + - Replies land directly in the immediate parent’s history. -## Internals -- Per-thread queues: dict[str, Queue]. -- Scheduling via `organism.yaml`: - ```yaml - thread_scheduling: "breadth-first" # or "depth-first" (default: breadth-first) - ``` - - Depth from dot count. -- Optional hidden UUID mapping for extra safety (implementation detail). +3. **Broadcast**: + - Single payload to a capability with multiple gateways → fanned out. + - All responses pop to the same parent path, distinguished by their individual `` values. -## Design Rationale -- Explicit spawn = clear intent + bootstrap hook. -- Mandatory feedback = no LLM limbo. -- Readable IDs = easy copying without UUID mangling. -- Server control = sovereignty + no collisions. +4. **Internal Uniqueness**: + - Readable paths are mapped to UUIDs via a bidirectional resolver. + - New unique logical path → new UUID and queue. + - Ensures collision-free scheduling while keeping wire paths clean and meaningful. -Future: Alias registry, thread metadata primitives. +## Termination +- Paths become idle when their queues empty. +- Detection of `` (meta namespace) in the root path triggers terminal egress to the originating client. -The organism branches reliably, visibly, and recoverably. +## Key Advantages +- Complete thread obliviousness eliminates prompt bloat and copy errors. +- Natural sequential delegation and parallelism without manual management. +- Full provenance via trustworthy `` injection. +- Audit trails are self-documenting call traces built from registered capability names. + +The organism owns memory and topology. Threads are the living, transparent traces of computation. \ No newline at end of file