From cf49154ddd65971ee6221ad38a41f1f94000e163 Mon Sep 17 00:00:00 2001 From: Donna Date: Mon, 26 Jan 2026 06:59:51 +0000 Subject: [PATCH] Add LLM-created sequences at runtime to edge analysis spec - Runtime policy: green-only (no YOLO on yellow) - LLM clarification flow when wiring fails - Edge hints payload: map, constant, drop, expression - Structured error response for LLM to resolve issues Conservative but flexible: LLM can provide explicit instructions to turn yellow into green. Co-authored-by: Dan --- docs/edge-analysis-spec.md | 98 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 98 insertions(+) diff --git a/docs/edge-analysis-spec.md b/docs/edge-analysis-spec.md index d5a9126..ba681fc 100644 --- a/docs/edge-analysis-spec.md +++ b/docs/edge-analysis-spec.md @@ -299,6 +299,104 @@ def generate_transformer(edge_mapping: EdgeMapping) -> Callable: 3. Transformer is called between each step in the sequence +## LLM-Created Sequences (Runtime) + +Agents can dynamically create sequences at runtime. The sequencer factory runs the same analysis but with stricter rules. + +### Runtime Policy + +| Confidence | Design-time (Canvas) | Run-time (LLM) | +|------------|---------------------|----------------| +| 🟢 High | Auto-wire, run | Auto-wire, run | +| 🟡 Medium | Show warning, let user decide | **Block**, request clarification | +| 🔴 Low | Show error, require manual | **Block**, request clarification | + +**Rationale:** Letting LLMs YOLO on uncertain mappings is risky. Better to ask for explicit instructions. + +### LLM Clarification Flow + +``` +1. LLM requests sequence: A → B → C +2. Factory analyzes: + - A → B: green ✓ + - B → C: yellow (ambiguous) +3. Factory responds with structured error: + - Which edge failed + - What B outputs + - What C expects + - Suggested resolutions +4. LLM provides hints: + - Explicit mappings + - Constants for missing fields + - Fields to drop +5. Factory re-analyzes with hints +6. If green: run. If not: back to step 3. +``` + +### Edge Hints Payload + +LLMs can provide explicit wiring instructions: + +```xml + + transformer, uploader + ... + + + + + /uploads/output.json + + + + + +``` + +### Hint Types + +| Hint | Syntax | Effect | +|------|--------|--------| +| Map field | `` | Wire source field to target field | +| Set constant | `value` | Set target field to literal value | +| Drop field | `` | Explicitly ignore source field | +| Expression | `concat(X.a, X.b)` | Compute target from expression | + +### Error Response to LLM + +When wiring fails: + +```xml + + wiring_failed + + + + + + + + + + + + + + + destination has no source + factor will be dropped + metadata will be dropped + + + Provide mapping for 'destination' or set as constant. + +``` + +This gives the LLM enough information to either: +- Provide the missing hints +- Try a different sequence +- Ask the user for help + ## Future Enhancements ### v1.1 — Type Coercion