Add LLM-created sequences at runtime to edge analysis spec
- Runtime policy: green-only (no YOLO on yellow) - LLM clarification flow when wiring fails - Edge hints payload: map, constant, drop, expression - Structured error response for LLM to resolve issues Conservative but flexible: LLM can provide explicit instructions to turn yellow into green. Co-authored-by: Dan
This commit is contained in:
parent
ca7e426d57
commit
cf49154ddd
1 changed files with 98 additions and 0 deletions
|
|
@ -299,6 +299,104 @@ def generate_transformer(edge_mapping: EdgeMapping) -> Callable:
|
|||
|
||||
3. Transformer is called between each step in the sequence
|
||||
|
||||
## LLM-Created Sequences (Runtime)
|
||||
|
||||
Agents can dynamically create sequences at runtime. The sequencer factory runs the same analysis but with stricter rules.
|
||||
|
||||
### Runtime Policy
|
||||
|
||||
| Confidence | Design-time (Canvas) | Run-time (LLM) |
|
||||
|------------|---------------------|----------------|
|
||||
| 🟢 High | Auto-wire, run | Auto-wire, run |
|
||||
| 🟡 Medium | Show warning, let user decide | **Block**, request clarification |
|
||||
| 🔴 Low | Show error, require manual | **Block**, request clarification |
|
||||
|
||||
**Rationale:** Letting LLMs YOLO on uncertain mappings is risky. Better to ask for explicit instructions.
|
||||
|
||||
### LLM Clarification Flow
|
||||
|
||||
```
|
||||
1. LLM requests sequence: A → B → C
|
||||
2. Factory analyzes:
|
||||
- A → B: green ✓
|
||||
- B → C: yellow (ambiguous)
|
||||
3. Factory responds with structured error:
|
||||
- Which edge failed
|
||||
- What B outputs
|
||||
- What C expects
|
||||
- Suggested resolutions
|
||||
4. LLM provides hints:
|
||||
- Explicit mappings
|
||||
- Constants for missing fields
|
||||
- Fields to drop
|
||||
5. Factory re-analyzes with hints
|
||||
6. If green: run. If not: back to step 3.
|
||||
```
|
||||
|
||||
### Edge Hints Payload
|
||||
|
||||
LLMs can provide explicit wiring instructions:
|
||||
|
||||
```xml
|
||||
<CreateSequence>
|
||||
<steps>transformer, uploader</steps>
|
||||
<initial_payload>...</initial_payload>
|
||||
|
||||
<edge_hints>
|
||||
<edge from="transformer" to="uploader">
|
||||
<map from="result" to="payload"/>
|
||||
<constant field="destination">/uploads/output.json</constant>
|
||||
<drop field="factor"/>
|
||||
<drop field="metadata"/>
|
||||
</edge>
|
||||
</edge_hints>
|
||||
</CreateSequence>
|
||||
```
|
||||
|
||||
### Hint Types
|
||||
|
||||
| Hint | Syntax | Effect |
|
||||
|------|--------|--------|
|
||||
| Map field | `<map from="X" to="Y"/>` | Wire source field to target field |
|
||||
| Set constant | `<constant field="Y">value</constant>` | Set target field to literal value |
|
||||
| Drop field | `<drop field="X"/>` | Explicitly ignore source field |
|
||||
| Expression | `<expr field="Y">concat(X.a, X.b)</expr>` | Compute target from expression |
|
||||
|
||||
### Error Response to LLM
|
||||
|
||||
When wiring fails:
|
||||
|
||||
```xml
|
||||
<SequenceError>
|
||||
<code>wiring_failed</code>
|
||||
<edge from="transformer" to="uploader"/>
|
||||
|
||||
<source_fields>
|
||||
<field name="result" type="string"/>
|
||||
<field name="factor" type="int"/>
|
||||
<field name="metadata" type="object"/>
|
||||
</source_fields>
|
||||
|
||||
<target_fields>
|
||||
<field name="payload" type="string" required="true" mapped="result" confidence="0.85"/>
|
||||
<field name="destination" type="string" required="true" mapped="" confidence="0"/>
|
||||
</target_fields>
|
||||
|
||||
<issues>
|
||||
<issue type="unmapped_required">destination has no source</issue>
|
||||
<issue type="unmapped_source">factor will be dropped</issue>
|
||||
<issue type="unmapped_source">metadata will be dropped</issue>
|
||||
</issues>
|
||||
|
||||
<suggestion>Provide mapping for 'destination' or set as constant.</suggestion>
|
||||
</SequenceError>
|
||||
```
|
||||
|
||||
This gives the LLM enough information to either:
|
||||
- Provide the missing hints
|
||||
- Try a different sequence
|
||||
- Ask the user for help
|
||||
|
||||
## Future Enhancements
|
||||
|
||||
### v1.1 — Type Coercion
|
||||
|
|
|
|||
Loading…
Reference in a new issue