dullfig/xml-pipeline
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
xml-pipeline/xml_pipeline/memory
History
dullfig 6790c7a46c Add shared backend for multiprocess pipeline support 
Introduces SharedBackend Protocol for cross-process state sharing:
- InMemoryBackend: default single-process storage
- ManagerBackend: multiprocessing.Manager for local multi-process
- RedisBackend: distributed deployments with TTL auto-GC

Adds ProcessPoolExecutor support for CPU-bound handlers:
- worker.py: worker process entry point
- stream_pump.py: cpu_bound handler dispatch
- Config: backend and process_pool sections in organism.yaml

ContextBuffer and ThreadRegistry now accept optional backend
parameter while maintaining full backward compatibility.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-20 20:18:22 -08:00
..
__init__.py Add shared backend for multiprocess pipeline support 2026-01-20 20:18:22 -08:00
context_buffer.py Add shared backend for multiprocess pipeline support 2026-01-20 20:18:22 -08:00
manager_backend.py Add shared backend for multiprocess pipeline support 2026-01-20 20:18:22 -08:00
memory_backend.py Add shared backend for multiprocess pipeline support 2026-01-20 20:18:22 -08:00
redis_backend.py Add shared backend for multiprocess pipeline support 2026-01-20 20:18:22 -08:00
shared_backend.py Add shared backend for multiprocess pipeline support 2026-01-20 20:18:22 -08:00