M07C04: Resource Safety – Context Managers, Cleanup Guarantees, and Partial Consumption¶

Module 07 – Main Track Core

Main track: Cores 1, 3–10 (Ports & Adapters + Capability Protocols → Production).
This is a required core. Every production FuncPipe system must guarantee resource safety.

Progression Note¶

Module 7 takes the lawful containers and pipelines from Module 6 and puts all effects behind explicit boundaries.

Module	Focus	Key Outcomes
6	Monadic Flows as Composable Pipelines	Lawful `and_then`, Reader/State/Writer patterns, error-typed flows
7	Effect Boundaries & Resource Safety	Ports & adapters, capability protocols, resource-safe IO, idempotency
8	Async / Concurrent Pipelines	Backpressure, timeouts, resumability, fairness (built on 6–7)

Core question
How do you guarantee resource cleanup — even on errors or partial stream consumption — using context managers and contextlib, while keeping the core pure and composable?

What you now have after M07C01–M07C03 + this core - Pure domain core
- Zero direct I/O in domain code
- All I/O behind swappable ports
- Effectful operations described as pure data (IOPlan)
- Typed capability protocols for every common effect
- Reliable resource cleanup in all adapters, with explicit shell cooperation for deterministic cleanup on partial consumption

What the rest of Module 7 adds - Idempotent effect design
- Transaction/session patterns
- Incremental migration playbook
- Production story: CI, golden tests, shadow traffic

You are now three steps away from a complete production-grade functional architecture.

1. Laws & Invariants (machine-checked where possible)¶

Law / Invariant	Description	Enforcement
Cleanup Guarantee	Resources are released at most once; on normal exit or exception they are released.	Mock tests + Hypothesis
Partial Consumption Safety	Early iterator termination does not deliberately leak resources; shells explicitly close resource-owning iterators (via `contextlib.closing` or equivalent) for deterministic cleanup across implementations.	Property tests + shell tests
No Handle Escape	Resource-owning iterators must not yield live handles (file objects, DB connections, sockets); only fully detached values may escape.	Code review + tests
Nested Safety	`ExitStack` guarantees LIFO cleanup even with dynamic nesting.	Tests
Isolation	Resource management lives only in adapters/shells; core never opens resources directly.	mypy --strict + review

Important reality check: Resource-owning iterators clean up on exhaustion or explicit close. In CPython, GC finalization is effectively immediate, but not guaranteed across implementations. Our deterministic guarantee comes from shell-level explicit closing (contextlib.closing). We never rely solely on GC.

2. Decision Table – Which Resource Safety Pattern?¶

Scenario	Nested?	Dynamic?	Lazy Stream?	Recommended Pattern
Single file/connection	No	No	No	Plain `with open(...)`
Multiple known resources	Yes	No	No	Nested `with` statements
Dynamic/conditional resources	Yes	Yes	No	`contextlib.ExitStack`
Streaming over file/DB	Yes	No	Yes	Resource-owning iterator + shell wraps in `closing(...)`

Golden rule: For streaming adapters, use a resource-owning iterator and always wrap it in contextlib.closing (or equivalent) in the shell. This is the only pattern that gives deterministic cleanup on partial consumption across all Python implementations.

3. Public API – No New Domain API¶

This core adds no new public domain types. Resource safety is an implementation detail of adapters/shells. The visible contract remains the capability protocols from M07C01/C03.

4. Reference Implementations – Safe Adapters¶

4.1 Resource-Owning Iterator (streaming read)¶

# src/funcpipe_rag/infra/adapters/file_storage.py
#
# Full implementation lives in the repo. Key points:
# - `read_docs` is a resource-owning iterator (generator + `with open`)
# - parse failures yield `ErrInfo(code="PARSE_ROW", stage="storage.read_docs", ctx=...)`
# - OS errors yield `ErrInfo(code="IO_READ", stage="storage.read_docs", ...)`
class FileStorage(Storage): ...

4.2 Atomic Write with ExitStack¶

# src/funcpipe_rag/infra/adapters/file_storage.py
#
# Full implementation lives in the repo (atomic temp+fsync+replace, cleanup on failure).
def write_chunks(path: str, chunks: Iterator[Chunk]) -> Result[None, ErrInfo]: ...

4.3 Shell Responsibility – Deterministic Cleanup¶

# shell/pipeline.py
from contextlib import closing

def process_partial(storage: StorageRead, path: str, limit: int):
    with closing(storage.read_docs(path)) as docs:   # ← deterministic cleanup
        for i, res in enumerate(docs):
            if i >= limit:
                break
            # process res

5. Property-Based Proofs (selected)¶

# Repo tests for this contract:
# - tests/unit/infra/adapters/test_file_storage.py
#
# Optional exercise:
# - add a property test that forces a mid-stream exception during `write_chunks`
#   and asserts no leaked temp files remain.

6. Big-O & Allocation Guarantees¶

Operation	Time	Call-stack	Heap	Allocation
Context enter/exit	O(1)	O(1)	O(1)	O(1)
Resource-owning iterator	O(N) streaming	O(1)	O(1)	O(1) per item

Constant overhead; in I/O-bound pipelines the cost is negligible.

7. Anti-Patterns & Immediate Fixes¶

Anti-Pattern	Symptom	Fix
Open outside iterator	Leaks on partial consumption	Resource-owning iterator + shell closing
Manual try/finally nesting	Error-prone, verbose	`ExitStack`
Relying solely on GC for cleanup	Non-deterministic leaks	Explicit `closing(...)` in shell
Yielding live handles	Use-after-close crashes	Yield only detached values

8. Pre-Core Quiz¶

Resource safety even on…? → Partial consumption (with shell closing)
Preferred pattern for streaming adapters? → Resource-owning iterator
Deterministic cleanup requires…? → Shell wraps iterator in closing(...)
No Handle Escape prevents…? → Yielding live file/DB objects
Where do context managers live? → Adapters/shells only

9. Post-Core Exercise¶

Convert your current file read adapter to a true resource-owning iterator.
Add an ExitStack-based write adapter that opens multiple temp files conditionally.
Write a property test that injects a mid-stream exception and asserts no file leaks.
Refactor a shell to always use contextlib.closing on resource-owning iterators and prove via mock that cleanup happens even on early break.

Next → M07C05: Functional Logging & Tracing (Logs as Data, Monoidal Accumulation)

You now have reliable resource safety in every adapter — files, connections, and locks clean up correctly even if the pipeline aborts halfway. Combined with ports, capability protocols, and IOPlan, your effectful code is finally as safe as your pure core. The remaining cores are specialisations and production patterns.