Skip to main content

Durable Execution

Long-running agents outlive the processes that start them. WorkflowEnvelope is the execution boundary that makes a run survivable: it checkpoints progress so a crash, a redeploy, or a pause for human input resumes where it left off instead of starting over.

You don't construct an envelope — the runtime wraps your workflow in one. What you control is where the checkpoints fall.

What creates a checkpoint

Every tool call is a durable step:

from waxell_runtime import agent, workflow, tool


@agent(name="report_builder", description="Builds a quarterly report")
class ReportBuilder:

@tool
async def fetch_data(self, ctx, quarter: str) -> dict:
return await ctx.domain("finance", "quarterly", quarter=quarter)

@tool
async def summarize(self, ctx, data: dict) -> str:
return await ctx.llm.generate(prompt=f"Summarize: {data}")

@workflow("build")
async def build(self, ctx, quarter: str):
# Each ctx.tool(...) is a checkpoint boundary.
data = await ctx.tool("fetch_data", quarter=quarter)

# If the process dies here, resuming replays the line above from the
# saved result rather than re-fetching.
summary = await ctx.tool("summarize", data=data)

return {"quarter": quarter, "summary": summary}

Two things to note, because they're the usual mistakes:

  • Everything is async. ctx.tool, ctx.domain, and ctx.llm.generate are all awaited.
  • Checkpointing is automatic. There's no durable=True flag and no separate "durable workflow" type — a workflow's tool calls are its checkpoints.

Work done between tool calls is not checkpointed. If a computation is expensive enough that you'd hate to repeat it, put it in a @tool so its result is saved.

How resume works

  1. Before a step runs, the runtime records that it's starting.
  2. After it completes, the result is checkpointed.
  3. On resume, any step with a saved result is replayed from that result — the function does not run again.
  4. Execution continues from the first step without one.

This is why tools should be the unit of work: the checkpoint granularity is the tool boundary.

Run states

A run's lifecycle, from RunState:

StateMeaningTerminal
PENDINGAccepted, not yet started
RUNNINGExecuting
PAUSEDWaiting on something external
RESUMINGPicking back up from a checkpoint
COMPLETEDFinished successfully
FAILEDAn error inside the agent
BLOCKEDStopped by governance — a policy or budget
INTERRUPTEDThe process died (SIGTERM, OOM, hard kill) before finishing
Poll for all four terminal states

BLOCKED and INTERRUPTED are terminal. Code that waits for only COMPLETED or FAILED will hang forever on a policy-blocked or process-killed run.

INTERRUPTED is deliberately distinct from FAILED: an agent that errored is a different problem from a process that was killed under it, and collapsing them hides infrastructure issues.

Why a run is paused

PAUSED isn't one thing — PausedReason tells you what it's waiting on:

ReasonWaiting for
awaiting_childA spawned sub-agent
awaiting_signalAn inbound signal
awaiting_userA human answer (ctx.ask_user)
awaiting_timerA scheduled wake-up
awaiting_approvalA governance approval
suspended_orphanIts parent went away

A paused run holds no compute. It resumes when the thing it's waiting on arrives.

Where state is stored

The storage backend is chosen by which RuntimeBackend is configured, not by a separate setting:

  • Local developmentInMemoryBackend, automatic, nothing to configure. State lives in the process, so it does not survive a restart.
  • ProductionDjangoRuntimeBackend, wired automatically when waxell_infra is in INSTALLED_APPS. Checkpoints persist to the database and survive restarts and redeploys.

See Backends for how that wiring happens and how to supply your own.

Next Steps