Skip to main content

LLM Calls

Agents call models through ctx.llm, available on any workflow or tool context.

Generating​

from waxell_runtime import agent, workflow, tool


@agent(name="classifier", description="Classifies inbound messages")
class Classifier:

    @tool
    async def classify_intent(self, ctx, message: str) -> dict:
        """Classify the user's intent."""
        return await ctx.llm.generate(
            prompt=(
                "Classify this message as one of: question, complaint, "
                f'request, feedback. Reply with JSON {{"intent": ...}}.\n\n{message}'
            ),
            output_format="json",
            task="classification",
        )

    @workflow("triage")
    async def triage(self, ctx, message: str):
        result = await ctx.tool("classify_intent", message=message)
        return {"intent": result.get("intent")}

Two things worth knowing:

  • Put the model call in a @tool. Tool calls are the runtime's checkpoint boundary, so a model call inside a tool is replayed from its saved result on resume instead of being paid for twice. See Durable Execution.
  • task= drives routing. It's a hint the three-layer model routing uses to pick a provider, so a tenant can change models without a code change.

Parameters​

ParameterTypeDefaultMeaning
promptstrrequiredThe prompt
output_formatstr"text""text" returns a str; "json" returns a parsed dict
taskstr"general"Routing hint ("email", "summary", …)
inject_lessonsboolTrueWhether metacog prepends accumulated learnings to this call

model, temperature, and max_tokens pass through to the provider.

Structured output​

Ask for JSON and validate it. A model can always return something malformed, and you want that caught at the boundary rather than three calls later:

from pydantic import BaseModel


class Analysis(BaseModel):
    sentiment: str
    confidence: float
    topics: list[str]


@tool
async def analyze(self, ctx, text: str) -> dict:
    raw = await ctx.llm.generate(
        prompt=f"Analyze and return JSON with sentiment, confidence, topics:\n\n{text}",
        output_format="json",
        task="analysis",
    )
    return Analysis.model_validate(raw).model_dump()

Letting the model call tools​

generate_with_tools runs a tool-calling loop rather than returning a single completion. Use it when the model should gather information before it answers — querying a domain, checking a record — instead of answering from the prompt alone.

For intent routing across several behaviours, @router wraps this pattern:

from waxell_sdk import router, RouterContext


@router("main_router", decision="classify_intent", auto_context_tools=True)
async def main_router(ctx: RouterContext) -> dict:
    """Route user messages to the right capability."""
    return await ctx.route()

auto_context_tools=True exposes your read-only domain actions so the model can look things up before deciding.

What gets recorded​

Every model call is governed and recorded:

  • Mid-run policy evaluates each tool call before it executes, so a disallowed action is stopped at the call.
  • LlmCallRecord captures model, tokens in/out, cost, and provider instance — visible in Observe and Billing → Usage.
  • Limits bound cost, turns, and wall time; a run that reaches one finishes BLOCKED with an ExecutionIncident rather than raising.

See How the Runtime Works.