Structured Generation Agent

A multi-agent comparison of three constrained text generation frameworks: Outlines (schema/regex-constrained decoding), Guidance (template programs via model operators), and LMQL (query-language constraints). Each framework runs as a child agent, exercising the exact methods their instrumentors wrap.

Environment variables

This example requires OPENAI_API_KEY, WAXELL_API_KEY, and WAXELL_API_URL. Use --dry-run to skip real API calls.

Architecture

Key Code

Outlines: Schema-Constrained Decoding

Four @tool calls exercise the Outlines SequenceGeneratorAdapter.__call__ via json, regex, text, and choice factories.

@waxell.tool(name="generate_json", tool_type="structured_gen")
def generate_json(prompt: str) -> dict:
    """Generate JSON-conforming output using Outlines json factory + adapter."""
    json_adapter = mock_outlines_json(model="mistralai/Mistral-7B", schema=PRODUCT_SCHEMA)
    raw_output = json_adapter(prompt)  # SequenceGeneratorAdapter.__call__
    parsed = json.loads(raw_output)
    return {"framework": "outlines", "output_type": "json", "product": parsed.get("name", "")}

@waxell.tool(name="generate_regex", tool_type="structured_gen")
def generate_regex(prompt: str, pattern: str) -> dict:
    regex_adapter = mock_outlines_regex(model="mistralai/Mistral-7B", pattern=pattern)
    return {"framework": "outlines", "output_type": "regex", "result": regex_adapter(prompt)}

Guidance: Template Programs via Model Operators

Four @tool calls exercise Model.__add__, Model.__call__, Chat.__call__, and gen().

@waxell.tool(name="run_model_program", tool_type="structured_gen")
def run_model_program(model: MockGuidanceModel, template: str) -> dict:
    """Execute a guidance program using Model.__add__ (+ operator)."""
    result = model + template
    return {"framework": "guidance", "mode": "model.__add__", "summary": result["summary"]}

@waxell.tool(name="run_chat", tool_type="structured_gen")
def run_chat(chat_model: MockGuidanceChat, prompt: str) -> dict:
    """Execute a guidance chat using Chat.__call__."""
    result = chat_model(prompt)
    return {"framework": "guidance", "mode": "chat.__call__", "response": result["response"]}

What this demonstrates

• Outlines instrumentor -- SequenceGeneratorAdapter.__call__ and all 4 factory functions (json, regex, text, choice).
• Guidance instrumentor -- Model.__add__, Model.__call__, Chat.__call__, and gen() primitive.
• LMQL instrumentor -- lmql.run() and LMQLQueryFunction.__call__ (@lmql.query).
• 10 structured generation operations -- 4 Outlines + 4 Guidance + 2 LMQL across 3 child agents.
• @reasoning comparison -- documents tradeoffs between schema-constrained, template-based, and query-language approaches.
• @decision for framework selection -- routes to individual or all frameworks based on query analysis.

Run it

# Dry-run mode (no API key needed)
cd dev/waxell-dev
python -m app.demos.structured_gen_agent --dry-run

# Live mode
export OPENAI_API_KEY="sk-..."
python -m app.demos.structured_gen_agent

Source

dev/waxell-dev/app/demos/structured_gen_agent.py