LLM Wrappers Agent

Demonstrates three Pythonic LLM wrapper libraries: Mirascope (call decorators and streaming), Magentic (prompt functions), and Marvin (AI extraction/classification). A wrapper-runner child agent exercises all 3 libraries while a wrapper-evaluator assesses coverage and synthesizes a comparison.

Environment variables

This example requires OPENAI_API_KEY, WAXELL_API_KEY, and WAXELL_API_URL. Use --dry-run to skip real API calls.

Architecture

Key Code

Mirascope: Call Decorators and Streaming

@tool-decorated functions exercise Mirascope's create_factory, stream_factory, and OpenAICallResponse.

@waxell.tool(tool_type="llm_wrapper")
def run_mirascope_call(prompt: str, model: str = "gpt-4o-mini") -> dict:
    """Run Mirascope create_factory (call decorator pattern)."""
    response = mock_mirascope_create(prompt=prompt, model=model)
    return {"framework": "mirascope", "mode": "call", "content": response.content[:200]}

@waxell.tool(tool_type="llm_wrapper")
def run_mirascope_stream(prompt: str, model: str = "gpt-4o-mini") -> dict:
    """Run Mirascope stream_factory (streaming call decorator)."""
    chunks = list(mock_mirascope_stream(prompt=prompt, model=model))
    return {"framework": "mirascope", "mode": "stream", "chunks": len(chunks)}

Magentic: Prompt Functions

@tool-decorated functions exercise Magentic's PromptFunction.__call__ and ChatModel.complete.

@waxell.tool(tool_type="llm_wrapper")
def run_magentic_prompt(prompt: str) -> dict:
    """Run Magentic PromptFunction.__call__ (prompt function pattern)."""
    result = mock_magentic_prompt_fn(prompt)
    return {"framework": "magentic", "mode": "prompt_function", "output": result[:200]}

@waxell.tool(tool_type="llm_wrapper")
def run_magentic_complete(messages: list) -> dict:
    """Run Magentic ChatModel.complete (chat model pattern)."""
    result = mock_magentic_complete(messages)
    return {"framework": "magentic", "mode": "chat_model", "output": result[:200]}

Marvin: AI Extraction and Classification

@tool-decorated functions exercise Marvin's extract, classify, cast, generate, and AIFunction.__call__.

@waxell.tool(tool_type="llm_wrapper")
def run_marvin_extract(text: str, target_type: str) -> dict:
    """Run Marvin extract() for structured entity extraction."""
    entities = mock_marvin_extract(text, target_type)
    return {"framework": "marvin", "mode": "extract", "entities": len(entities)}

@waxell.tool(tool_type="llm_wrapper")
def run_marvin_classify(text: str, labels: list) -> dict:
    """Run Marvin classify() for text classification."""
    label = mock_marvin_classify(text, labels)
    return {"framework": "marvin", "mode": "classify", "label": label}

What this demonstrates

• Mirascope instrumentor -- create_factory, stream_factory, and OpenAICallResponse.__init__ traced with framework context.
• Magentic instrumentor -- PromptFunction.__call__ and ChatModel.complete with prompt template metadata.
• Marvin instrumentor -- extract, classify, cast, generate, and AIFunction.__call__ (@marvin.fn) traced.
• waxell.decide() for wrapper routing -- manual routing decision selecting which libraries to exercise.
• @retrieval for result gathering -- collects outputs from all 3 libraries for comparison.
• @reasoning for coverage assessment -- documents which API surface each library covers.

Run it

# Dry-run mode (no API key needed)
cd dev/waxell-dev
python -m app.demos.llm_wrappers_agent --dry-run

# Live mode
export OPENAI_API_KEY="sk-..."
python -m app.demos.llm_wrappers_agent

Source

dev/waxell-dev/app/demos/llm_wrappers_agent.py