Skip to main content

Instructor Agent

Demonstrates Instructor-style structured output extraction with Pydantic response models. A parent orchestrator coordinates an instructor-runner (entity extraction with parsing) and an instructor-evaluator (sentiment classification and quality assessment).

Environment variables

This example requires OPENAI_API_KEY, WAXELL_API_KEY, and WAXELL_API_URL. Use --dry-run to skip real API calls.

Architecture​

Key Code​

Entity Extraction with Pydantic Validation​

The runner extracts entities via LLM and validates the output against a response model.

@waxell.observe(agent_name="instructor-runner", workflow_name="instructor-extraction")
async def run_extraction(query: str, openai_client, response_model: str = "EntityExtraction", waxell_ctx=None):
    waxell.tag("framework", "instructor")
    waxell.metadata("response_model", response_model)

    response = await openai_client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[
            {"role": "system", "content": "Extract entities. Return JSON: {entities: [{name, type, description}]}"},
            {"role": "user", "content": query},
        ],
    )
    extracted_raw = response.choices[0].message.content
    parsed = parse_structured_output(raw_text=extracted_raw, response_model=response_model)
    waxell.metadata("validation_passed", parsed["valid"])

Quality Assessment with @reasoning​

The evaluator assesses extraction completeness using chain-of-thought reasoning.

@waxell.reasoning_dec(step="extraction_quality_assessment")
async def evaluate_extraction_quality(entities: list, sentiment: dict, query: str) -> dict:
    entity_count = len(entities) if isinstance(entities, list) else 0
    confidence = sentiment.get("confidence", 0)
    return {
        "thought": f"Extracted {entity_count} entities with sentiment confidence {confidence}.",
        "evidence": [f"Entity count: {entity_count}", f"Confidence: {confidence}"],
        "conclusion": "High quality extraction" if entity_count >= 2 and confidence > 0.7 else "Needs refinement",
    }

waxell.score("extraction_quality", 0.88, comment="Based on entity count and sentiment confidence")
waxell.score("validation_pass", True, data_type="boolean")

What this demonstrates​

  • Instructor-style extraction -- LLM output parsed and validated against Pydantic response models (EntityExtraction, SentimentResult).
  • @tool for structured parsing -- parse_structured_output validates raw LLM text against a response model.
  • @decision for model selection -- LLM-powered choice between EntityExtraction, RelationExtraction, and EventExtraction.
  • @reasoning for quality assessment -- chain-of-thought evaluation of extraction completeness.
  • Multi-agent extraction pipeline -- runner handles extraction, evaluator handles sentiment and quality scoring.

Run it​

# Dry-run mode (no API key needed)
cd dev/waxell-dev
python -m app.demos.instructor_agent --dry-run

# Live mode
export OPENAI_API_KEY="sk-..."
python -m app.demos.instructor_agent

Source​

dev/waxell-dev/app/demos/instructor_agent.py