REST API Reference

All Observe API endpoints live under /api/v1/observe/ on your Waxell control plane.

Authentication

Every request must include one of:

• API key header: X-Wax-Key: wax_sk_...
• Bearer token: Authorization: Bearer wax_sk_...

info

Endpoints under /api/v1/observe/ use API key authentication (for SDK/programmatic access).

Endpoints under /api/v1/observability/, /api/v1/evaluations/, and /api/v1/prompts/ use session authentication (for the web UI). These require an authenticated user session (cookie-based or JWT).

---

Runs

Start a Run

Start a new execution run for an agent.

POST /api/v1/observe/runs/start/

Request Body:

{
  "agent_name": "my-agent",
  "workflow_name": "default",
  "inputs": {},
  "metadata": {},
  "trace_id": ""
}

Field	Type	Required	Description
agent_name	string	Yes	Name of the agent
workflow_name	string	No	Workflow name (default: "default")
inputs	object	No	Input data for this run
metadata	object	No	Arbitrary metadata
trace_id	string	No	External trace ID for correlation

Response:

{
  "run_id": "run_abc123",
  "workflow_id": "wf_xyz789",
  "started_at": "2025-01-15T10:30:00Z"
}

Example:

curl -X POST "https://acme.waxell.dev/api/v1/observe/runs/start/" \
  -H "X-Wax-Key: wax_sk_..." \
  -H "Content-Type: application/json" \
  -d '{
    "agent_name": "support-bot",
    "workflow_name": "handle-ticket",
    "inputs": {"query": "How do I reset my password?"}
  }'

---

Complete a Run

Mark an execution run as completed.

POST /api/v1/observe/runs/{run_id}/complete/

Request Body:

{
  "result": {},
  "status": "success",
  "error": "",
  "steps": []
}

Field	Type	Required	Description
result	object	No	Result data from the run
status	string	No	"success" or "error" (default: "success")
error	string	No	Error message if the run failed
steps	array	No	Additional steps to record with completion

Response:

{
  "status": "ok"
}

Example:

curl -X POST "https://acme.waxell.dev/api/v1/observe/runs/run_abc123/complete/" \
  -H "X-Wax-Key: wax_sk_..." \
  -H "Content-Type: application/json" \
  -d '{
    "result": {"answer": "Click Forgot Password on the login page."},
    "status": "success"
  }'

---

Record LLM Calls

Record one or more LLM API calls for a run.

POST /api/v1/observe/runs/{run_id}/llm-calls/

Request Body:

{
  "calls": [
    {
      "model": "gpt-4o",
      "tokens_in": 150,
      "tokens_out": 80,
      "cost": 0.001175,
      "task": "answer_question",
      "prompt_preview": "How do I reset...",
      "response_preview": "Click Forgot Password..."
    }
  ]
}

Field	Type	Required	Description
calls	array	Yes	Array of LLM call objects
calls[].model	string	Yes	Model identifier
calls[].tokens_in	integer	Yes	Input token count
calls[].tokens_out	integer	Yes	Output token count
calls[].cost	number	No	Cost in USD
calls[].task	string	No	Label for this call
calls[].prompt_preview	string	No	Preview of the prompt
calls[].response_preview	string	No	Preview of the response

Response:

{
  "status": "ok",
  "count": 1
}

Example:

curl -X POST "https://acme.waxell.dev/api/v1/observe/runs/run_abc123/llm-calls/" \
  -H "X-Wax-Key: wax_sk_..." \
  -H "Content-Type: application/json" \
  -d '{
    "calls": [
      {
        "model": "gpt-4o",
        "tokens_in": 150,
        "tokens_out": 80,
        "cost": 0.001175,
        "task": "answer_question"
      }
    ]
  }'

---

Record Steps

Record execution steps for a run.

POST /api/v1/observe/runs/{run_id}/steps/

Request Body:

{
  "steps": [
    {
      "step_name": "retrieve_context",
      "output": {"doc_count": 5}
    },
    {
      "step_name": "generate_answer",
      "output": {"length": 142}
    }
  ]
}

Field	Type	Required	Description
steps	array	Yes	Array of step objects
steps[].step_name	string	Yes	Name identifying the step
steps[].output	object	No	Output data for the step

Response:

{
  "status": "ok",
  "count": 2
}

Example:

curl -X POST "https://acme.waxell.dev/api/v1/observe/runs/run_abc123/steps/" \
  -H "X-Wax-Key: wax_sk_..." \
  -H "Content-Type: application/json" \
  -d '{
    "steps": [
      {"step_name": "retrieve_context", "output": {"doc_count": 5}},
      {"step_name": "generate_answer", "output": {"length": 142}}
    ]
  }'

---

Record Scores (SDK)

Record scores (user feedback, quality metrics) for a run from the SDK.

POST /api/v1/observe/runs/{run_id}/scores/

Authentication: API key

Request Body:

{
  "scores": [
    {
      "name": "accuracy",
      "data_type": "numeric",
      "numeric_value": 0.95,
      "comment": "Very accurate response"
    },
    {
      "name": "thumbs_up",
      "data_type": "boolean",
      "numeric_value": 1.0,
      "string_value": "true"
    }
  ]
}

Field	Type	Required	Description
scores	array	Yes	Array of score objects
scores[].name	string	Yes	Score name (e.g. "accuracy", "thumbs_up")
scores[].data_type	string	No	One of "numeric", "categorical", "boolean" (default: "numeric")
scores[].numeric_value	number	No	Numeric score value
scores[].string_value	string	No	String score value (for categorical/boolean)
scores[].comment	string	No	Free-text comment

Response:

{
  "recorded": 2
}

Example:

curl -X POST "https://acme.waxell.dev/api/v1/observe/runs/42/scores/" \
  -H "X-Wax-Key: wax_sk_..." \
  -H "Content-Type: application/json" \
  -d '{
    "scores": [
      {"name": "accuracy", "data_type": "numeric", "numeric_value": 0.95},
      {"name": "thumbs_up", "data_type": "boolean", "numeric_value": 1.0}
    ]
  }'

---

Fetch Prompt (SDK)

Retrieve a managed prompt for use in your agent code. Supports fetching by label, version number, or latest.

GET /api/v1/observe/prompts/{name}/

Authentication: API key

Query Parameters:

Parameter	Type	Required	Description
label	string	No	Fetch the version this label points to (e.g. "production")
version	integer	No	Fetch an exact version number. Takes precedence over label

If neither label nor version is provided, the latest version (highest version number) is returned.

Response:

{
  "name": "support-prompt",
  "version": 3,
  "prompt_type": "chat",
  "content": [
    {"role": "system", "content": "You are a helpful assistant for {{company}}."},
    {"role": "user", "content": "{{user_query}}"}
  ],
  "config": {"model": "gpt-4o", "temperature": 0.7},
  "labels": ["production"]
}

Example:

# Fetch by label
curl "https://acme.waxell.dev/api/v1/observe/prompts/support-prompt/?label=production" \
  -H "X-Wax-Key: wax_sk_..."

# Fetch specific version
curl "https://acme.waxell.dev/api/v1/observe/prompts/support-prompt/?version=2" \
  -H "X-Wax-Key: wax_sk_..."

# Fetch latest
curl "https://acme.waxell.dev/api/v1/observe/prompts/support-prompt/" \
  -H "X-Wax-Key: wax_sk_..."

---

Record Behavior Spans

Record behavior tracking spans (tool calls, retrievals, decisions, reasoning, retries) for a run.

POST /api/v1/observe/runs/{run_id}/spans/

Request Body:

{
  "spans": [
    {
      "name": "web_search",
      "kind": "tool",
      "status": "ok",
      "duration_ms": 250,
      "position": 1,
      "attributes": {"tool_type": "api"},
      "input_data": {"query": "latest news"},
      "output_data": {"results": ["..."]}
    },
    {
      "name": "retrieve:pinecone",
      "kind": "retriever",
      "status": "ok",
      "duration_ms": 120,
      "position": 2,
      "attributes": {"source": "pinecone", "doc_count": 5, "top_k": 10},
      "input_data": {"query": "billing FAQ", "top_k": 10},
      "output_data": {"documents": ["..."]}
    },
    {
      "name": "route_to_agent",
      "kind": "decision",
      "status": "ok",
      "duration_ms": null,
      "position": 3,
      "attributes": {"confidence": 0.95},
      "input_data": {"options": ["billing", "technical"], "reasoning": "User mentioned invoice"},
      "output_data": {"chosen": "billing", "confidence": 0.95}
    }
  ]
}

Field	Type	Required	Description
spans	array	Yes	Array of span objects
spans[].name	string	Yes	Span name
spans[].kind	string	Yes	One of: "tool", "retriever", "decision", "reasoning", "retry"
spans[].status	string	No	"ok" or "error" (default: "ok")
spans[].duration_ms	integer	No	Duration in milliseconds
spans[].position	integer	No	Ordering position within the run
spans[].attributes	object	No	Kind-specific attributes
spans[].input_data	object	No	Input data for this span
spans[].output_data	object	No	Output data for this span

Response:

{
  "status": "ok",
  "count": 3
}

Example:

curl -X POST "https://acme.waxell.dev/api/v1/observe/runs/run_abc123/spans/" \
  -H "X-Wax-Key: wax_sk_..." \
  -H "Content-Type: application/json" \
  -d '{
    "spans": [
      {
        "name": "web_search",
        "kind": "tool",
        "status": "ok",
        "duration_ms": 250,
        "attributes": {"tool_type": "api"},
        "input_data": {"query": "latest news"}
      }
    ]
  }'

---

Policy

Check Policy

Check whether an agent is allowed to execute.

POST /api/v1/observe/policy-check/

Request Body:

{
  "agent_name": "my-agent",
  "workflow_name": "default",
  "agent_id": ""
}

Field	Type	Required	Description
agent_name	string	Yes	Name of the agent
workflow_name	string	No	Workflow name for scoped policies
agent_id	string	No	Specific agent instance ID

Response:

{
  "action": "allow",
  "reason": "",
  "metadata": {}
}

Field	Type	Description
action	string	One of: "allow", "block", "warn", "throttle", "redact", "skip", "retry"
reason	string	Human-readable explanation
metadata	object	Additional data (e.g., budget info)

Example:

curl -X POST "https://acme.waxell.dev/api/v1/observe/policy-check/" \
  -H "X-Wax-Key: wax_sk_..." \
  -H "Content-Type: application/json" \
  -d '{
    "agent_name": "support-bot",
    "workflow_name": "handle-ticket"
  }'

---

Events

Record Events

Record governance or audit events.

POST /api/v1/observe/events/

Request Body:

{
  "events": [
    {
      "event_type": "policy_check",
      "agent_name": "my-agent",
      "action": "allow",
      "timestamp": "2025-01-15T10:30:00Z"
    }
  ]
}

Field	Type	Required	Description
events	array	Yes	Array of event objects

Event objects are flexible -- include any fields relevant to your audit requirements.

Response:

{
  "status": "ok",
  "count": 1
}

Example:

curl -X POST "https://acme.waxell.dev/api/v1/observe/events/" \
  -H "X-Wax-Key: wax_sk_..." \
  -H "Content-Type: application/json" \
  -d '{
    "events": [
      {
        "event_type": "manual_override",
        "agent_name": "support-bot",
        "operator": "admin@acme.com",
        "reason": "Emergency bypass"
      }
    ]
  }'

---

Model Costs

Get Model Costs

Retrieve the merged system + tenant model cost table.

GET /api/v1/observe/model-costs/

Response:

Returns the complete model cost table with tenant overrides applied on top of system defaults.

Example:

curl "https://acme.waxell.dev/api/v1/observe/model-costs/" \
  -H "X-Wax-Key: wax_sk_..."

---

Set Model Cost Override

Set a tenant-level cost override for a specific model.

PUT /api/v1/observe/model-costs/{model_id}/

Request Body:

{
  "input_cost_per_million": 2.00,
  "output_cost_per_million": 8.00
}

Field	Type	Required	Description
input_cost_per_million	number	Yes	Input token cost per 1M tokens (USD)
output_cost_per_million	number	Yes	Output token cost per 1M tokens (USD)

Example:

curl -X PUT "https://acme.waxell.dev/api/v1/observe/model-costs/gpt-4o/" \
  -H "X-Wax-Key: wax_sk_..." \
  -H "Content-Type: application/json" \
  -d '{
    "input_cost_per_million": 2.00,
    "output_cost_per_million": 8.00
  }'

---

Delete Model Cost Override

Remove a tenant-level cost override, reverting to system defaults.

DELETE /api/v1/observe/model-costs/{model_id}/

Example:

curl -X DELETE "https://acme.waxell.dev/api/v1/observe/model-costs/gpt-4o/" \
  -H "X-Wax-Key: wax_sk_..."

---

Observability Endpoints (Session Auth)

These endpoints power the Observe web UI. They require an authenticated user session.

List LLM Calls

Search and filter LLM calls across all runs.

GET /api/v1/observability/llm-calls/

Authentication: Session

Query Parameters:

Parameter	Type	Description
search	string	Full-text search across model, task, prompt/response previews
model	string	Filter by model name (supports multiple via repeated param)
agent	string	Filter by agent name
min_cost	number	Minimum cost filter
max_cost	number	Maximum cost filter
min_tokens	integer	Minimum total token count
max_tokens	integer	Maximum total token count
start	string	Start date/time (ISO 8601)
end	string	End date/time (ISO 8601)
sort	string	Sort field. One of: created_at, -created_at, cost, -cost, total_tokens, -total_tokens, model, -model (default: -created_at)
limit	integer	Page size (default: 25, max: 100)
offset	integer	Pagination offset (default: 0)

Response:

{
  "results": [
    {
      "id": 42,
      "run_id": 17,
      "model": "gpt-4o",
      "task": "answer_question",
      "agent_name": "support-bot",
      "workflow_name": "handle-ticket",
      "tokens_in": 150,
      "tokens_out": 80,
      "total_tokens": 230,
      "cost": 0.001175,
      "created_at": "2025-01-15T10:30:00Z",
      "prompt_preview": "How do I reset...",
      "response_preview": "Click Forgot Password...",
      "prompt_hash": "abc123..."
    }
  ],
  "count": 1,
  "next": null,
  "previous": null,
  "aggregates": {
    "total_cost": 0.001175,
    "total_tokens": 230,
    "total_calls": 1,
    "models_used": ["gpt-4o"]
  }
}

Example:

curl "https://acme.waxell.dev/api/v1/observability/llm-calls/?model=gpt-4o&limit=10" \
  -H "Cookie: sessionid=..."

---

Get LLM Call Detail

Retrieve full details for a single LLM call.

GET /api/v1/observability/llm-calls/{call_id}/

Authentication: Session

Response:

{
  "id": 42,
  "run_id": 17,
  "model": "gpt-4o",
  "task": "answer_question",
  "agent_name": "support-bot",
  "workflow_name": "handle-ticket",
  "tokens_in": 150,
  "tokens_out": 80,
  "total_tokens": 230,
  "cost": 0.001175,
  "created_at": "2025-01-15T10:30:00Z",
  "prompt_preview": "Full prompt text...",
  "response_preview": "Full response text...",
  "prompt_hash": "abc123..."
}

Example:

curl "https://acme.waxell.dev/api/v1/observability/llm-calls/42/" \
  -H "Cookie: sessionid=..."

---

List Distinct Models

Get the distinct model names used across all LLM calls. Useful for populating filter dropdowns.

GET /api/v1/observability/llm-calls/models/

Authentication: Session

Response:

{
  "models": ["claude-3-5-sonnet", "gpt-4o", "gpt-4o-mini"]
}

Example:

curl "https://acme.waxell.dev/api/v1/observability/llm-calls/models/" \
  -H "Cookie: sessionid=..."

---

List Sessions

List sessions with aggregated metrics. A session groups multiple runs that share a session_id.

GET /api/v1/observability/sessions/

Authentication: Session

Query Parameters:

Parameter	Type	Description
search	string	Search by session ID
agent	string	Filter by agent name
start	string	Start date/time (ISO 8601)
end	string	End date/time (ISO 8601)
sort	string	One of: last_activity, -last_activity, first_run, -first_run, run_count, -run_count (default: -last_activity)
limit	integer	Page size (default: 25, max: 100)
offset	integer	Pagination offset (default: 0)

Response:

{
  "results": [
    {
      "session_id": "sess_a1b2c3d4e5f6g7h8",
      "run_count": 5,
      "first_run": "2025-01-15T10:00:00Z",
      "last_activity": "2025-01-15T10:15:00Z",
      "total_duration": 45.5,
      "total_cost": 0.0523,
      "total_tokens": 12500,
      "agents": ["support-bot", "research-agent"]
    }
  ],
  "count": 1,
  "next": null,
  "previous": null
}

Example:

curl "https://acme.waxell.dev/api/v1/observability/sessions/?agent=support-bot" \
  -H "Cookie: sessionid=..."

---

Get Session Detail

Get full session details including all runs and aggregated metrics.

GET /api/v1/observability/sessions/{session_id}/

Authentication: Session

Response:

{
  "session_id": "sess_a1b2c3d4e5f6g7h8",
  "aggregates": {
    "run_count": 3,
    "total_duration": 15.2,
    "total_cost": 0.0234,
    "total_tokens": 5600,
    "agents": ["support-bot"]
  },
  "runs": [
    {
      "id": 17,
      "agent_name": "support-bot",
      "workflow_name": "handle-ticket",
      "started_at": "2025-01-15T10:00:00Z",
      "completed_at": "2025-01-15T10:00:05Z",
      "duration": 5.1,
      "status": "success",
      "cost": 0.0078,
      "tokens": 1800,
      "trace_id": ""
    }
  ]
}

Example:

curl "https://acme.waxell.dev/api/v1/observability/sessions/sess_a1b2c3d4e5f6g7h8/" \
  -H "Cookie: sessionid=..."

---

List Users

List tracked end-users with aggregate metrics. Users are identified by the user_id passed during run creation.

GET /api/v1/observability/users/

Authentication: Session

Query Parameters:

Parameter	Type	Description
search	string	Search by user ID
agent	string	Filter by agent name
start	string	Start date/time (ISO 8601)
end	string	End date/time (ISO 8601)
sort	string	One of: last_seen, -last_seen, first_seen, -first_seen, run_count, -run_count (default: -last_seen)
limit	integer	Page size (default: 25, max: 100)
offset	integer	Pagination offset (default: 0)

Response:

{
  "results": [
    {
      "user_id": "user_abc123",
      "run_count": 42,
      "first_seen": "2025-01-01T08:00:00Z",
      "last_seen": "2025-01-15T10:30:00Z",
      "total_duration": 210.5,
      "total_cost": 0.5234,
      "total_tokens": 125000,
      "agents": ["support-bot", "research-agent"]
    }
  ],
  "count": 1,
  "next": null,
  "previous": null
}

Example:

curl "https://acme.waxell.dev/api/v1/observability/users/?sort=-run_count" \
  -H "Cookie: sessionid=..."

---

Get User Detail

Get detailed information for a specific tracked user, including cost breakdown by model and recent runs.

GET /api/v1/observability/users/{user_id}/

Authentication: Session

Response:

{
  "user_id": "user_abc123",
  "aggregates": {
    "run_count": 42,
    "total_duration": 210.5,
    "total_cost": 0.5234,
    "total_tokens": 125000,
    "agents": ["support-bot"],
    "first_seen": "2025-01-01T08:00:00Z",
    "last_seen": "2025-01-15T10:30:00Z"
  },
  "cost_by_model": [
    {
      "model": "gpt-4o",
      "total_cost": 0.45,
      "total_tokens": 100000,
      "call_count": 30
    }
  ],
  "runs": [
    {
      "id": 17,
      "agent_name": "support-bot",
      "workflow_name": "handle-ticket",
      "started_at": "2025-01-15T10:30:00Z",
      "completed_at": "2025-01-15T10:30:05Z",
      "duration": 5.1,
      "status": "success",
      "cost": 0.0078,
      "tokens": 1800
    }
  ]
}

Example:

curl "https://acme.waxell.dev/api/v1/observability/users/user_abc123/" \
  -H "Cookie: sessionid=..."

---

Model Analytics

Get model usage analytics with time-series data for cost, token, and call-count trends.

GET /api/v1/observability/analytics/models/

Authentication: Session

Query Parameters:

Parameter	Type	Description
period	string	Time period: "1d", "7d", or "30d" (default: "7d")
agent	string	Filter by agent name

Response:

{
  "period": "7d",
  "aggregates": {
    "total_cost": 12.345,
    "total_tokens": 2500000,
    "total_calls": 1500
  },
  "model_totals": [
    {
      "model": "gpt-4o",
      "total_cost": 10.5,
      "total_tokens": 2000000,
      "total_calls": 1200,
      "percentage": 85.1
    }
  ],
  "time_series": [
    {
      "date": "2025-01-15",
      "model": "gpt-4o",
      "cost": 1.5,
      "tokens": 300000,
      "calls": 180
    }
  ]
}

Example:

curl "https://acme.waxell.dev/api/v1/observability/analytics/models/?period=30d" \
  -H "Cookie: sessionid=..."

---

Scoring Endpoints (Session Auth)

Endpoints for creating and querying quality scores.

List Scores

GET /api/v1/evaluations/scores/

Authentication: Session

Query Parameters:

Parameter	Type	Description
name	string	Filter by score name
source	string	Filter by source ("manual", "sdk", "evaluator")
data_type	string	Filter by data type ("numeric", "categorical", "boolean")
run_id	integer	Filter by run ID
llm_call_id	integer	Filter by LLM call ID
sort	string	Sort field (default: -created_at). Allowed: created_at, -created_at, name, -name, numeric_value, -numeric_value, source, -source
limit	integer	Page size (default: 25, max: 100)
offset	integer	Pagination offset

Response:

{
  "results": [
    {
      "id": "uuid",
      "name": "accuracy",
      "data_type": "numeric",
      "source": "manual",
      "numeric_value": 0.95,
      "string_value": null,
      "comment": "Very accurate",
      "metadata": {},
      "author_user_id": "1",
      "evaluator_id": null,
      "evaluator_name": null,
      "run_id": "17",
      "run_agent_name": "support-bot",
      "llm_call_id": null,
      "llm_call_model": null,
      "created_at": "2025-01-15T10:30:00Z"
    }
  ],
  "count": 1,
  "next": null,
  "previous": null,
  "aggregates": {
    "total_count": 1,
    "avg_numeric_value": 0.95,
    "score_names": ["accuracy"]
  }
}

Example:

curl "https://acme.waxell.dev/api/v1/evaluations/scores/?name=accuracy&sort=-numeric_value" \
  -H "Cookie: sessionid=..."

---

Create Score

Create a manual score for a run or LLM call.

POST /api/v1/evaluations/scores/create/

Authentication: Session

Request Body:

{
  "run_id": 17,
  "name": "accuracy",
  "data_type": "numeric",
  "value": 0.95,
  "comment": "Very accurate response",
  "metadata": {}
}

Field	Type	Required	Description
name	string	Yes	Score name
run_id	integer	Conditional	Run to score (at least one of run_id or llm_call_id required)
llm_call_id	integer	Conditional	LLM call to score
data_type	string	No	One of "numeric", "categorical", "boolean" (default: "numeric")
value	any	Yes	Score value (number for numeric, string for categorical, bool for boolean)
comment	string	No	Free-text comment
metadata	object	No	Arbitrary metadata

Response: 201 Created

{
  "id": "uuid",
  "name": "accuracy",
  "data_type": "numeric",
  "source": "manual",
  "numeric_value": 0.95,
  "string_value": null,
  "run_id": "17",
  "llm_call_id": null,
  "created_at": "2025-01-15T10:30:00Z"
}

Example:

curl -X POST "https://acme.waxell.dev/api/v1/evaluations/scores/create/" \
  -H "Cookie: sessionid=..." \
  -H "Content-Type: application/json" \
  -d '{
    "run_id": 17,
    "name": "accuracy",
    "data_type": "numeric",
    "value": 0.95,
    "comment": "Very accurate response"
  }'

---

Score Analytics

Get score distributions and time-series trends.

GET /api/v1/evaluations/scores/analytics/

Authentication: Session

Query Parameters:

Parameter	Type	Description
name	string	Filter by score name
period	string	Time period: "1d", "7d", "30d" (default: "7d")
agent	string	Filter by agent name

Response:

{
  "period": "7d",
  "since": "2025-01-08T10:30:00Z",
  "distributions": [
    {
      "name": "accuracy",
      "data_type": "numeric",
      "total_count": 150,
      "avg": 0.87,
      "min": 0.2,
      "max": 1.0,
      "time_series": [
        {"date": "2025-01-15", "avg": 0.89, "count": 25}
      ]
    },
    {
      "name": "sentiment",
      "data_type": "categorical",
      "total_count": 80,
      "value_counts": [
        {"value": "positive", "count": 50},
        {"value": "neutral", "count": 20},
        {"value": "negative", "count": 10}
      ]
    }
  ]
}

Example:

curl "https://acme.waxell.dev/api/v1/evaluations/scores/analytics/?name=accuracy&period=30d" \
  -H "Cookie: sessionid=..."

---

Prompt Endpoints (Session Auth)

Endpoints for managing versioned prompts, labels, and the playground.

List Prompts

GET /api/v1/prompts/

Authentication: Session

Response:

{
  "results": [
    {
      "id": "uuid",
      "name": "support-prompt",
      "description": "Main support agent prompt",
      "prompt_type": "chat",
      "tags": ["support", "production"],
      "latest_version": 3,
      "version_count": 3,
      "labels": ["production", "staging"],
      "created_at": "2025-01-10T08:00:00Z",
      "updated_at": "2025-01-15T10:00:00Z"
    }
  ],
  "count": 1
}

Example:

curl "https://acme.waxell.dev/api/v1/prompts/" \
  -H "Cookie: sessionid=..."

---

Create Prompt

Create a new prompt with an initial version (version 1).

POST /api/v1/prompts/

Authentication: Session

Request Body:

{
  "name": "support-prompt",
  "description": "Main support agent prompt",
  "prompt_type": "chat",
  "content": [
    {"role": "system", "content": "You are a helpful assistant for {{company}}."},
    {"role": "user", "content": "{{user_query}}"}
  ],
  "config": {"model": "gpt-4o", "temperature": 0.7},
  "tags": ["support"],
  "commit_message": "Initial version"
}

Field	Type	Required	Description
name	string	Yes	Unique prompt name
content	any	Yes	Prompt content. String for text prompts, array of message objects for chat prompts
description	string	No	Description
prompt_type	string	No	"text" or "chat" (default: "text")
config	object	No	Default config (model, temperature, max_tokens)
tags	array	No	Searchable tags
metadata	object	No	Arbitrary metadata
commit_message	string	No	Version commit message (default: "Initial version")

Response: 201 Created

Example:

curl -X POST "https://acme.waxell.dev/api/v1/prompts/" \
  -H "Cookie: sessionid=..." \
  -H "Content-Type: application/json" \
  -d '{
    "name": "support-prompt",
    "prompt_type": "chat",
    "content": [{"role": "system", "content": "You are a helpful assistant."}],
    "config": {"model": "gpt-4o"}
  }'

---

Get/Update/Delete Prompt

GET    /api/v1/prompts/{prompt_id}/
PUT    /api/v1/prompts/{prompt_id}/
DELETE /api/v1/prompts/{prompt_id}/

Authentication: Session

• GET returns prompt with all versions and labels
• PUT updates metadata fields only (name, description, tags). To update content, create a new version
• DELETE permanently deletes the prompt and all versions/labels

Example:

# Get prompt detail with all versions
curl "https://acme.waxell.dev/api/v1/prompts/uuid-here/" \
  -H "Cookie: sessionid=..."

# Update metadata
curl -X PUT "https://acme.waxell.dev/api/v1/prompts/uuid-here/" \
  -H "Cookie: sessionid=..." \
  -H "Content-Type: application/json" \
  -d '{"description": "Updated description", "tags": ["support", "v2"]}'

---

List Versions

GET /api/v1/prompts/{prompt_id}/versions/

Authentication: Session

Response:

{
  "results": [
    {
      "version": 3,
      "content_preview": "[{\"role\": \"system\", \"content\": \"You are...",
      "config": {"model": "gpt-4o"},
      "content_hash": "sha256...",
      "commit_message": "Improved system prompt",
      "created_by": "admin@acme.com",
      "labels": ["production"],
      "created_at": "2025-01-15T10:00:00Z"
    }
  ],
  "count": 3
}

Example:

curl "https://acme.waxell.dev/api/v1/prompts/uuid-here/versions/" \
  -H "Cookie: sessionid=..."

---

Create Version

Create a new version of a prompt. The version number auto-increments.

POST /api/v1/prompts/{prompt_id}/versions/

Authentication: Session

Request Body:

{
  "content": [
    {"role": "system", "content": "Updated system prompt for {{company}}."},
    {"role": "user", "content": "{{user_query}}"}
  ],
  "config": {"model": "gpt-4o", "temperature": 0.5},
  "commit_message": "Improved system prompt tone"
}

Field	Type	Required	Description
content	any	Yes	Prompt content
config	object	No	Config overrides for this version
commit_message	string	No	Commit message

Response: 201 Created

{
  "version": 4,
  "content_hash": "sha256...",
  "commit_message": "Improved system prompt tone",
  "created_by": "admin@acme.com",
  "created_at": "2025-01-15T10:30:00Z"
}

Example:

curl -X POST "https://acme.waxell.dev/api/v1/prompts/uuid-here/versions/" \
  -H "Cookie: sessionid=..." \
  -H "Content-Type: application/json" \
  -d '{"content": "Updated prompt text", "commit_message": "Simplified wording"}'

---

Get Version Detail

GET /api/v1/prompts/{prompt_id}/versions/{version_num}/

Authentication: Session

Response:

{
  "prompt_id": "uuid",
  "prompt_name": "support-prompt",
  "prompt_type": "chat",
  "version": 3,
  "content": [...],
  "config": {"model": "gpt-4o"},
  "content_hash": "sha256...",
  "commit_message": "Improved system prompt",
  "created_by": "admin@acme.com",
  "labels": ["production"],
  "created_at": "2025-01-15T10:00:00Z"
}

Example:

curl "https://acme.waxell.dev/api/v1/prompts/uuid-here/versions/3/" \
  -H "Cookie: sessionid=..."

---

Set/Remove Label

Labels are movable pointers to specific versions (e.g., "production", "staging").

PUT    /api/v1/prompts/{prompt_id}/labels/{label}/
DELETE /api/v1/prompts/{prompt_id}/labels/{label}/

Authentication: Session

PUT Request Body:

{
  "version": 3
}

Field	Type	Required	Description
version	integer	Yes	Version number to point the label to

PUT Response:

{
  "label": "production",
  "version": 3,
  "prompt_id": "uuid",
  "created": false
}

Example:

# Set production label to version 3
curl -X PUT "https://acme.waxell.dev/api/v1/prompts/uuid-here/labels/production/" \
  -H "Cookie: sessionid=..." \
  -H "Content-Type: application/json" \
  -d '{"version": 3}'

# Remove staging label
curl -X DELETE "https://acme.waxell.dev/api/v1/prompts/uuid-here/labels/staging/" \
  -H "Cookie: sessionid=..."

---

Playground: Execute Prompt

Execute a prompt against an LLM with variable substitution.

POST /api/v1/prompts/playground/

Authentication: Session

Request Body:

{
  "content": [
    {"role": "system", "content": "You help users of {{company}}."},
    {"role": "user", "content": "{{query}}"}
  ],
  "config": {
    "model": "gpt-4o-mini",
    "temperature": 0.7,
    "max_tokens": 1024
  },
  "variables": {
    "company": "Acme Corp",
    "query": "How do I reset my password?"
  }
}

Field	Type	Required	Description
content	any	Yes	Prompt content (string or chat messages)
config	object	No	LLM configuration
config.model	string	No	Model to use (default: "gpt-4o-mini")
config.temperature	number	No	Sampling temperature (default: 0.7)
config.max_tokens	integer	No	Max output tokens (default: 1024)
variables	object	No	Template variables to replace {{var}} placeholders

Response:

{
  "output": "To reset your password, click the Forgot Password link...",
  "model": "gpt-4o-mini",
  "tokens_in": 45,
  "tokens_out": 120,
  "cost": 0.000079,
  "latency_ms": 1250
}

Example:

curl -X POST "https://acme.waxell.dev/api/v1/prompts/playground/" \
  -H "Cookie: sessionid=..." \
  -H "Content-Type: application/json" \
  -d '{
    "content": "Summarize: {{text}}",
    "config": {"model": "gpt-4o-mini"},
    "variables": {"text": "The quick brown fox..."}
  }'

---

Playground: Compare Variants

Execute multiple prompt variants side-by-side for comparison.

POST /api/v1/prompts/playground/compare/

Authentication: Session

Request Body:

{
  "variants": [
    {
      "content": "Summarize concisely: {{text}}",
      "config": {"model": "gpt-4o-mini", "temperature": 0.3},
      "variables": {"text": "..."}
    },
    {
      "content": "Provide a detailed summary of: {{text}}",
      "config": {"model": "gpt-4o", "temperature": 0.7},
      "variables": {"text": "..."}
    }
  ]
}

Field	Type	Required	Description
variants	array	Yes	Array of variant objects (max 10). Each has content, config, and variables

Response:

{
  "results": [
    {"output": "...", "model": "gpt-4o-mini", "tokens_in": 30, "tokens_out": 50, "cost": 0.00003, "latency_ms": 800},
    {"output": "...", "model": "gpt-4o", "tokens_in": 35, "tokens_out": 150, "cost": 0.0016, "latency_ms": 2100}
  ]
}

Example:

curl -X POST "https://acme.waxell.dev/api/v1/prompts/playground/compare/" \
  -H "Cookie: sessionid=..." \
  -H "Content-Type: application/json" \
  -d '{
    "variants": [
      {"content": "Short: {{text}}", "config": {"model": "gpt-4o-mini"}, "variables": {"text": "hello"}},
      {"content": "Long: {{text}}", "config": {"model": "gpt-4o"}, "variables": {"text": "hello"}}
    ]
  }'

---

Prompt Metrics

Get usage metrics per prompt version, linked via content_hash to actual LLM call records.

GET /api/v1/prompts/{prompt_id}/metrics/

Authentication: Session

Response:

{
  "prompt_id": "uuid",
  "prompt_name": "support-prompt",
  "totals": {
    "call_count": 1500,
    "total_tokens": 350000,
    "total_cost": 3.45
  },
  "versions": [
    {"version": 1, "content_hash": "abc...", "call_count": 500, "total_tokens": 100000, "total_cost": 1.0},
    {"version": 2, "content_hash": "def...", "call_count": 1000, "total_tokens": 250000, "total_cost": 2.45}
  ]
}

Example:

curl "https://acme.waxell.dev/api/v1/prompts/uuid-here/metrics/" \
  -H "Cookie: sessionid=..."

---

Error Responses

API errors return standard HTTP status codes with a JSON body:

{
  "error": "Description of the error"
}

Status Code	Meaning
400	Bad request (invalid JSON, missing required fields)
401	Unauthorized (missing or invalid API key)
404	Not found (invalid run ID or model ID)
409	Conflict (duplicate name)
429	Rate limited
500	Internal server error

Next Steps

• Python SDK Reference -- Use these endpoints through the Python client
• Installation & Configuration -- Set up API credentials