Waxell

skill

agent-observability-build-eval-from-annotations High risk

datadog-labsdatadog-labs/agent-skillsupdated

Fit a Datadog LLM-Obs evaluator to human labels. Takes an annotation queue, works out where in the trace the labelled property actually lives, drafts an LLM-judge that predicts the human label, scores that judge against the already-labelled rows with a metric agreed with the user, then hill-climbs it, inspect the errors, make one focused change, re-score, keep it only if it beats the best, for a bounded number of iterations, and finally publishes the winner to Datadog as a DISABLED evaluator (not a Datadog draft, a real evaluator with `enabled: false`). Use when the user says "build an…
datadog-labs's own description, from the skill's files.
Stars
185
Version
v1
Copies
0

Overview

blockHigh risk

Waxell rates it high risk: 5 findings, highest severity high.

Highest severity
high
Findings
5
What ran
Static analysis + deep scan

A clean result means nothing alarming in what is published, not a guarantee. How the checks work

We don't republish the skill's instructions. Read them on GitHub at the reviewed commit.

Security

Security scan

Checked yesterday

Result

High risk Waxell rates it high risk: 5 findings, highest severity high.

Checked with static analysis + deep scan: servers it launches, packages it installs at runtime, scripts piped to a shell and bundled binaries. The deep scan also read the instructions for prompt injection and data exfiltration. Nothing was run. How the checks work

Findings (5)

high · 1

  • high

    Excessive Agency: External Model or Provider Selection

    references/judge_runner.py:8 · skillspector:EA5

medium · 4

  • medium

    Without declared permissions the skill's intent is opaque and cannot be validated.

    SKILL.md:1 · skillspector:LP3

  • medium

    Dangerous Code Execution: subprocess module call

    references/judge_runner.py:97 · skillspector:AST4

  • medium

    Data Exfiltration: External Transmission

    references/judge_runner.py:129 · skillspector:E1

  • medium

    Dangerous Code Execution: subprocess module call

    references/judge_runner.py:168 · skillspector:AST4

Versions

Version history

Each distinct content we have seen, newest first, with the verdict it got at the time.

One version so far: v1, seen 2026-11-16.

See and control which skills your people use

Inside your organization, Waxell scans each skill and plugin before anyone installs it, approves what passes and publishes it to a private marketplace, and puts every MCP tool call behind one governed URL.

30 minutes. No commitment.

SOC 2 Type II attested · ISO/IEC 27001:2022 certified · HIPAA with a BAA · aligned to the EU AI Act and NIST AI RMF

More ai & agents skills and plugins

See all