skill
nemo-evaluator-sdk Review findings
orchestra-researchorchestra-research/ai-research-skillsupdated
Evaluates LLMs across 100+ benchmarks from 18+ harnesses (MMLU, HumanEval, GSM8K, safety, VLM) with multi-backend execution. Use when needing scalable evaluation on local Docker, Slurm HPC, or cloud platforms. NVIDIA's enterprise-grade platform with container-first architecture for reproducible benchmarking.
- Stars
- 14,234
- Version
- v1
- Copies
- 0
Overview
Waxell's checks found 2 findings to review, highest severity high.
- Highest severity
- high
- Findings
- 2
- What ran
- Static analysis
A clean result means nothing alarming in what is published, not a guarantee. How the checks work
We don't republish the skill's instructions. Read them on GitHub at the reviewed commit.
Security
Security scan
Checked 2 days ago
Result
Review findings Waxell's checks found 2 findings to review, highest severity high.
Checked with static analysis: servers it launches, packages it installs at runtime, scripts piped to a shell and bundled binaries. Nothing was run. How the checks work
Findings (2)
high · 2
- high
Tells Claude to install unpinned packages (nemo-evaluator-launcher)
SKILL.md:19 · skill_risk:S2
- high
Tells Claude to install unpinned packages (nemo-evaluator)
references/custom-benchmarks.md:94 · skill_risk:S2
Versions
Version history
Each distinct content we have seen, newest first, with the verdict it got at the time.
One version so far: v1 (1.0.0), seen 2026-11-03.
See and control which skills your people use
Inside your organization, Waxell scans each skill and plugin before anyone installs it, approves what passes and publishes it to a private marketplace, and puts every MCP tool call behind one governed URL.
30 minutes. No commitment.
SOC 2 Type II attested · ISO/IEC 27001:2022 certified · HIPAA with a BAA · aligned to the EU AI Act and NIST AI RMF
More engineering skills and plugins
serving-llms-vllmorchestra-researchServes LLMs with high throughput using vLLM's PagedAttention and continuous batching. Use when deploying production LLM APIs, optimizing… Review findingsEngineeringResearch14k
langchainorchestra-researchFramework for building LLM-powered applications with agents, chains, and RAG. Supports multiple providers (OpenAI, Anthropic, Google),… Review findingsEngineeringResearch14k
langsmith-observabilityorchestra-researchLLM observability platform for tracing, evaluation, and monitoring. Use when debugging LLM applications, evaluating model outputs against… Review findingsEngineeringResearch14k
model-mergingorchestra-researchMerge multiple fine-tuned models using mergekit to combine capabilities without retraining. Use when creating specialized models by… Review findingsEngineeringProductivity14k