skill
fine-tuning-with-trl Review findings
orchestra-researchorchestra-research/ai-research-skillsupdated
Fine-tune LLMs using reinforcement learning with TRL - SFT for instruction tuning, DPO for preference alignment, PPO/GRPO for reward optimization, and reward model training. Use when need RLHF, align model with preferences, or train from human feedback. Works with HuggingFace Transformers.
- Stars
- 14,234
- Version
- v1
- Copies
- 0
Overview
Waxell's checks found 1 finding to review, highest severity high.
- Highest severity
- high
- Findings
- 1
- What ran
- Static analysis
A clean result means nothing alarming in what is published, not a guarantee. How the checks work
We don't republish the skill's instructions. Read them on GitHub at the reviewed commit.
Security
Security scan
Checked 2 days ago
Result
Review findings Waxell's checks found 1 finding to review, highest severity high.
Checked with static analysis: servers it launches, packages it installs at runtime, scripts piped to a shell and bundled binaries. Nothing was run. How the checks work
Findings (1)
high · 1
- high
Tells Claude to install unpinned packages (trl, transformers, datasets, peft, accelerate)
SKILL.md:19 · skill_risk:S2
Versions
Version history
Each distinct content we have seen, newest first, with the verdict it got at the time.
One version so far: v1 (1.0.0), seen 2026-11-03.
See and control which skills your people use
Inside your organization, Waxell scans each skill and plugin before anyone installs it, approves what passes and publishes it to a private marketplace, and puts every MCP tool call behind one governed URL.
30 minutes. No commitment.
SOC 2 Type II attested · ISO/IEC 27001:2022 certified · HIPAA with a BAA · aligned to the EU AI Act and NIST AI RMF
More research skills and plugins
- verl-rl-trainingorchestra-researchProvides guidance for training LLMs with reinforcement learning using verl (Volcano Engine RL). Use when implementing RLHF, GRPO, PPO, or… Review findingsResearchAI & agents14k
- constitutional-aiorchestra-researchAnthropic's method for training harmless AI through self-improvement. Two-phase approach - supervised learning with… No issuesResearchAI & agents14k
- llamaguardorchestra-researchMeta's 7-8B specialized moderation model for LLM input/output filtering. 6 safety categories - violence/hate, sexual content, weapons,… Review findingsResearchAI & agents14k
- nemo-guardrailsorchestra-researchNVIDIA's runtime safety framework for LLM applications. Features jailbreak detection, input/output validation, fact-checking,… Review findingsResearchAI & agents14k