SkillEvaluator Mastery NVIDIA SkillEvaluator

Module 6: Installing & Running SkillEvaluator

Based on the reference §4 and docs/13_NVIDIA_EVALUATOR_UPGRADE_POLICY.md.

General upstream install

# Prerequisites: uv (or pip), Python 3.12 or 3.13, git; Docker only for Tier 3
uv tool install --python 3.13 "skillevaluator[all] @ git+https://github.com/NVIDIA/SkillEvaluator.git"

skillevaluator --version
skillevaluator health-check

Smaller footprints exist: skillevaluator (Tier 1 only), [security] (+ Bandit/pip-audit), [tier2,tier3] (+ dedup and live-agent eval).

External security tools install separately to avoid dependency conflicts:

brew install gitleaks
uv tool install git+https://github.com/NVIDIA/SkillSpector.git

Why pin an exact commit

Installing @main is fine for exploration; for anything you intend to certify against, pin an exact commit. SkillEvaluator's own release notes and this repository's own experience both found real, breaking behavioral differences worth pinning against.

uv venv .venv --python 3.13
uv pip install --python .venv/bin/python \
  'skillevaluator[all] @ git+https://github.com/NVIDIA/SkillEvaluator.git@<pinned-sha>'

This repository also tracks a local patch for a real judge-truncation bug found in its pinned version (patches/skillevaluator-0.2.1-judge-max-tokens.patch) — apply it only if your pinned version predates NVIDIA's own upstream fix.

Per-tier run commands

See Modules 2-4 for the exact Tier 1/2/3 commands and their required environment variables.

Check your understanding

5 questions on this module. Get 4 right to complete it.

  1. Which two commands verify an install?
  2. Why pin an exact SkillEvaluator commit instead of installing @main?
  3. Which install extra adds deduplication and live-agent evaluation?
  4. When should you apply this repository's judge-max-tokens patch?
  5. Which prerequisite is needed only for Tier 3?