Seth HobsonCreated wshobson/agents, 41k+ GitHub stars3,200 installs on skills.shEval Harness First
Build the evaluation harness that gates every fine-tuning run : golden sets.
Use Edge to load the skill eval-harness-first from wshobson/agents for my task. If Edge is not set up, read getedge.cc/SKILL.md and ask me first.
2 static scans, 0 counted findings.
Edge checks it again each time your agent loads it.
From the wshobson public repository. Edge did not write or edit this skill.
What it does
Build the evaluation harness that gates every fine-tuning run : golden sets, per-failure-mode graders, judge calibration, and base-model baselines. Use when starting a fine-tuning effort, when converting traces into an eval set, or when calibrating a judge against human labels.
Why Seth Hobson
Founder of Major 7 Apps. Each fact links its public source.
- 41k+ GitHub stars on wshobson/agents
His plugin marketplace of agents, skills and commands for coding agents has over 41,000 stars.
GitHub - 15+ years building data systems
He has built data applications, services and distributed systems for over 15 years, mostly in finance.
GitHub profile - Founder of Major 7 Apps
He founded Major 7 Apps, an independent studio shipping developer tools and native Apple apps.
Personal site
Public sources, checked Oct 26, 2026.
How it works
- 1Copy
Try for free gives you one prompt.
- 2Paste
Paste it into Claude Code, Codex or Cursor.
- 3Ask
Ask for the outcome. Your agent loads Eval Harness First.
Related skills
Eval-driven development (EDD) framework for AI coding sessions — define capability and regression evals before coding, grade with code-based, model-based, rule, or human graders, and track pass@k and pass^k reliability.
Expert guidance for fine-tuning LLMs with LLaMA-Factory - WebUI no-code, 100+ models, 2/3/4/5/6/8-bit QLoRA, multimodal support
Build a knowledge base from web content with Firecrawl. Use for local reference docs, RAG-ready chunks, fine-tuning datasets, documentation mirrors, topic corpora, or LLM-ready markdown organized from web sources.
Audit an executed AI phase's evaluation coverage and produce an EVAL-REVIEW.md remediation plan.
Tests/benchmarks a third-party LLM endpoint (OpenAI- or Anthropic-compatible): availability, fidelity, speed, concurrency, protocol compliance, quality regression. Use before hand-rolling a curl loop, when onboarding a p
Scaffolds eval.yaml evaluation specs for agent skills in the dotnet/skills repository. Use when creating skill tests, writing evaluation stimuli, defining graders and rubrics, sizing an eval for statistical power, or set
No evidence record yet
No measured run with and without this skill yet, so this page makes no performance claim. 2 static scans, 0 counted findings. Security state at 2026-11-17: reviewed. Static scans check the code, not how well the skill works.
State definitionsSkill details
Security scansReviewed
2 static scans, 0 counted findings. Security state at 2026-11-17: reviewed. Edge checks it again each time your agent loads it.
Edge static checks (Edge-run): edge-static/1.0.2. Scope: Static patterns for remote execution, credentials with network sends, obfuscation, prompt overrides, hidden Unicode, binaries and persistence. Raw findings: not supplied.
Edge committed-secret checks (Edge-run): edge-secrets/1.0.0. Scope: Credentials committed in the skill's files (values never shown). Raw findings: not supplied.
Socket (skills.sh mirror): Version unavailable. Scope: Not supplied by the mirrored record. Raw findings: not supplied.
Snyk (skills.sh mirror): Version unavailable. Scope: Not supplied by the mirrored record. Raw findings: not supplied.
- Edge static checks
- Pass · 0 counted findings · 2026-11-17
- Edge committed-secret checks
- Pass · 0 counted findings · 2026-11-17
Creator, source & licencewshobson
From the wshobson public repository. Edge did not write or edit this skill.
- Repository
- wshobson/agents
- Skill file
- View source at revision
- Revision
- 156b7a5e7a8b93642628a339ee4039c925b34c7f
- Package hash
- sha256:d6bfa5651b9547653f77dfbb2b32841f43e5706729951093131798d1f598094c
- Licence
- MIT
- Description
- Build the evaluation harness that gates every fine-tuning run : golden sets, per-failure-mode graders, judge calibration, and base-model baselines. Use when starting a fine-tuning effort, when converting traces into an eval set, or when calibrating a judge against human labels.
Quoted from the skill’s own description
Read skill files3 files
3 files, 24.2 KB at revision 156b7a5e7a8b.
Questions about Eval Harness First
What does Eval Harness First do?
Build the evaluation harness that gates every fine-tuning run : golden sets. Published by Seth Hobson (wshobson/agents).
Is Eval Harness First safe to use?
Edge security state: Reviewed (2 Edge-run static scans, scanned Nov 17, 2026). The Evidence tab shows the scan.
How do I use Eval Harness First in Claude Code, Codex or Cursor?
Click Try for free, pick your agent and follow its one step. Edge is free, with no Edge account.