Seth HobsonCreated wshobson/agents, 41k+ GitHub stars3,200 installs on skills.sh

Eval Harness First

Build the evaluation harness that gates every fine-tuning run : golden sets.

Use in my agent
Use Edge to load the skill eval-harness-first from wshobson/agents for my task. If Edge is not set up, read getedge.cc/SKILL.md and ask me first.
Works in any agentClaude CodeCodexCursorClaude appChatGPT+ any other agent
wshobson/agents
Security state: reviewed

2 static scans, 0 counted findings.

Edge checks it again each time your agent loads it.

SKILL.md3 files · 24.2 KB

From the wshobson public repository. Edge did not write or edit this skill.

What it does

Build the evaluation harness that gates every fine-tuning run : golden sets, per-failure-mode graders, judge calibration, and base-model baselines. Use when starting a fine-tuning effort, when converting traces into an eval set, or when calibrating a judge against human labels.

SH

Why Seth Hobson

Founder of Major 7 Apps. Each fact links its public source.

  • 41k+ GitHub stars on wshobson/agents

    His plugin marketplace of agents, skills and commands for coding agents has over 41,000 stars.

    GitHub
  • 15+ years building data systems

    He has built data applications, services and distributed systems for over 15 years, mostly in finance.

    GitHub profile
  • Founder of Major 7 Apps

    He founded Major 7 Apps, an independent studio shipping developer tools and native Apple apps.

    Personal site

Public sources, checked Oct 26, 2026.

How it works

  1. 1
    Copy

    Try for free gives you one prompt.

  2. 2
    Paste

    Paste it into Claude Code, Codex or Cursor.

  3. 3
    Ask

    Ask for the outcome. Your agent loads Eval Harness First.

Related skills

AM
Eval Harness
affaan-m

Eval-driven development (EDD) framework for AI coding sessions — define capability and regression evals before coding, grade with code-based, model-based, rule, or human graders, and track pass@k and pass^k reliability.

+491 installs this week
DA
Llama Factory
davila7

Expert guidance for fine-tuning LLMs with LLaMA-Factory - WebUI no-code, 100+ models, 2/3/4/5/6/8-bit QLoRA, multimodal support

FI
Firecrawl Knowledge Base
firecrawl

Build a knowledge base from web content with Firecrawl. Use for local reference docs, RAG-ready chunks, fine-tuning datasets, documentation mirrors, topic corpora, or LLM-ready markdown organized from web sources.

OG
GSD Eval Review
open-gsd

Audit an executed AI phase's evaluation coverage and produce an EVAL-REVIEW.md remediation plan.

DA
LLM Eval Harness
daymade

Tests/benchmarks a third-party LLM endpoint (OpenAI- or Anthropic-compatible): availability, fidelity, speed, concurrency, protocol compliance, quality regression. Use before hand-rolling a curl loop, when onboarding a p

DO
Create Skill Test
dotnet

Scaffolds eval.yaml evaluation specs for agent skills in the dotnet/skills repository. Use when creating skill tests, writing evaluation stimuli, defining graders and rubrics, sizing an eval for statistical power, or set

+319 installs this week
Evidence

No evidence record yet

No measured run with and without this skill yet, so this page makes no performance claim. 2 static scans, 0 counted findings. Security state at 2026-11-17: reviewed. Static scans check the code, not how well the skill works.

State definitions

Skill details

Security scansReviewed

2 static scans, 0 counted findings. Security state at 2026-11-17: reviewed. Edge checks it again each time your agent loads it.

Edge static checks (Edge-run): edge-static/1.0.2. Scope: Static patterns for remote execution, credentials with network sends, obfuscation, prompt overrides, hidden Unicode, binaries and persistence. Raw findings: not supplied.

Edge committed-secret checks (Edge-run): edge-secrets/1.0.0. Scope: Credentials committed in the skill's files (values never shown). Raw findings: not supplied.

Socket (skills.sh mirror): Version unavailable. Scope: Not supplied by the mirrored record. Raw findings: not supplied.

Snyk (skills.sh mirror): Version unavailable. Scope: Not supplied by the mirrored record. Raw findings: not supplied.

Edge static checks
Pass · 0 counted findings · 2026-11-17
Edge committed-secret checks
Pass · 0 counted findings · 2026-11-17
Creator, source & licencewshobson

From the wshobson public repository. Edge did not write or edit this skill.

Repository
wshobson/agents
Revision
156b7a5e7a8b93642628a339ee4039c925b34c7f
Package hash
sha256:d6bfa5651b9547653f77dfbb2b32841f43e5706729951093131798d1f598094c
Licence
MIT
Description
Build the evaluation harness that gates every fine-tuning run : golden sets, per-failure-mode graders, judge calibration, and base-model baselines. Use when starting a fine-tuning effort, when converting traces into an eval set, or when calibrating a judge against human labels.
Quoted from the skill’s own description
Read skill files3 files

3 files, 24.2 KB at revision 156b7a5e7a8b.

Questions about Eval Harness First

What does Eval Harness First do?

Build the evaluation harness that gates every fine-tuning run : golden sets. Published by Seth Hobson (wshobson/agents).

Is Eval Harness First safe to use?

Edge security state: Reviewed (2 Edge-run static scans, scanned Nov 17, 2026). The Evidence tab shows the scan.

How do I use Eval Harness First in Claude Code, Codex or Cursor?

Click Try for free, pick your agent and follow its one step. Edge is free, with no Edge account.