Claude Haiku 5.5 launch dayEvals · Oct 7, 2026
Haiku 5.5 with the right skill vs Opus 5.5
20 founder tasks, 3 runs each. Haiku 5.5 had one preselected expert skill per task type. Opus 5.5 had none. GPT and DeepSeek judged blind.
Judgements: Haiku 5.5 + skill vs Opus 5.5
20 founder tasks · 3 runs each · GPT and DeepSeek, both orders
240 blind judge votes. Margins are modest and judges often disagreed. On the same tasks, Haiku with a skill beat plain Haiku by 91 votes to 65, with 84 ties.
Try it in your agent
Free. Edge finds the right expert skill when your agent needs it.
In your terminal (Claude Code), then start a new session:
claude mcp add edge -s user -- npx -y @getedge/mcp
npx -y @getedge/mcp setup claudeOr paste into Claude Code:
set up https://getedge.cc/SKILL.mdUsing Codex, Cursor or ChatGPT? See setup guides
Task by task
Click a task to compare answers and read every verdict.
| Task and skill | vs Opus 5.5 | vs plain Haiku 5.5 | Result vs Opus |
|---|---|---|---|
Skill: dunford-obviously-awesome, based on April Dunford's public method | 11 to 9, 4 ties | 8 to 8 | Haiku+skill ahead |
Skill: fitzpatrick-mom-test, based on Rob Fitzpatrick's public method | 5 to 10, 9 ties | 16 to 1 | Opus ahead |
Skill: kazanjy-founding-sales, based on Pete Kazanjy's public method | 6 to 10, 8 ties | 9 to 5 | Opus ahead |
Skill: braun-poke-the-bear, based on Josh Braun's public method | 14 to 4, 6 ties | 9 to 8 | Haiku+skill ahead |
Skill: pierri-homepage-positioning, based on Anthony Pierri's public method | 12 to 10, 2 ties | 5 to 16 | Haiku+skill ahead |
Skill: kallaway-layered-hook, based on Kallaway's public method | 9 to 10, 5 ties | 11 to 8 | Opus ahead |
Skill: wes-kao-spiky-pov, based on Wes Kao's public method | 13 to 4, 7 ties | 6 to 4 | Haiku+skill ahead |
Skill: costa-negotiation-dynamics, based on Luís Almeida Costa's public method | 7 to 7, 10 ties | 11 to 5 | Even |
Skill: founders-handbook, based on 1984 Ventures' Founders Handbook's public method | 3 to 5, 16 ties | 6 to 4 | Opus ahead |
Skill: laja-wynter-message-testing, based on Peep Laja's public method | 8 to 6, 10 ties | 10 to 6 | Haiku+skill ahead |
Input file the models got
Grading rubric (written before any run, from the task alone)
Verdicts for this pair
Without skills: Haiku vs Opus on 10 separate tasks (25 to 26, 29 ties)
Without skills: plain Haiku 5.5 vs plain Opus 5.5
Run first, on 10 different tasks (2 runs each, same judges): Haiku 25, Opus 26, ties 29. Close to even, which is why the skills test above uses held-out tasks.
| Task | Haiku | Tie | Opus | Split | Result |
|---|---|---|---|---|---|
Write the positioning for our product. Details are in notes.md. | 3 | 3 | 2 | Haiku ahead | |
I'm validating an idea, notes in idea.md. Write the questions for my customer interviews next week. | 3 | 2 | 3 | Even | |
I need to start selling this myself. Context in sales.md. Give me a plan for the next 90 days. | 0 | 4 | 4 | Opus ahead | |
Write a cold email to the VP of Operations at a 400-person regional grocery chain about our product. Product info in product.md. | 7 | 1 | 0 | Haiku ahead | |
Write the copy for our homepage (hero and the sections below it). Product details in product.md. | 5 | 1 | 2 | Haiku ahead | |
Write the script for a 30-second vertical video (TikTok/Reels) about the topic in topic.md. Focus on making people stay past the first seconds. | 2 | 2 | 4 | Opus ahead | |
Write a LinkedIn post for me based on notes.md. I want it to get discussion, not just likes. | 4 | 2 | 2 | Haiku ahead | |
I have a negotiation on Monday, details in deal.md. Prepare me. | 1 | 5 | 2 | Opus ahead | |
Work out what each founder will own after our Series A, and how much more they'd own if SAFE B had been raised at a $15M cap instead. Numbers in raise.md. | 0 | 7 | 1 | Opus ahead | |
We're launching next month and can't agree on the main message. Info in options.md. Tell us how to decide and which one to go with. | 0 | 2 | 6 | Opus ahead |
How we ran it
How we ran it
The plan, tasks, rubrics and the rule for what counts as a win were written down before the runs and are published with the data.
- Agent
- Claude Code CLI in a fresh, empty home directory per run. Same prompt, same input file, default settings. Models pinned:
claude-haiku-5-5,claude-opus-5-5. - Skill
- For the Haiku + skill runs, one Edge skill was installed in
~/.claude/skills, chosen per task type before any run. Haiku decided itself whether to read it. No other change. - Tasks
- 30 founder situations across 10 task types, written before any run. The prompts read like normal requests and never name a framework. 10 were used for the first comparison, the other 20 held out for the skills test.
- Rubrics
- Written by GPT from the task text alone, 6 criteria each, scored on outcomes. No credit for naming a method or author.
- Judges
- GPT (via Codex) and DeepSeek V3.2 (via AWS Bedrock). Each sees the task, input, rubric and two unlabeled answers, in both orders. No Claude model judges.
- Cost
- Tokens per run times list price: Haiku 5.5 $0.10 / $0.50 per million input / output tokens, Opus 5.5 $4 / $20.
- Limits
- 3 runs per task, judges often disagree (DeepSeek calls ties more often than GPT). Business tasks, not coding. Opus did better on founder-led sales, interviews, short-video scripts and SAFE math.
Raw data: study2.json (skills test) and data.json (plain models), with every prompt, input, rubric, answer, verdict and cost.
Edge · published Oct 7, 2026