{"cases": [{"id": "pos-1", "family": "positioning", "prompt": "Write the positioning for our product. Details are in notes.md.", "input": "notes.md", "input_text": "# Ledgerline\nSeed-stage, 6 people, Lisbon. Software for finance teams at 50-500 person companies that run month-end close in spreadsheets.\nWhat it does: pulls bank feeds, card spend and the ERP trial balance into one checklist, auto-matches transactions, flags variances above a threshold, and keeps an audit trail of who signed off on what.\nCustomers today: 14 paying (avg EUR 9k/yr). Best ones: two logistics companies and a dental-clinic group, all with 3-6 person finance teams and multiple entities. Churned: two startups under 30 people (\"we just use our accountant\").\nWhat customers say: \"close went from 11 days to 6\", \"our auditor stopped asking for screenshots\", \"I finally know which reconciliations are late before the CFO asks\".\nAlternatives prospects use: Excel + shared drive, BlackLine (they say it is too expensive and takes 6 months to implement), their ERP's built-in module (NetSuite/Business Central) which they say nobody uses.\nOur website currently says: \"AI-powered financial operations platform for modern finance teams.\"\n", "rubric": [{"id": "c1", "criterion": "Focus the positioning on the buyers most likely to value Ledgerline.", "full_marks_when": "Targets finance teams at 50\u2013500-person companies still managing month-end close in spreadsheets, with particular emphasis on multi-entity businesses and small finance teams like the existing 3\u20136-person teams. Treats logistics and dental clinics as evidence of fit, not proof that only those industries should be targeted.", "zero_when": "Targets generic modern businesses, solo accountants, or very small startups without acknowledging that two sub-30-person startups churned because their accountant was sufficient."}, {"id": "c2", "criterion": "Make clear what Ledgerline is and the job it helps customers accomplish.", "full_marks_when": "Describes Ledgerline in a readily understood category tied to month-end close and reconciliations, making its purpose clearer than the current broad financial-operations tagline.", "zero_when": "Recycles vague AI-platform language or presents Ledgerline as an ERP, accounting service, or general finance suite."}, {"id": "c3", "criterion": "Explain why a prospect would choose Ledgerline over their actual alternatives.", "full_marks_when": "Addresses spreadsheet-and-shared-drive workflows and the reported shortcomings of BlackLine and unused ERP modules. Explains Ledgerline's appeal through its consolidated workflow, automated reconciliation support, and sign-off visibility without inventing price or implementation advantages.", "zero_when": "Ignores the stated alternatives, substitutes unrelated competitors, or asserts unsupported superiority such as guaranteed cheaper pricing or deployment in days."}, {"id": "c4", "criterion": "Connect specific product capabilities to meaningful customer benefits.", "full_marks_when": "Links the combined checklist, transaction matching, variance flags, and sign-off audit trail to a faster close, earlier visibility into late reconciliations, and easier audit evidence collection. Establishes a coherent reason to buy rather than merely listing features.", "zero_when": "Offers generic efficiency claims with no connection to Ledgerline's capabilities, or bases the differentiation on invented functionality."}, {"id": "c5", "criterion": "Use the supplied evidence accurately and distinguish reported results from promises.", "full_marks_when": "Uses customer evidence faithfully and labels it as reported experience rather than a universal guarantee. If quantifying the close improvement, correctly states that 11 days to 6 saves 5 days, approximately a 45.5% reduction. Preserves any other cited facts, including 14 paying customers and average annual revenue per customer of EUR 9,000, and avoids unsupported claims about AI, compliance certification, or guaranteed outcomes.", "zero_when": "Misstates input facts, makes arithmetic errors, fabricates evidence, or turns the reported six-day close into a guaranteed result for every customer."}, {"id": "c6", "criterion": "Deliver positioning that the team can use immediately.", "full_marks_when": "Provides a clear positioning statement and concise customer-facing wording, such as a replacement headline and supporting sentence, that consistently express the chosen audience, close-related problem, product value, and reason to choose Ledgerline. The wording is concrete and usable without another strategy exercise.", "zero_when": "Provides only questions, advice, a template with blanks, or a discussion of positioning techniques without writing actual positioning for Ledgerline."}], "runs": {"haiku-1": {"cost_usd": 0.0043, "seconds": 17, "output_tokens": 3362, "files": [{"name": "positioning.md", "text": "# Ledgerline positioning\n\n## Positioning statement\n\nFor finance teams at 50-500 person companies that run month-end close in spreadsheets, Ledgerline is close management software that pulls bank, card and ERP data into one checklist, auto-matches transactions, flags variances, and keeps an audit trail of who signed off on what. Unlike spreadsheets or enterprise close suites, it is built for a 3-6 person finance team and gets them closing faster without a systems project.\n\n## Market category\n\nMonth-end close management. Ledgerline competes in the category BlackLine defines, but for mid-market teams that BlackLine's price and implementation timeline exclude.\n\n## Competitive alternatives\n\nWhat prospects use today, and the weakness they describe:\n\n- **Excel and a shared drive.** The default. Teams know its pain (late reconciliations, no clear sign-off history) and still choose it because it is free and familiar.\n- **BlackLine.** The known enterprise option. Prospects say it is too expensive and takes six months to implement.\n- **The ERP's built-in module (NetSuite, Business Central).** Available, but prospects say nobody uses it.\n\n## Unique attributes\n\n- One checklist across bank feeds, card spend and the ERP trial balance, instead of stitching sources together by hand.\n- Automatic transaction matching with variance flags above a threshold.\n- A sign-off audit trail that auditors accept without screenshots.\n- Visibility into which reconciliations are late before anyone has to ask.\n\n## Value to customers\n\n- Shorter close. One customer: \"close went from 11 days to 6.\"\n- Less audit back-and-forth. One customer: \"our auditor stopped asking for screenshots.\"\n- Earlier warning on late work. One customer: \"I finally know which reconciliations are late before the CFO asks.\"\n\n## Target customer\n\n- 50-500 employees, with a finance team of roughly 3-6 people.\n- Multiple entities. The two logistics companies and the dental-clinic group all run more than one.\n- Closing in spreadsheets today.\n\nNot the target: companies under about 30 people. Both churned startups in that range said \"we just use our accountant.\" Qualify these out early.\n\n## Website copy\n\nCurrent headline: \"AI-powered financial operations platform for modern finance teams.\" It names no problem, no workflow and no outcome, and nothing in the customer feedback mentions AI.\n\nProposed:\n\n- **Headline:** Month-end close without the spreadsheets.\n- **Subhead:** Ledgerline pulls bank, card and ERP data into one checklist, matches transactions automatically, flags variances, and records who signed off on every reconciliation.\n- **Proof point:** \"Close went from 11 days to 6.\" (with customer attribution, once permission is confirmed)\n\n## Open items before publishing\n\n- Verify the BlackLine comparison (price and six-month implementation) before using it on the site. Right now it comes only from prospects. Publish it only with a source or as a prospect quote.\n- Confirm Ledgerline's own implementation time. The contrast with BlackLine only works if it is measurably shorter.\n- Get permission to use customer names and quotes. The evidence is three quotes from 14 paying customers.\n- Check whether multi-entity support is a shipped feature. The target customer depends on it, but the notes describe customers with multiple entities, not the product's support for them.\n"}]}, "haiku-2": {"cost_usd": 0.00499, "seconds": 23, "output_tokens": 4477, "files": [{"name": "positioning.md", "text": "# Ledgerline Positioning\n\n## Positioning statement\n\nFor finance teams at 50\u2013500 person companies that run month-end close across spreadsheets and shared drives, Ledgerline is the month-end close checklist that brings bank feeds, card spend and the ERP trial balance into one place, matches transactions automatically, flags variances as they appear, and records who signed off on what. Unlike Excel, it gives the CFO and the auditor one live view. Unlike BlackLine, it does not take six months to implement.\n\n## Target customer\n\n**Primary:** companies with 50\u2013500 people, multiple entities, and a finance team of 3\u20136 people that closes the books in spreadsheets.\n\nEvidence: the three best customers (two logistics companies and a dental-clinic group) all fit this profile.\n\n**Not the target:** companies under 30 people that rely on their accountant to close. Two of these churned with the reason \"we just use our accountant.\" Treat this as a segment signal, though two data points is a small sample.\n\n## Market category\n\n**Month-end close.** It is the term the buyer uses, and it is specific enough to search for.\n\nAvoid \"financial operations platform.\" It describes nothing the buyer currently does and does not separate us from anyone.\n\n## Competitive alternatives\n\n| Alternative | What prospects say | Our angle |\n|---|---|---|\n| Excel + shared drive | Works, but status lives in people's heads and auditors ask for screenshots | One checklist with a sign-off audit trail |\n| BlackLine | Too expensive; takes about six months to implement | Faster to deploy and sized for mid-market teams (must be verified, see below) |\n| ERP built-in module (NetSuite, Business Central) | Nobody uses it | We sit on top of the ERP trial balance rather than replace it |\n\n## Differentiated capabilities\n\n1. One checklist across bank feeds, card spend and the ERP trial balance.\n2. Automatic transaction matching.\n3. Variance flags above a set threshold.\n4. Sign-off audit trail that auditors accept without screenshots.\n5. Visibility into late reconciliations before the CFO asks.\n\n## Value, in customers' words\n\n- \"Close went from 11 days to 6.\" (Quote from one customer. Do not generalize this to every customer.)\n- \"Our auditor stopped asking for screenshots.\"\n- \"I finally know which reconciliations are late before the CFO asks.\"\n\nLead with speed and visibility. These are the outcomes customers actually describe.\n\n## Messaging\n\n**Drop \"AI-powered\" from the lead.** None of the customer quotes or the prospect alternatives mention AI. The value customers name is speed, audit evidence and visibility. If auto-matching uses machine learning, say what it does for the buyer. If it does not, remove the claim.\n\nReplacement headline options:\n\n- \"Month-end close, run from one checklist.\"\n- \"Know which reconciliations are late before your CFO asks.\"\n- \"Audit-ready month-end close for finance teams of three to six.\"\n\n## Open items before publishing\n\n- **Our own implementation time.** The BlackLine contrast only works if we can state how long Ledgerline takes to go live. The notes do not include this.\n- **Pricing comparison.** Our average is EUR 9k/yr. Do not cite BlackLine pricing unless it has been verified.\n- **ERP coverage.** The \"sits on top of your ERP\" claim depends on which ERPs we integrate with. The notes do not say, and NetSuite and Business Central appear only as prospect alternatives.\n- **Vertical pattern.** The logistics and dental clinic pattern comes from three customers out of 14. Test it before building vertical messaging.\n- **Speed claim.** The 11-to-6-day result is from one customer. Gather more before using it in headlines.\n"}]}, "opus-1": {"cost_usd": 0.13777, "seconds": 69, "output_tokens": 3932, "files": [{"name": "positioning.md", "text": "# Ledgerline Positioning\n\n*Draft v1, 2026-10-08. Based on notes.md (14 paying customers, 2 churned).*\n\n---\n\n## The short version\n\n**For** finance teams of 3\u20136 people at multi-entity companies (roughly 50\u2013500 employees) who run month-end close in spreadsheets,\n**Ledgerline is** month-end close software\n**that** pulls bank feeds, card spend and the ERP trial balance into one close checklist, auto-matches transactions, flags variances, and records who signed off on what.\n**Unlike** BlackLine, it doesn't take six months or an enterprise budget to set up. **Unlike** the close module in NetSuite or Business Central, your team will actually use it.\n\n**One-liner:** Month-end close software for finance teams that have outgrown spreadsheets but not their budget.\n\n---\n\n## 1. Competitive alternatives (what we're really up against)\n\nWhat prospects do today if Ledgerline doesn't exist:\n\n| Alternative | Why they use it | Why it falls short |\n|---|---|---|\n| **Excel + shared drive** (the main one) | Free, everyone knows it, flexible | No live status, matching is manual, sign-offs live in email, auditors ask for screenshots |\n| **BlackLine** | The known \"proper\" close tool | Prospects call it too expensive and say it takes ~6 months to implement. Built for much bigger teams |\n| **ERP's built-in module** (NetSuite, Business Central) | Already paid for | Prospects say \"nobody uses it\" |\n| **Outsourcing to an accountant** | Simple for very small companies | Fine below ~30 people. That's why our two startup customers churned |\n\nOur real competitor is the spreadsheet, not BlackLine. Most deals are a switch away from Excel, so positioning should show what changes compared to spreadsheets. BlackLine is a reference point for price and how long setup takes.\n\n## 2. What's different about us\n\n- **One checklist that pulls in all three sources**: bank feeds, card spend, ERP trial balance\n- **Auto-matching** of transactions across those sources\n- **Variance flags** above a threshold you set\n- **Built-in audit trail**: who prepared, who reviewed, who signed off, and when\n- **Fits a mid-market team**: priced around \u20ac9k/yr and set up without a long project (contrast with BlackLine)\n\n## 3. Value (what those features mean for the customer)\n\n| Feature | Value | Proof (customer words) |\n|---|---|---|\n| Unified checklist + auto-matching | **Faster close** | \"Close went from 11 days to 6\" |\n| Audit trail of sign-offs | **Painless audits** | \"Our auditor stopped asking for screenshots\" |\n| Live checklist + variance flags | **Control and visibility** | \"I finally know which reconciliations are late before the CFO asks\" |\n\nThese three outcomes are the core of the message: **faster close, cleaner audits, no surprises.**\n\n## 4. Best-fit customer\n\n**Who loves it:** companies with **multiple entities** and a **3\u20136 person finance team** that runs close in spreadsheets. Examples so far: two logistics companies and a dental-clinic group.\n\nSigns a prospect is a good fit:\n- More than one legal entity, so more reconciliations and intercompany work\n- A finance team big enough to need hand-offs and sign-offs, but too small to have a systems person\n- Close takes more than ~7 working days\n- An external audit each year\n- Already on, or moving to, an ERP such as NetSuite or Business Central\n\n**Not a fit (yet):** companies under ~30 people with no in-house finance team, where an outside accountant does the books. Both churned customers looked like this. Don't chase them.\n\n**Buyer:** Head of Finance / Financial Controller (owns close and feels the pain).\n**Economic approver:** CFO (cares about speed, control, audit risk).\n\n## 5. Market category\n\n**Category: month-end close software** (or \"close management\").\n\nWhy this category:\n- Buyers already search for it and budget for it. BlackLine has taught them what it is\n- It frames the decision as \"a close tool sized for us\" rather than an unfamiliar new thing\n- It says plainly what the product does. \"Financial operations platform\" could mean AP, spend management, FP&A or something else entirely\n\n**Why we should drop the current website line.** \"AI-powered financial operations platform for modern finance teams\":\n- *AI-powered*: says how it's built, not what it does for the customer, and every competitor makes the same claim. None of the customer quotes mention AI.\n- *Financial operations platform*: too broad. It doesn't say \"close.\"\n- *Modern finance teams*: says nothing about who it's for. Our ICP is specific: mid-market, multi-entity, running on spreadsheets.\n\nAuto-matching can be described as automation inside the product copy. It shouldn't be the headline.\n\n## 6. Messaging\n\n### Website hero (options)\n\n**A. Outcome-led (recommended)**\n> **Close the month in days, not weeks.**\n> Ledgerline brings your bank feeds, card spend and trial balance into one close checklist, matches transactions automatically, and keeps a sign-off trail your auditors will love. Built for lean, multi-entity finance teams.\n\n**B. Against the alternative**\n> **Your month-end close, out of spreadsheets.**\n> One checklist for every reconciliation across every entity. Auto-matching, variance flags, and an audit trail of who signed off on what. Live in weeks, not six months.\n\n**C. Control-led**\n> **Know what's late before your CFO asks.**\n> Month-end close software for multi-entity finance teams. Every reconciliation, every sign-off, in one place.\n\n### Three supporting pillars\n\n1. **Close faster.** Bank, card and ERP data in one place, with transactions matched automatically. *\"Close went from 11 days to 6.\"*\n2. **Audit-ready by default.** Every reconciliation shows who prepared it, who reviewed it, and when. *\"Our auditor stopped asking for screenshots.\"*\n3. **See everything, every month.** A live checklist across all your entities, with variances above your threshold flagged. *\"I finally know which reconciliations are late before the CFO asks.\"*\n\n### Handling the main alternatives in sales conversations\n\n- **\"We're fine with Excel.\"** The spreadsheet isn't slow. The parts around it are: chasing status, matching by hand, gathering sign-off evidence. Keep Excel for analysis and move the close itself into Ledgerline.\n- **\"We looked at BlackLine.\"** Same job, sized for your team. Live in weeks, at a fraction of the cost. *(Confirm actual implementation time before using a number.)*\n- **\"Our ERP has a close module.\"** Ask: is anyone using it? Ledgerline sits on top of NetSuite or Business Central and pulls in the bank and card data the ERP module doesn't connect.\n\n### Boilerplate (for decks, press, LinkedIn)\n\n> Ledgerline is month-end close software for mid-sized, multi-entity finance teams. It brings bank feeds, card spend and the ERP trial balance into a single close checklist, auto-matches transactions, flags variances, and records every sign-off for audit. Ledgerline is based in Lisbon.\n\n---\n\n## 7. Open questions / things to validate\n\n- **The \"11 \u2192 6 days\" figure** is from one customer. Before putting a number in the headline, check whether it holds across most of the 14. If it varies, use \"close in days, not weeks\" without a number.\n- **Implementation time:** how long does a typical onboarding actually take? \"Live in weeks\" needs real data behind it.\n- **Vertical focus:** logistics and multi-site healthcare (dental groups) are the best accounts. Is there a pattern worth targeting, such as many locations or entities, or high transaction volume? Consider landing pages for each vertical once there are 3\u20135 customers in each.\n- **Size band:** the notes say 50\u2013500 employees, but the best accounts have 3\u20136 person finance teams. Finance-team size and entity count are probably better qualifiers than headcount.\n- **ERP coverage:** list which ERPs are supported today so the \"sits on top of your ERP\" claim is accurate.\n"}]}, "opus-2": {"cost_usd": 0.13416, "seconds": 42, "output_tokens": 3953, "files": [{"name": "positioning.md", "text": "# Ledgerline Positioning\n\n*Draft v1, 2026-10-08. Source: notes.md. Anything not in the notes is marked **[ASSUMPTION]** or listed under Open Questions.*\n\n---\n\n## 1. Positioning statement\n\n**For** finance teams of 3\u20136 people at multi-entity companies with 50\u2013500 employees, who run month-end close out of spreadsheets and a shared drive,\n\n**Ledgerline is** month-end close software\n\n**that** brings bank feeds, card spend and the ERP trial balance into one close checklist, matches transactions automatically, flags variances above a threshold you set, and records who signed off on what.\n\n**Unlike** Excel plus a shared drive, it shows you which reconciliations are late while there's still time to fix them, and it gives your auditor a sign-off trail instead of screenshots. **Unlike** BlackLine, it's priced and scoped for a mid-sized team, at about EUR 9k a year. **Unlike** the close module in NetSuite or Business Central, it's a tool the team actually uses.\n\n---\n\n## 2. Best-fit customer\n\nBased on the customers who get the most value today (two logistics companies and a dental-clinic group):\n\n| Trait | Best fit | Why it matters |\n|---|---|---|\n| Company size | 50\u2013500 employees | Big enough that close is a real multi-day project |\n| Finance team | 3\u20136 people | Several people share the close, so it's hard to see who's done what |\n| Structure | **Multiple entities** (or locations) | More bank accounts, more reconciliations and more intercompany work, so spreadsheets break first |\n| Current process | Excel + shared drive, possibly an unused ERP close module | The status quo we replace |\n| Pain trigger | Audit friction, a slow close, the CFO asking for status | The moments that start a purchase |\n| Buyer / user | Financial controller or Head of Finance (buyer), with the CFO as sponsor **[ASSUMPTION: confirm who signed the 14 deals]** | |\n\n**Not a fit:** companies under ~30 people, or any team that hands its books to an outside accountant. Both churned customers fit this description (\"we just use our accountant\"). Sales should qualify these out early.\n\n**Signal to look for:** \"multi-entity\" looks like a better predictor than industry. Logistics and dental groups share a pattern: many sites or entities, lots of transactions, and a small central finance team. Look for others with the same shape, such as franchise groups, clinic or vet chains, multi-site retail, and holding companies. **[ASSUMPTION: extrapolated from 3 customers. Validate before targeting these verticals.]**\n\n---\n\n## 3. Competitive alternatives (what we really compete with)\n\n| Alternative | Why prospects use it | Where it falls short | Our angle |\n|---|---|---|---|\n| **Excel + shared drive** (main competitor) | Free, familiar, flexible | No live status, manual matching, no sign-off trail, and the auditor asks for screenshots | One checklist with live status, auto-matching and a built-in audit trail |\n| **BlackLine** | The known enterprise standard | \"Too expensive,\" \"takes 6 months to implement\" | Built and priced for a 3\u20136 person team (~EUR 9k/yr) |\n| **ERP close module** (NetSuite, Business Central) | Already paid for | \"Nobody uses it\" | Pulls in bank and card data alongside the trial balance, so the team actually works in it **[ASSUMPTION: find out *why* teams don't use the ERP module]** |\n| **Outside accountant** | Cheap for very small companies | \u2014 | Not our market. Don't compete here |\n\n---\n\n## 4. Differentiated capabilities \u2192 value \u2192 proof\n\n| Capability | Value to the customer | Proof |\n|---|---|---|\n| Bank feeds, card spend and ERP trial balance in **one close checklist** | One place to see where the close stands | \"I finally know which reconciliations are late before the CFO asks\" |\n| **Auto-matching** of transactions | Less manual tick-and-tie, so a shorter close | \"Close went from 11 days to 6\" |\n| **Variance flags** above a set threshold | Problems surface during close, not after it | (no quote yet) |\n| **Sign-off audit trail** (who approved what, and when) | Easier audits with less back-and-forth | \"Our auditor stopped asking for screenshots\" |\n\n**The three value pillars:**\n1. **Close faster.** Days back every month.\n2. **Know where the close stands.** See late items before the CFO asks.\n3. **Make audits easier.** A sign-off trail instead of screenshots.\n\n---\n\n## 5. Market category\n\n**Recommended: \"month-end close software\"** (or \"close management software\"), with a mid-market qualifier.\n\n- Buyers already have budget and intent for this category, and BlackLine has taught them what it means. Our job is to be the right-sized version.\n- \"AI-powered financial operations platform\" is too broad. It doesn't say what problem we solve, it puts us next to dozens of vague \"FinOps\" and AI tools, and none of our customers' quotes mention AI.\n\n---\n\n## 6. Messaging\n\n### Homepage headline options\n1. **Month-end close software for finance teams that have outgrown spreadsheets.**\n2. **Close the books in days, not weeks, without leaving Excel behind overnight.** *(only if we support a gradual move from Excel. Verify first)*\n3. **Close faster. Know what's late. Give your auditor a trail, not screenshots.**\n\n**Recommended:** option 1 as the headline, with option 3's three pillars as the subhead or the three feature blocks below it.\n\n### Subhead (draft)\n> Ledgerline pulls your bank feeds, card spend and ERP trial balance into one close checklist. It matches transactions automatically, flags variances, and records every sign-off, so your team closes faster and your auditor stops asking for screenshots.\n\n### Proof bar\n- \"Close went from 11 days to 6.\"\n- \"Our auditor stopped asking for screenshots.\"\n- \"I finally know which reconciliations are late before the CFO asks.\"\n\n*(Attribute each quote with role and company type, e.g. \"Financial Controller, multi-entity logistics group\", once customers approve.)*\n\n### Comparison one-liners for sales\n- **vs. Excel:** \"Keep your process and lose the chasing. Everyone works from one checklist, and the audit trail builds itself.\"\n- **vs. BlackLine:** \"Close software built for a five-person finance team: live in weeks, not six months, at a fraction of the cost.\" *(confirm our actual implementation time before using \"weeks\")*\n- **vs. the ERP module:** \"You already own a close module. Ask your team when they last opened it.\"\n\n### Words to use and avoid\n- **Use:** close, reconciliations, sign-off, auditor, multi-entity, days, late.\n- **Avoid:** \"platform,\" \"financial operations,\" \"modern,\" and \"AI-powered\" as the lead claim. Only mention AI if it's how matching actually works, and then as a detail, not the headline.\n\n---\n\n## 7. Open questions / evidence gaps\n\n1. **Implementation time.** How long does onboarding actually take? This is the most important contrast with BlackLine, and the notes don't say.\n2. **How typical is \"11 \u2192 6 days\"?** Is it one customer or the average? Don't present it as typical until we know.\n3. **Why don't teams use the ERP close module?** Is it a UX problem, missing bank and card data, or setup effort? The answer shapes how we pitch against it.\n4. **Is multi-entity the real driver?** Check all 14 customers for entity count against retention and expansion.\n5. **Who buys?** Controller, Head of Finance or CFO, and who signs?\n6. **How does matching work?** Rules, ML or both? This decides whether \"AI\" belongs anywhere in the copy.\n7. **ERP coverage.** Which ERPs do we integrate with today? It limits who we can target.\n8. **Small sample.** These conclusions come from 14 customers and 2 churns. Revisit at ~30 customers.\n"}]}}, "verdicts": [{"pair": "1", "judge": "gpt", "order": "haiku shown first", "winner": "haiku", "why": "A is focused and mostly faithful, though implementation and auditor-acceptance claims overreach. B invents deployment speed, price superiority, ERP limitations, and audit functionality; later validation caveats do not repair unsupported claims in its proposed positioning.", "haiku_scores": {"c1": 2, "c2": 2, "c3": 1, "c4": 2, "c5": 1, "c6": 2}, "opus_scores": {"c1": 2, "c2": 2, "c3": 0, "c4": 1, "c5": 0, "c6": 1}}, {"pair": "1", "judge": "gpt", "order": "opus shown first", "winner": "haiku", "why": "B provides clearer, usable positioning with better evidence discipline. A invents implementation speed, cost superiority, ERP limitations, and audit functionality. B still overclaims \"without a systems project\" and generalizes one auditor\u2019s experience into auditor acceptance.", "haiku_scores": {"c1": 2, "c2": 2, "c3": 1, "c4": 2, "c5": 1, "c6": 2}, "opus_scores": {"c1": 2, "c2": 2, "c3": 0, "c4": 1, "c5": 0, "c6": 1}}, {"pair": "1", "judge": "deepseek", "order": "haiku shown first", "winner": "tie", "why": "Both accurately target the buyer, clarify the product, address alternatives, link features to benefits, use evidence correctly, and provide usable copy.", "haiku_scores": {"c1": 2, "c2": 2, "c3": 2, "c4": 2, "c5": 2, "c6": 2}, "opus_scores": {"c1": 2, "c2": 2, "c3": 2, "c4": 2, "c5": 2, "c6": 2}}, {"pair": "1", "judge": "deepseek", "order": "opus shown first", "winner": "tie", "why": "Both accurately target the buyer, define the product, contrast alternatives, link features to benefits, use evidence correctly, and provide usable positioning.", "haiku_scores": {"c1": 2, "c2": 2, "c3": 2, "c4": 2, "c5": 2, "c6": 2}, "opus_scores": {"c1": 2, "c2": 2, "c3": 2, "c4": 2, "c5": 2, "c6": 2}}, {"pair": "2", "judge": "gpt", "order": "haiku shown first", "winner": "opus", "why": "B better connects capabilities to benefits and supplies usable headline/subhead copy. Both invent implementation advantages. B additionally treats average customer revenue as pricing, asserts fractional cost, and promises screenshot-free audits. Later caveats do not repair unsupported positioning claims.", "haiku_scores": {"c1": 2, "c2": 2, "c3": 0, "c4": 1, "c5": 0, "c6": 2}, "opus_scores": {"c1": 2, "c2": 2, "c3": 0, "c4": 2, "c5": 0, "c6": 2}}, {"pair": "2", "judge": "gpt", "order": "opus shown first", "winner": "tie", "why": "A connects capabilities to benefits better but misuses average revenue as pricing and asserts unsupported advantages. B handles customer evidence more carefully but still claims unverified implementation speed and mostly lists capabilities. Both target the right buyers and provide usable wording.", "haiku_scores": {"c1": 2, "c2": 2, "c3": 0, "c4": 1, "c5": 1, "c6": 2}, "opus_scores": {"c1": 2, "c2": 2, "c3": 0, "c4": 2, "c5": 0, "c6": 2}}, {"pair": "2", "judge": "deepseek", "order": "haiku shown first", "winner": "haiku", "why": "A is more precise and avoids unsupported claims (e.g., price comparison, 'weeks' implementation). B makes assumptions that violate the rubric.", "haiku_scores": {"c1": 2, "c2": 2, "c3": 2, "c4": 2, "c5": 2, "c6": 2}, "opus_scores": {"c1": 2, "c2": 2, "c3": 1, "c4": 2, "c5": 1, "c6": 2}}, {"pair": "2", "judge": "deepseek", "order": "opus shown first", "winner": "opus", "why": "A provides more complete, actionable positioning and clearer competitive angles, fully satisfying all criteria.", "haiku_scores": {"c1": 2, "c2": 2, "c3": 1, "c4": 2, "c5": 2, "c6": 1}, "opus_scores": {"c1": 2, "c2": 2, "c3": 2, "c4": 2, "c5": 2, "c6": 2}}]}, {"id": "int-1", "family": "interviews", "prompt": "I'm validating an idea, notes in idea.md. Write the questions for my customer interviews next week.", "input": "idea.md", "input_text": "Idea: an app for parents of kids with food allergies that scans restaurant menus and flags dishes that likely contain the allergen, plus a shared profile grandparents and babysitters can see.\nMe: dad of a 6-year-old with a peanut and sesame allergy. I've built the scanner prototype.\nInterviews booked: 8 parents from a local allergy-parents Facebook group, 30 minutes each, video calls.\nWhat I want to find out: would they pay EUR 6/month, and which feature matters more.\nI already believe: eating out is the scariest moment, and parents would use the app every time they go out.\n", "rubric": [{"id": "c1", "criterion": "Provides an interview guide the asker can use directly in next week's calls.", "full_marks_when": "Includes actual questions in a practical sequence, with useful follow-up probes and realistic timing within each 30-minute call. Prioritizes discovery before introducing the prototype or price.", "zero_when": "Provides general interview advice instead of usable questions, or a script that cannot reasonably fit a 30-minute call."}, {"id": "c2", "criterion": "Tests the asker's assumptions without steering parents toward confirming them.", "full_marks_when": "Asks about recent, specific allergy-management experiences, including the last restaurant visit, existing precautions, frequency of eating out, and other difficult situations. Allows evidence that eating out is not the scariest moment or that an app would not be used on every outing.", "zero_when": "Treats those beliefs as established facts or relies on leading questions such as whether parents would use the app every time they eat out."}, {"id": "c3", "criterion": "Investigates willingness to pay EUR 6/month using evidence stronger than polite agreement.", "full_marks_when": "Explores current spending, paid alternatives, who makes purchase decisions, and concrete reasons to subscribe or decline. Introduces EUR 6/month neutrally after understanding current behavior and probes what would justify that recurring cost.", "zero_when": "Omits pricing or treats a hypothetical yes to 'Would you pay EUR 6/month?' as sufficient evidence of demand."}, {"id": "c4", "criterion": "Produces evidence for choosing between menu scanning and a shared caregiver profile.", "full_marks_when": "Investigates actual menu-selection and caregiver-handoff experiences, including grandparents or babysitters where relevant. Presents both features neutrally and asks which single feature parents would choose first, why, and what recent problem it would solve.", "zero_when": "Only asks which feature sounds appealing, promotes the scanner because it already exists, or fails to investigate the shared profile."}, {"id": "c5", "criterion": "Investigates the trust and safety constraints that could determine whether either feature is usable.", "full_marks_when": "Includes questions about hidden ingredients, cross-contact, checking with restaurant staff, responses to uncertain or missed allergen flags, and what would make parents trust or reject scanner results. For shared profiles, probes accuracy, updates, caregiver access, and information parents are comfortable sharing. Does not imply that a menu scan guarantees a dish is safe.", "zero_when": "Frames flagged or unflagged dishes as reliably safe or unsafe, encourages replacing existing safety checks with the scanner, or ignores trust and caregiver-information reliability."}, {"id": "c6", "criterion": "Stays accurate to the input and respects what this interview sample can establish.", "full_marks_when": "Correctly uses eight parents recruited from a local allergy-parents Facebook group, 30-minute video calls, a EUR 6/month price, an existing scanner prototype, and the asker's six-year-old child with peanut and sesame allergies. Does not assume interviewees share those allergens or have tried the prototype, and does not treat eight group members' responses as proof of market-wide demand.", "zero_when": "Materially misstates the price, participants, call format or duration, prototype status, or child's age or allergies; invents interview findings; or claims these interviews can establish representative demand."}], "runs": {"haiku-1": {"cost_usd": 0.00495, "seconds": 24, "output_tokens": 4437, "files": [{"name": "interview-questions.md", "text": "# Customer interview guide: food allergy app for parents\n\n30 minutes per call, 8 parents from the local allergy-parents Facebook group.\n\n## What we need to learn\n\n1. Would parents pay EUR 6/month? (Section 5)\n2. Which matters more: the restaurant menu scanner or the shared profile for caregivers? (Section 4)\n3. Belief to test: eating out is the scariest moment for parents. (Section 1)\n4. Belief to test: parents would use the app every time they go out. (Sections 1 and 2)\n\n## Interviewer notes\n\n- Ask about what happened last time, not what they would do. \"Would you use it?\" gets polite yeses.\n- Don't mention the prototype or the EUR 6 price until the sections where they come up.\n- Don't correct or defend the idea. If someone disagrees, ask why.\n- Record exact quotes and numbers. Mark each answer as past behavior or future intention.\n- Eight people from one Facebook group are likely more engaged than the average parent. Say so in the findings.\n\n## Opening (2 min)\n\n- Thanks for the time. I'm not selling anything today. I'm trying to learn how you manage this, and criticism helps more than encouragement.\n- Ask permission to record or take notes.\n\n## 1. Eating out (8 min)\n\n- Tell me about the last time you ate out with your child. Where did you go, and who chose the place?\n- How did you check the menu before ordering? Did you ask staff, and what did they say?\n- Walk me through how you decided a dish was safe. What were you looking for?\n- Think about the last time you weren't sure about a dish. What did you do?\n- Which situations feel most stressful: restaurants, friends' homes, parties, school, travel? Rank them.\n- How often do you eat out in a typical month? Has that changed since the diagnosis?\n\n## 2. Current tools and workarounds (5 min)\n\n- What do you use today to manage this? Include apps, cards, calling ahead, and asking staff.\n- Have you paid for anything in this area? What, how much, and do you still pay for it?\n- Have you stopped using anything? Why?\n- What's the most frustrating part of what you use now?\n\n## 3. Other caregivers (6 min)\n\n- Who else looks after your child regularly? Think grandparents, babysitters, school, friends.\n- How do you currently tell them about the allergy? Is anything written down?\n- Has anyone been unsure about something your child was given? What happened afterward?\n- In an emergency, what would someone else need to know, and where would they look for it?\n\n## 4. Feature priority (6 min)\n\nNow describe the two features, neutrally and without the prototype:\n\n- **Restaurant menu scanner:** flags dishes that likely contain the allergen.\n- **Shared child profile:** grandparents and babysitters can see the allergy details and emergency steps.\n\n- If you could have only one, which would it be, and why?\n- Give 100 points across these: menu scanner, shared profile, other (what?).\n- What would make you not trust a menu scanner? Think accuracy, cross-contamination, hidden ingredients.\n- Which parts would need to be right every time, and which would be fine as a starting point?\n\n## 5. Price (7 min)\n\nAsk these before mentioning EUR 6.\n\n- If a tool solved your biggest problem here, what would you expect to pay per month?\n- At what price would it seem too cheap to be reliable?\n- At what price would it feel too expensive to consider?\n- Who would pay: you, your partner, a grandparent, someone else? Who makes that decision?\n\nNow mention the price:\n\n- The planned price is EUR 6/month. What's your first reaction?\n- What would you need to see to pay that?\n- What do you pay for now, or what would you use instead if this didn't exist?\n- Would you put your name or card down for early access today? If yes, record it. If no, ask what's holding you back.\n\n## Close (2 min)\n\n- What should I have asked that I didn't?\n- Who else should I talk to? Is there someone in your life with the same problem who isn't in the group?\n- Can I follow up with you when there's something to try?\n\n## After each interview\n\nFill in right away while it's fresh:\n\n- Last time they ate out, and what they did:\n- Ranking of stressful situations (answer to section 1):\n- Feature choice and points (section 4):\n- Price they named before hearing EUR 6, and their reaction to EUR 6:\n- Did they commit to anything (early access, card, follow-up)?\n- Quotes worth keeping:\n"}]}, "haiku-2": {"cost_usd": 0.00536, "seconds": 28, "output_tokens": 5108, "files": [{"name": "interview-questions.md", "text": "# Customer interview guide: allergy menu scanner and shared profile\n\n8 parents, 30 minutes each, video calls.\n\nGoals:\n1. Would they pay EUR 6/month?\n2. Does the menu scanner or the shared profile matter more?\n\nYour beliefs to test (not confirm):\n- Eating out is the scariest moment.\n- Parents would use the app every time they go out.\n\n## Ground rules\n\n- Ask what they did, not what they would do. \"Would you use this?\" gets polite yeses.\n- Don't demo the prototype until the demo step in section 4. Showing it early anchors every answer after it.\n- After any positive answer, ask \"why?\" and \"when was the last time that happened?\"\n- Let silences run.\n- Ask them to criticize. You're a parent with a prototype, so people may be kind to you.\n\n## Opening (2 min)\n\n- \"I'm researching how families handle food safety outside the home. I'm not selling anything today.\"\n- Ask permission to record or take notes. Tell them they can skip any question.\n- Ask how old their child is and how long ago the allergy was diagnosed. Note the allergens and severity.\n\n## 1. Daily life (6 min)\n\n1. Walk me through a typical week. Where does your child eat outside your home?\n2. Which of those situations worries you most? Rank them if you can: restaurants, school or nursery, birthday parties, playdates, grandparents, babysitters, travel.\n3. Tell me about the last time your child ate out. What happened from arriving to ordering?\n4. In the last month, how many times did your child eat out? Count them.\n\n## 2. Current workarounds (6 min)\n\n5. What do you do before going to a restaurant? Call ahead, check the menu online, ask the staff?\n6. Tell me about a time it went wrong, or nearly did. What happened afterward?\n7. When you asked a restaurant about ingredients, did you trust the answer? Why or why not?\n8. Have you ever avoided a place or skipped an outing because of the allergy?\n9. What tools do you use now: apps, group chats, printed cards, notes? What do you like and dislike about each?\n10. Have you ever paid for any of them? What did you pay, and why did you keep or stop paying?\n\n## 3. Sharing care with others (6 min)\n\n11. Who else looks after your child regularly?\n12. How do you tell them about the allergy now? What do they need to know?\n13. Tell me about the last time someone else fed your child. How did you find out what they ate?\n14. Who would need to see this information, and what would they need to see?\n\n## 4. Feature priority (5 min)\n\nShow or read aloud:\n- (A) Check restaurant menus for dishes that may contain the allergen.\n- (B) One shared child profile that grandparents and babysitters can see.\n\n15. If you could only have one, which one, and why?\n16. If both worked perfectly, which would you open more often?\n17. If one disappeared tomorrow, which would you miss more?\n\nThen, if they're interested, demo the scanner:\n\n18. What's missing? What would stop you from trusting a flag like this when your child is about to eat?\n\n## 5. Price (5 min)\n\nDon't ask \"would you pay EUR 6?\" Ask about current spending first, then share the price.\n\n19. What do you spend now on anything that helps your child eat safely (medicine, special products, cards)? Roughly per month?\n20. Who in your household decides on spending for this?\n21. The planned price is EUR 6 a month. What's your first reaction?\n22. What would make you say yes right away? What would make you say no?\n23. Would you pay every month, or only during some periods, such as the school year?\n\nRecord their exact words, not just \"yes\" or \"no.\"\n\n## Close (2 min)\n\n24. Is there anything I should have asked but didn't?\n25. Who else should I talk to? Would you introduce me?\n26. If we launch, can I add you to a waitlist or early-access list? Note whether they give an email. That's a stronger signal than a verbal yes.\n\n## After each call\n\nRecord:\n- How often they eat out, and the last incident\n- Their top feature choice, with the reason\n- Their exact reaction to EUR 6 and what would make them say yes or no\n- Whether they gave an email for the waitlist\n- Which of your two beliefs this call supports or contradicts (see questions 2, 4 and 16)\n\nTreat a \"yes\" with no specifics as no signal.\n\n## Sampling caveat\n\nEight people from one Facebook group are self-selected. They're active, anxious about allergies, and likely more engaged than the average parent. Results may overstate demand. Note this when you summarize.\n"}]}, "opus-1": {"cost_usd": 0.12704, "seconds": 41, "output_tokens": 3679, "files": [{"name": "interview-guide.md", "text": "# Customer Interview Guide: Allergy Menu Scanner + Shared Profile\n\n**Format:** 30 min video call, 8 parents from the local allergy-parents Facebook group\n**Goal:** Find out (1) whether parents will pay EUR 6/month, and (2) which matters more to them: the menu scanner or the shared caregiver profile.\n\n---\n\n## How to use this guide\n\n- **Ask about the past, not the future.** \"Tell me about the last time\u2026\" gets you real behavior. \"Would you\u2026?\" gets you polite guesses. Parents will want to be nice to a fellow allergy dad, so hypothetical questions will mostly tell you \"yes.\"\n- **Don't show or describe the app until the last section.** Once they know what you built, everything they say gets coloured by it.\n- **Test your beliefs, don't confirm them.** You believe (a) eating out is the scariest moment and (b) parents would use the app every time they eat out. Neither is mentioned in the questions on purpose. Watch whether they bring it up themselves.\n- **Talk less than 20% of the time.** When they stop talking, wait 3 seconds before you say anything. Follow-ups that work: \"Tell me more,\" \"Why was that?\", \"What happened next?\", \"Can you give me an example?\"\n- **You share their experience, which is a risk.** If they say something you relate to, don't say \"same with my son!\" That moves the conversation to your story. Write it down and keep going.\n\n---\n\n## 0. Opening (2 min)\n\n> \"Thanks for making time. I'm a parent of a kid with food allergies too, and I'm trying to understand how other families handle day-to-day life with it. I'm not selling anything today. There are no right answers, and the most helpful thing you can do is tell me how things really are, including the boring or messy parts. Is it OK if I take notes?\"\n\nDon't mention the app or the scanner.\n\n---\n\n## 1. Context (3 min)\n\n1. Tell me a bit about your child and their allergies. How old are they, and how long have you known?\n2. Who looks after your child day to day? Who else looks after them sometimes (grandparents, babysitters, school, other parents)?\n\n*Listen for: how many caregivers there are, and how severe the allergy is (EpiPen? past reactions?).*\n\n---\n\n## 2. Where the worry actually is (7 min)\n\n*This tests your belief that eating out is the scariest moment. Don't suggest it.*\n\n3. Thinking about the last month, when did managing the allergy take the most effort or worry?\n4. Tell me about the last time you felt really anxious about your child's allergy. What was happening?\n5. Has your child had a reaction, or a close call, in the last year or two? What happened? Where were they, and who were they with?\n6. If you could make one part of living with this allergy go away, what would it be?\n\n*Score afterwards: did they bring up restaurants without being asked? Or school, birthday parties, other people's houses, packaged food, travel?*\n\n---\n\n## 3. Eating out: actual behavior (7 min)\n\n*Ask these even if they didn't bring up restaurants. How often they eat out matters for the \"every time\" belief.*\n\n7. When did you last eat out with your child? Walk me through it, from choosing where to go to getting the bill.\n8. How often do you eat out as a family? Has that changed since the diagnosis?\n9. How do you decide whether a restaurant is safe? What did you do last time?\n10. Have you ever had a bad experience at a restaurant, or a near miss? What happened?\n11. Do you use anything to help with this today? Apps, cards for the chef, a list of \"safe\" places, Facebook group recommendations? What do you like about it, and what annoys you?\n12. Have you ever paid for anything to help manage the allergy (apps, chef cards, courses, a consultant, special products)? What was it, and how much did it cost?\n\n*Listen for: workarounds they've built themselves (strong signal), money already spent (the best predictor of whether they'll pay), and whether they stick to a few trusted places (that would weaken the scanner's value).*\n\n---\n\n## 4. Other caregivers: actual behavior (6 min)\n\n*This tests the shared-profile feature.*\n\n13. Tell me about the last time someone else looked after your child for a meal or more. How did you get them ready?\n14. What did you tell them about the allergy? How: in person, written down, a WhatsApp message, a note on the fridge?\n15. Has anything gone wrong, or nearly gone wrong, when someone else was in charge? What happened?\n16. How confident are you that grandparents or babysitters really understand what's safe? Why?\n17. How often does this handover happen? Weekly, monthly, a few times a year?\n\n*Listen for: how often handovers happen compared with eating out, and how much worry each one causes. You can compare this directly with section 3 to work out which feature matters more.*\n\n---\n\n## 5. Reaction to the idea and commitment (5 min)\n\n*Only now describe what you've built, in two sentences and in neutral terms:*\n\n> \"I've been working on an app with two parts. One scans a restaurant menu and flags dishes that likely contain your child's allergens. The other is a shared allergy profile that grandparents and babysitters can see on their phones.\"\n\n18. What's your first reaction? *(Then be quiet.)*\n19. If you could only have one of those two, which would you pick? Why?\n20. Think of the last restaurant visit you told me about. Would the scanner have changed anything? What exactly?\n21. What would worry you about trusting an app to flag allergens? *(Liability and accuracy concerns are important here. Note them word for word.)*\n22. **Pricing:** \"It would cost EUR 6 a month.\" Then stay quiet and watch how they react before you ask anything else. Follow-up: \"How does that compare to what you'd expect?\"\n\n**Commitment ask (pick one, and always ask it):**\n- \"I'm running a small pilot next month. Can I sign you up now? It's EUR 6 for the first month, or a pre-order.\" *(Strongest signal.)*\n- \"Would you give me the names of the grandparents or babysitter so we can test the shared profile with them?\" *(Shows the caregiver problem is real.)*\n- \"Could you introduce me to two other parents who'd be good to talk to?\"\n\nA friendly \"sounds great, let me know when it's out\" with no commitment counts as a **no**.\n\n---\n\n## 6. Wrap-up (1 min)\n\n23. Is there anything about living with your child's allergy that I should have asked about but didn't?\n24. Can I follow up with you if I have another question?\n\n---\n\n## After each interview: 5-minute debrief sheet\n\nFill this in straight after the call, before the next one.\n\n| Question | Notes |\n|---|---|\n| Biggest worry they named *without being asked* | |\n| Did they bring up restaurants on their own? (Y/N) | |\n| How often they eat out with the child | |\n| How often someone else cares for the child | |\n| Reactions or close calls: where and with whom | |\n| Current workarounds / tools | |\n| Money already spent on allergy management | |\n| Scanner vs. profile preference (and why) | |\n| Trust/accuracy concerns | |\n| Reaction to EUR 6/month (words + body language) | |\n| Commitment given? (pre-order / intro / caregiver contacts / none) | |\n| Surprising quote | |\n\n## After all 8: what to look for\n\n- **Belief \"eating out is the scariest moment\":** Count how many of the 8 brought up restaurants unprompted as their top worry. If fewer than half did, the scanner may not be the main product.\n- **Belief \"they'd use it every time they go out\":** Look at how often they eat out (Q8) and how many stick to the same trusted places (Q9). Families who eat out once a month at two places they already know don't need a scanner every time.\n- **Which feature matters more:** Compare how often each situation happens and how many incidents or close calls each one caused (Q5, Q10, Q15). Also check the forced choice in Q19. What they've actually done counts for more than what they say they'd pick.\n- **Willingness to pay EUR 6:** Count the real commitments, not the \"yes\"es. Money already spent on similar things (Q12) is your best supporting evidence. If you get 0 or 1 pre-orders out of 8, take the price or the problem seriously.\n- **Sample caveat:** Parents in an allergy Facebook group are more engaged than average. If they won't commit, most parents won't either. If they do, still check with parents outside the group before you scale up.\n"}]}, "opus-2": {"cost_usd": 0.13554, "seconds": 46, "output_tokens": 4038, "files": [{"name": "interview-guide.md", "text": "# Customer Interview Guide \u2014 Food-Allergy Parents\n\n**Format:** 30-minute video call, 8 parents from the local allergy-parents Facebook group\n**Goal:** Learn how parents handle allergy risk today. Answer two questions without leading anyone to them:\n1. Would they pay EUR 6/month?\n2. Which matters more: the **menu scanner** or the **shared caregiver profile**?\n\n---\n\n## Before the calls: ground rules\n\nYou're a fellow allergy parent and you've built a prototype. People in the group will want to be supportive, and that makes it easy to collect polite answers that turn out to be false.\n\n- **Ask about the past, not the future.** \"Tell me about the last time\u2026\" is reliable. \"Would you\u2026?\" mostly gets you politeness.\n- **Don't pitch until the last 5 minutes.** Don't mention the app, the scanner or the price before then. If they ask, say: \"I'll tell you everything at the end. First I want to hear how it goes for you.\"\n- **Treat your own beliefs as things to test.** You believe (a) eating out is the scariest moment and (b) parents would use the app every time they go out. Listen for evidence against both. Section 2 doesn't name restaurants for exactly this reason.\n- **Talk less than 20% of the time.** Silence is fine. Count to five before you fill a pause.\n- **Dig into emotion and workarounds.** If they say \"that was stressful\" or \"I always\u2026\", follow up with \"Can you walk me through that?\"\n- **Don't share your own story early.** It anchors them on peanuts, restaurants and your way of coping. A sentence of rapport at the start is fine.\n- **Record the call (with permission)** or have someone else take notes, so you can stay present.\n\n---\n\n## The script\n\n### 0. Opening (2 min)\n\n> \"Thanks for making time. I'm a parent of a kid with food allergies too, and I'm trying to understand how other families handle it day to day. There are no right answers, and nothing you say will hurt my feelings. Honestly, the most useful thing is if you tell me where things *aren't* a problem. Mind if I record so I don't have to scribble?\"\n\n### 1. Context (3 min)\n\n1. Tell me a bit about your child and their allergies. How old are they, what are they allergic to, and how long have you known?\n2. How severe is it? Have you ever had to use an EpiPen or go to the ER?\n3. Who looks after your child besides you? (Partner, grandparents, babysitter, daycare, school, friends' parents)\n\n*Note: allergen(s), severity, age, number of other caregivers. These are your segmentation variables.*\n\n### 2. Where the risk actually lives (7 min) \u2014 *tests \"eating out is scariest\"*\n\n4. Think about the last month. When were you most worried about your child's allergy? What was going on?\n5. Walk me through a normal week. Where does your child eat food you didn't make yourself?\n6. Has there been a close call or a reaction? Tell me about the most recent one: where it happened, who was there, and what went wrong.\n7. If you could make one situation completely safe, which would it be?\n\n*Don't suggest restaurants. If they bring them up unprompted, that counts. If they talk about school, birthday parties or grandparents first, that counts too.*\n\n### 3. Eating out (7 min) \u2014 *tests the scanner and \"every time\"*\n\n8. When did you last eat out as a family? Walk me through it, from picking the place to the food arriving.\n9. How did you decide what your child could eat? Who did you ask, and what did you check?\n10. How often do you eat out? Has that changed since the diagnosis?\n11. Are there places you go back to because you trust them? How did that trust get built?\n12. Has a restaurant ever got it wrong? What happened?\n13. What do you do when you travel or try somewhere new?\n14. Have you used anything to help with this, like apps, allergen cards, websites or the restaurant's allergen sheet? What did you think of it? Do you still use it? If not, why did you stop?\n\n*Listen for: does the menu actually matter, or do they rely on talking to staff or the kitchen? Do they mostly stick to a few trusted places, so there'd be little to scan? Did they try tools and drop them?*\n\n### 4. Other caregivers (6 min) \u2014 *tests the shared profile*\n\n15. When did your child last spend time with someone else, like grandparents or a babysitter? How did you get them ready?\n16. What do you tell them about the allergy? How do you share it: in person, a note on the fridge, WhatsApp?\n17. Has a caregiver ever made a mistake or nearly made one? What happened?\n18. What worries you most when you're not there?\n19. How do you handle school or daycare? Any forms, action plans or meetings?\n\n*Listen for: how often this happens, whether the current method actually fails, and whether the worry is about information (\"they didn't know\") or judgment (\"they knew but didn't check the label\").*\n\n### 5. Money and effort already spent (3 min) \u2014 *the real willingness-to-pay signal*\n\n20. What do you currently spend money on because of the allergy? Think of special products, EpiPens, allergists, courses, apps or memberships.\n21. Have you ever paid for something just to feel safer or save time? What was it, and was it worth it?\n22. Are you paying for any app subscriptions at the moment? Which ones do you think are worth it?\n\n*Paying already for related things counts as strong evidence. \"I'd definitely pay for that\" counts as weak evidence.*\n\n### 6. Reveal and reaction (5 min)\n\nOnly now describe the idea, in neutral terms and briefly:\n\n> \"I'm exploring an app with two parts. One: you take a photo of a restaurant menu and it flags dishes that probably contain your child's allergens. Two: a shared allergy profile for your child that grandparents and babysitters can open on their phone.\"\n\n23. Which of the two, if either, would have helped in a situation you told me about earlier? *(Point to their own stories.)*\n24. If you could only have one, which would it be, and why?\n25. What would make you *not* trust the scanner?\n26. It would cost EUR 6/month. What's your gut reaction? *(Then stay quiet.)*\n27. **Commitment test:** pick one and offer it:\n    - \"I'm looking for a few families to try the prototype at their next restaurant visit. Can I put you down?\"\n    - \"Can I sign you up for early access at EUR 6/month, with the first month free?\"\n    - \"Would you introduce me to the grandparent or babysitter who looks after your child most often?\"\n\n*Write down exactly what they agree to. A yes that costs them time, money or an introduction is a real signal. \"Sounds great, let me know when it's out\" is a polite no.*\n\n### 7. Wrap-up (2 min)\n\n28. What should I have asked that I didn't?\n29. Who else should I talk to? Maybe a parent whose situation is very different from yours?\n\n---\n\n## After each call: 5-minute debrief\n\nFill this in right away, before the next call.\n\n| Field | Notes |\n|---|---|\n| Name / child age / allergens / severity | |\n| Number of other regular caregivers | |\n| Scariest situation they named **unprompted** | |\n| Eat-out frequency, and has it dropped? | |\n| How they check menus today (staff, sheet, app, avoid) | |\n| Tools tried and abandoned, and why | |\n| Caregiver mistakes or close calls, and current sharing method | |\n| Allergy spending today | |\n| Preferred feature, and the story behind it | |\n| Reaction to EUR 6/month (exact words) | |\n| Commitment given (trial / pre-pay / intro / none) | |\n| Surprise: what contradicted what I expected? | |\n| Best quote | |\n\n---\n\n## After all 8: how to read the results\n\n| Question | Evidence that supports it | Evidence against it |\n|---|---|---|\n| Eating out is the scariest moment | Most parents name restaurants **unprompted** in Q4/Q7 | School, parties or grandparents come up first; parents say \"we just don't eat out\" |\n| They'd use it every time they eat out | Eat out often, try new places, check menus themselves | Stick to 2\u20133 trusted places, rely on staff, rarely eat out |\n| Scanner > shared profile | Restaurant close calls, menu-checking pain | Caregiver mistakes, regular babysitter or grandparent care, WhatsApp-note workarounds |\n| Would pay EUR 6/month | Already pay for allergy-related products or apps; say yes to the pre-pay or trial | Only use free tools, gave up on apps before, \"let me know when it's out\" |\n\n**Rough bar:** you'll want at least 3 of 8 to agree to a concrete next step (trial, pre-pay or intro) before you treat the EUR 6/month question as a likely yes. Fewer than that, or polite enthusiasm with no commitments, means the problem or the price needs another look.\n\n**Watch out:** all 8 parents come from one active Facebook group. They're likely more engaged than the average allergy parent, so a \"yes\" here is an upper bound. Q29 can help you reach beyond the group.\n"}]}}, "verdicts": [{"pair": "1", "judge": "gpt", "order": "haiku shown first", "winner": "haiku", "why": "A investigates purchasing decisions and subscription justification better. Its timings total 36 minutes; B totals 31. Both lack caregiver reliability/privacy probes and complete safety questions. B invents a paid pilot and overgeneralizes noncommitment beyond the sample.", "haiku_scores": {"c1": 0, "c2": 2, "c3": 2, "c4": 1, "c5": 1, "c6": 2}, "opus_scores": {"c1": 1, "c2": 2, "c3": 1, "c4": 1, "c5": 1, "c6": 1}}, {"pair": "1", "judge": "gpt", "order": "opus shown first", "winner": "opus", "why": "A better grounds discovery and feature choice in recent experiences. Its timings total 31 minutes; B totals 36. Both miss caregiver reliability and privacy probes. A invents a pilot and overgeneralizes sample rejection; B investigates purchase decisions better.", "haiku_scores": {"c1": 0, "c2": 1, "c3": 2, "c4": 1, "c5": 1, "c6": 2}, "opus_scores": {"c1": 1, "c2": 2, "c3": 1, "c4": 2, "c5": 1, "c6": 1}}, {"pair": "1", "judge": "deepseek", "order": "haiku shown first", "winner": "tie", "why": "Both provide complete, practical interview guides that meet all rubric criteria effectively.", "haiku_scores": {"c1": 2, "c2": 2, "c3": 2, "c4": 2, "c5": 2, "c6": 2}, "opus_scores": {"c1": 2, "c2": 2, "c3": 2, "c4": 2, "c5": 2, "c6": 2}}, {"pair": "1", "judge": "deepseek", "order": "opus shown first", "winner": "opus", "why": "A more thoroughly investigates trust/safety constraints (c5) with specific probes, while B's are less detailed.", "haiku_scores": {"c1": 2, "c2": 2, "c3": 2, "c4": 2, "c5": 1, "c6": 2}, "opus_scores": {"c1": 2, "c2": 2, "c3": 2, "c4": 2, "c5": 2, "c6": 2}}, {"pair": "2", "judge": "gpt", "order": "haiku shown first", "winner": "haiku", "why": "A totals 32 minutes; B totals 38. A probes purchase decisions and subscription reasons better. B ties features to actual experiences but conflates trial commitments with payment evidence. Both omit key safety constraints and overstate sampling implications.", "haiku_scores": {"c1": 1, "c2": 2, "c3": 2, "c4": 1, "c5": 1, "c6": 1}, "opus_scores": {"c1": 0, "c2": 2, "c3": 1, "c4": 2, "c5": 1, "c6": 1}}, {"pair": "2", "judge": "gpt", "order": "opus shown first", "winner": "haiku", "why": "B better probes purchase decisions and subscription reasons. A totals 35 minutes; B totals 32. A wrongly treats trials or introductions as price validation. Both miss explicit cross-contact and profile-reliability probes and overstate recruitment-based conclusions.", "haiku_scores": {"c1": 1, "c2": 2, "c3": 2, "c4": 1, "c5": 1, "c6": 1}, "opus_scores": {"c1": 0, "c2": 2, "c3": 1, "c4": 2, "c5": 1, "c6": 1}}, {"pair": "2", "judge": "deepseek", "order": "haiku shown first", "winner": "tie", "why": "Both provide complete, practical interview guides that meet all rubric criteria accurately.", "haiku_scores": {"c1": 2, "c2": 2, "c3": 2, "c4": 2, "c5": 2, "c6": 2}, "opus_scores": {"c1": 2, "c2": 2, "c3": 2, "c4": 2, "c5": 2, "c6": 2}}, {"pair": "2", "judge": "deepseek", "order": "opus shown first", "winner": "opus", "why": "A provides a more complete, neutral, and practical guide with superior trust/safety probes and feature comparison.", "haiku_scores": {"c1": 2, "c2": 1, "c3": 2, "c4": 1, "c5": 1, "c6": 2}, "opus_scores": {"c1": 2, "c2": 2, "c3": 2, "c4": 2, "c5": 2, "c6": 2}}]}, {"id": "sal-1", "family": "founding-sales", "prompt": "I need to start selling this myself. Context in sales.md. Give me a plan for the next 90 days.", "input": "sales.md", "input_text": "Product: Crewcheck, compliance software for construction subcontractors (tracks safety certifications, insurance certificates, toolbox-talk sign-offs).\nMe: technical founder, never sold before. Co-founder is ex-site manager.\nTraction: 4 design partners using it free (electrical and HVAC subcontractors, 30-120 field workers). One said they'd pay \"if it saves the Monday paperwork\".\nPrice idea: USD 6 per worker per month.\nConstraints: 2 founders, USD 900k seed, 18 months runway, no sales hires planned until we have 10 paying customers.\nInvestors want: 10 paying customers and a repeatable sales motion by month 6.\n", "rubric": [{"id": "c1", "criterion": "Provides an executable 90-day plan for two founders, with sequenced work, owners, and measurable milestones.", "full_marks_when": "Breaks the 90 days into practical phases or weeks, assigns selling and follow-up to the technical founder and relevant discovery or field expertise to the ex-site-manager co-founder, and specifies concrete activities, outputs, and checkpoints that fit their limited capacity.", "zero_when": "Offers generic sales advice without timing, ownership, or measurable outputs, or relies on sales hires despite the hiring constraint."}, {"id": "c2", "criterion": "Uses the four free design partners to test willingness to pay and secure initial paid customers.", "full_marks_when": "Starts with conversations and observed workflows across the four partners, investigates what 'saves the Monday paperwork' means, measures a relevant before-and-after benefit, and proposes explicit paid conversion offers with decision dates. Treats the quoted comment as a hypothesis rather than proven purchase intent.", "zero_when": "Ignores the design partners, counts them as paying customers, assumes all four will convert, or treats the conditional comment as validated demand."}, {"id": "c3", "criterion": "Defines a focused initial customer and buyer based on the actual product and traction.", "full_marks_when": "Prioritizes electrical and HVAC construction subcontractors with roughly 30\u2013120 field workers, identifies likely buyers and daily users to validate, and connects outreach and demonstrations to safety certifications, insurance certificates, toolbox-talk sign-offs, and paperwork burden. Avoids unsupported promises of guaranteed compliance.", "zero_when": "Targets an unrelated or indiscriminately broad market, fails to identify whom the founders should contact, or invents product capabilities or compliance guarantees."}, {"id": "c4", "criterion": "Gives a novice seller usable instructions for acquiring, qualifying, and closing customers.", "full_marks_when": "Includes practical prospect sources, a tailored outreach example, discovery questions, a demonstration centered on the prospect's workflow, and explicit next steps toward a paid agreement. Explains how to establish urgency, buying authority, purchase requirements, and success criteria without giving away indefinite free trials.", "zero_when": "Merely tells the founder to network, cold-email, demo, or close without enough detail to conduct those activities, or substitutes sales terminology for usable guidance."}, {"id": "c5", "criterion": "Preserves the input facts and calculates proposed pricing correctly.", "full_marks_when": "Treats USD 6 per worker per month as an unvalidated price idea. If calculating account revenue, states that 30 workers cost USD 180/month and 120 workers cost USD 720/month; ten accounts within that size range would produce USD 1,800\u20137,200/month, conditional on worker counts and unchanged pricing. Distinguishes four free partners from paying customers and the 90-day planning horizon from the investors' month-six deadline.", "zero_when": "Contains a material factual or arithmetic error, presents hypothetical revenue as actual revenue, treats the suggested price as validated, or incorrectly states that investors require ten paying customers within 90 days."}, {"id": "c6", "criterion": "Builds evidence of a repeatable sales process and a credible path to ten paying customers by month six.", "full_marks_when": "Sets explicit 90-day learning and paid-customer milestones, tracks prospects through conversations, demonstrations, offers, and paid conversions, and records objections, sales-cycle length, onboarding effort, and customer usage. Uses clearly labeled funnel assumptions to connect activity to the month-six target, with checkpoints for changing the segment, message, offer, or pricing when results fall short.", "zero_when": "Promises ten customers without an acquisition rationale, equates repeatability with a CRM or documented script alone, or measures only outreach volume while ignoring paid conversion and customer adoption."}], "runs": {"haiku-1": {"cost_usd": 0.01067, "seconds": 69, "output_tokens": 13574, "files": [{"name": "sales-plan-90-day.md", "text": "# Crewcheck: 90-day founder-led sales plan\n\nWindow: 2026-10-08 (Day 1) to 2027-01-06 (Day 90).\nInvestor milestone: 10 paying customers and a repeatable sales motion by month 6 (about April 2027).\n\n## Starting point\n\n- 4 design partners (electrical and HVAC subcontractors, 30-120 field workers) are using Crewcheck free. None pay yet.\n- \"I'd pay if it saves the Monday paperwork\" is a conditional, not a commitment. Treat it as unvalidated demand.\n- At USD 6/worker/month, a 60-worker customer is about USD 4.3k a year. Ten customers at that size is about USD 43k ARR. Acquisition cost has to stay low, so the motion is founder-led, referral-heavy outreach. No paid ads, no field sales, no custom work for small accounts.\n- Two founders, no sales hires until 10 paying customers. Plan on roughly half of both founders' time going to sales.\n\n## Targets\n\n| Checkpoint | Paying customers | Discovery calls done | Paid pilots live | Also true |\n|---|---|---|---|---|\n| Day 30 (Nov 7) | 2 | 15 | 3 | All 4 design partners have given a clear yes or no on paid terms |\n| Day 60 (Dec 7) | 4 | 35 | 6 | Partner referrals are producing; each lost deal has a written reason |\n| Day 90 (Jan 6) | 6 (stretch 8) | 60 | 8 | Written playbook with conversion rates from real data |\n\nTen paying by month 6 requires the Day 90 pipeline above plus about two new paying customers a month from month 4 onward. Agree this checkpoint with investors now, so the Day 90 number is not a surprise.\n\n## Roles\n\n- **Technical founder (you):** demos, pilot setup and onboarding, product fixes that unblock deals, email and LinkedIn outreach to owners, and the funnel tracker. Also fixes anything that stalls a deal.\n- **Co-founder (ex-site manager):** leads discovery calls with owners and foremen, works the referral network (trade associations, GC relationships), and is the credible voice with site-level buyers.\n- **Both:** attend the first call with each new owner and the Monday pipeline review.\n\n## Who buys\n\n- **Economic buyer:** subcontractor owner or operations/office manager. Pays the bill.\n- **Champion and daily user:** site manager or foreman. Does the Monday paperwork and feels the pain. Gets them on your side early.\n- **Forcing function:** the general contractor (GC) who requires subcontractors to submit certificates and certifications. If GCs mandate Crewcheck, adoption becomes much easier. Test this in Phase 2.\n- **Ideal customer, v1:** electrical and HVAC subs with 30-120 field workers that already submit compliance paperwork to GCs. Confirm the target metro before building the list.\n\n## Offer and pricing\n\n- Keep USD 6/worker/month as list price for now. Test a monthly floor (for example USD 200) and an annual-prepay discount. Decide by Dec 7 using pilot data. Do not change terms mid-pilot.\n- Define \"worker\" for billing before the first paid pilot (for example, workers active on a site during the billing month). Per-worker pricing creates disputes if this is vague.\n- Run every new account as a **paid 30-day pilot** with written success criteria agreed on day 1, such as \"Monday paperwork cut from X hours to Y, zero expired certificates missed.\" Hold a go/no-go call on day 30.\n- Get each design partner's current weekly paperwork hours now. These numbers drive the ROI pitch.\n- ROI formula for the pitch: hours saved per week x 50 weeks x loaded hourly cost of whoever does the work. Placeholder example: 4 hours x 50 weeks x USD 50/hour = USD 10k a year, against a USD 2-9k annual price. Replace the placeholders with partner data before using it.\n\n## Phase 1: Days 1-30 (Oct 8 to Nov 7). Convert and define\n\n**Week 1**\n- Meet each design partner. Ask for a paid start date and a price. If they say no, ask what is missing and get the answer in writing. Set a two-week decision deadline.\n- Collect paperwork hours and a current example of a Monday workflow from each partner.\n- Set up a simple tracker (a spreadsheet is fine) with the stages in the sales process below.\n\n**Weeks 2-4**\n- Build a target list of 250-300 subcontractors in one metro, from trade association member directories and the GC vendor lists the partners share.\n- Write a one-page pitch and ROI sheet. Build a 15-minute demo around one workflow only: Monday toolbox-talk sign-offs plus certificate expiry alerts.\n- Ask each partner for two introductions. Offer one free month for each referral that becomes a paying customer.\n- Complete 15 discovery calls. Take notes on the exact words owners use about their paperwork.\n\n**Phase 1 outputs:** ICP v1, pitch v1, pricing draft, 15 call notes, 2 paying customers (target).\n\n## Phase 2: Days 31-60 (Nov 8 to Dec 7). Run the motion\n\n- **Outbound:** about 12 new contacts a week, by email, LinkedIn, and phone. Owners are on site early, so call before 8 a.m. or after 4 p.m.\n- **Demos:** about 2 a week, each followed by a pilot offer or a clear no.\n- **Paid pilots:** start with the 4 partners plus new accounts. Use the same success criteria template for every pilot.\n- **GC test:** hold two conversations with GCs that the partners work with. Ask: do you require compliance documents from subs, who collects them today, and would you require a shared tool?\n- **Trade association:** one talk, newsletter item, or member listing. This is cheap and builds credibility.\n- **Log every objection** with the exact wording. Review the log every Friday.\n\n**Phase 2 outputs:** 4 paying customers, the first referral source measured, the GC answer, a decision on price floor.\n\n## Phase 3: Days 61-90 (Dec 8 to Jan 6). Make it repeatable\n\n- The holidays (about Dec 20 to Jan 3) slow buyers and sites. Use that stretch to write the playbook and close pilots that are already running, not to start new outreach.\n- **Playbook v1:** ICP, discovery script, demo script, pilot terms and success criteria, onboarding checklist, objection responses, pricing rules, and conversion rate per stage.\n- **Funnel review:** find the leakiest stage and fix one thing there.\n- **Onboarding time:** measure days from signed contract to first workers added. Target under 7 days. Slow onboarding loses customers before they renew.\n- **References:** ask every paying customer for a named case study with the hours-saved number. These become the core of the sales deck.\n\n**Phase 3 outputs:** playbook v1, funnel metrics, 6 or more paying customers.\n\n## Funnel math for 10 paying customers\n\nAssumed conversion rates (all to be measured in Phase 1-2):\n\n| Stage | Assumed rate | Volume needed for 6 new paying customers |\n|---|---|---|\n| Contact to reply | 25% | 160 contacts |\n| Reply to discovery call | 60% | 40 calls |\n| Call to demo/pilot offer | 60% | 24 demos |\n| Demo to paid | 25% | 6 customers |\n\nPlus the 4 partners, that gives 10. If reply rates are weaker, plan for 250 contacts. At 12 contacts a week, that is about 20 weeks of outreach, which fits between November and March.\n\n## Weekly rhythm\n\n- **Monday (30 min, both founders):** pipeline review. Which deals move this week? What is blocked?\n- **Tracked weekly:** new contacts, replies, calls, demos, pilots started, paid conversions, MRR, pilot results (hours saved), and at-risk accounts.\n- **Friday:** update the objection log and the lost-deal reasons.\n\n## Decision points\n\n- **Day 30:** if fewer than 2 partners give a yes or no, escalate. The problem is the relationship or the price, not outbound.\n- **Day 60:** fewer than 2 paid conversions from partners or pilots means value or buyer is the problem. Interview the partners who said no before changing anything else.\n- **Day 60:** fewer than 10 replies from outreach means the message or the list is wrong. Rewrite the pitch and narrow the list before adding volume.\n- **Any time:** a pilot that misses its success criteria means the product or ICP needs work. Do not solve it with a discount.\n- **If the GC test shows strong pull:** move a share of outreach to GCs and their subcontractor lists.\n\n## Risks\n\n- **Partners never pay.** Mitigation: the Week 1 decision deadline.\n- **Founders are new to sales.** Read *The Mom Test* by Rob Fitzpatrick for discovery calls. Find one advisor who has sold to contractors and ask them to review call recordings or notes.\n- **Long, seasonal cycles.** Winter and the holidays slow decisions. Keep pilots short so they finish before the slowdown.\n- **Data sensitivity.** Certificates and worker details are personal data. GCs will send security questionnaires. Prepare a short security and data-handling page and a standard data processing agreement now. Only pursue SOC 2 if GCs require it.\n- **Low contract value.** Keep acquisition cost per customer low. Turn down custom work that is not needed by two or more customers.\n\n## Questions to answer this week\n\n1. Which metro and which trade come first? This decides the target list and the GC market.\n2. Has any partner signed anything or paid anything yet?\n3. How long does onboarding take today, from signed to first workers added?\n4. How do you want to define \"worker\" for billing?\n5. Are the partners' hours-saved numbers measured or estimated?\n"}]}, "haiku-2": {"cost_usd": 0.00814, "seconds": 50, "output_tokens": 9518, "files": [{"name": "90-day-sales-plan.md", "text": "# Crewcheck: 90-day founder-led sales plan\n\nDay 0 = 2026-10-08. Day 90 = 2027-01-06.\n\n## Starting position\n\n- Four free design partners are the fastest route to revenue. Converting them is the first deliverable, not a side task.\n- Your co-founder (ex-site manager) is the credibility asset with field buyers. They should lead discovery and demos with subs. You own pricing, contracts, onboarding, and writing the playbook.\n- Targets: 6 paying customers by Day 90, 10 by Day 180 (investor milestone). These are my proposed paces. Revisit at Day 30 and Day 60.\n\n## Check the math first\n\n- At USD 6 per worker, a 75-worker sub pays USD 450/month, about USD 5.4k/year.\n- Ten customers at that size is about USD 54k ARR.\n- Burn is roughly USD 50k/month (900k over 18 months), before any revenue.\n- So \"10 paying customers\" is a logo milestone, not a revenue one. Agree with investors now what month 6 must prove: average account size, pricing, and a credible path to 100+ customers.\n- A 30-worker sub pays USD 180/month. Test a monthly minimum on new accounts rather than guessing whether small subs are worth selling to.\n\n## Ideal customer (write down by Day 7)\n\n- Electrical and HVAC subcontractors with 30-120 field workers.\n- Buyer: the owner or office/ops manager who does the Monday paperwork. Confirm this with each partner.\n- Trigger: general contractor (GC) requirements for site entry, prequalification, or insurance certificates. Ask partners which GCs require which documents and how often paperwork gets rejected. This may be a stronger reason to buy than the sub's own pain.\n\n## Phase 1: Days 1-30, convert and prove\n\nGoal: 3 paying partners, a documented ROI, a pricing and contract template, and a target list.\n\nWeek 1\n- Book a 30-minute conversion call with each of the four partners. Ask for a paid start date and an invoice or card on file.\n- Offer a founder price lock (for example, 12 months at the current rate) in exchange for a reference and a case study. This is a fair trade and gives them a reason to commit.\n- Ask each partner for numbers: hours per week on Monday paperwork before and after, documents GCs asked for, and what they would lose without it.\n- Write the ICP and a one-page offer.\n\nWeeks 2-4\n- Build a list of 150 target subs in your market. Columns: trade, estimated headcount, website or LinkedIn, known GC relationships, and state license lookup status.\n- Use a two-page pilot-to-paid agreement: 30-day pilot, monthly billing, cancel anytime, converts to a paid order form.\n- Turn the best partner results into one case study.\n- Start 5 discovery calls per week. Ask about last Monday, not whether they would buy. Read *The Mom Test* (Rob Fitzpatrick) before starting.\n\nEnd of Day 30: 3 paying customers, 1 case study, 40+ target accounts with notes, 10 discovery calls done.\n\n## Phase 2: Days 31-60, build a repeatable motion\n\nGoal: find the two channels that produce deals, and run a measured pipeline.\n\nRun these channels in parallel, then keep the best two by Day 60:\n\n1. **Partner referrals.** Ask each paying partner for two introductions to other subs. Offer a free month for each closed referral.\n2. **Co-founder's network.** Former site managers and superintendents, many of whom now work for subs or GCs.\n3. **Insurance brokers who serve subcontractors.** They handle certificates of insurance every day. Pitch them as a referral partner. This is the cheapest test and potentially the highest leverage.\n4. **Trade associations and local contractor groups.** Speak at one meeting or sponsor one breakfast. Slow, but builds trust.\n5. **Founder-led outbound** to the 150-account list. Target 20 emails and 10 calls per day. A/B test the GC-requirement message against the time-saved message.\n\nStages to track weekly: Lead, Discovery call, Demo (30 minutes, on-site or video), Pilot (2-4 weeks, one crew), Paid. Record conversion at each stage.\n\nDemo structure: start with their last Monday, walk through the three documents (safety certs, insurance certificates, toolbox-talk sign-offs), show how each is tracked, then offer a pilot with no commitment to start.\n\nObjection answers: write a one-line response after the first 10 calls for each of these:\n- \"We already track this in a spreadsheet, GC portal, or project software.\"\n- \"Our office manager handles it.\"\n- \"Our crew size changes, so per-worker pricing is hard to predict.\"\n- \"Does our GC accept what Crewcheck produces?\"\n- \"Who owns the data if we leave?\"\n\nEnd of Day 60: 5 paying customers, at least one won deal from each of the two chosen channels, and known conversion rates per stage.\n\n## Phase 3: Days 61-90, scale what worked\n\n- Double activity on the two channels that produced deals.\n- Write a playbook: ICP, demo script, pilot agreement, onboarding checklist, objection answers, pricing rules. The first sales hire will use it.\n- Make onboarding repeatable so a new sub is live in under a day with minimal founder time. Measure days from signature to first tracked document.\n- Offer the minimum-fee tier to one cohort of new prospects and track whether it closes.\n- Holiday caution: late December is slow for construction. Front-load outreach into November and early December.\n\nEnd of Day 90: 6 paying customers, 8-10 active pilots or late-stage deals, 2 case studies, and a weekly pipeline report with stage conversion rates.\n\n## Weekly operating rhythm\n\n- **Daily:** one 60-minute outbound block and one 60-minute follow-up block.\n- **Monday:** 30-minute pipeline review. Count new conversations, demos, pilots started, closes, and MRR.\n- **Friday:** one call with a paying customer, to watch for churn risk and collect a reference or a quote.\n- **Weekly targets from Week 5:** 5 discovery calls, 3 demos, 1 pilot started, and 1 close from Month 2 onward.\n- Protect three days a week for selling. Build only what a paying customer has asked for.\n\n## Hiring\n\n- Keep your rule: no sales hires until 10 paying customers.\n- Start recruiting about two months before you expect to reach 10. The first hire should look like your co-founder: field-experienced (site manager, superintendent, or estimator) and comfortable selling to owners.\n\n## Decisions to make by Day 30\n\n1. Monthly minimum fee, or per-worker pricing only.\n2. Terms of the founder price lock.\n3. Whether investors accept 10 logos as the month-6 bar, or want an ARR figure.\n\n## Risks\n\n- **Partners who say yes but don't pay.** Set a paid start date. If it slips, remove them from the paying count. Do not count free users as customers.\n- **Single-trade concentration.** If the first six customers are all electrical, test HVAC deliberately, or the playbook will be narrower than it looks.\n- **Founder bandwidth.** Product bugs will pull you back into engineering. Schedule sales time as protected blocks.\n- **Never having sold.** Run 10 discovery calls before pitching anything. Listen for the Monday-paperwork detail and repeat it back.\n\n## Open questions\n\n1. Which region or state? This determines the target list and licensing lookups.\n2. What do partners pay for similar tools today, if anything?\n3. Do GCs currently reject subs' paperwork? How often, and for what reasons?\n4. Does Crewcheck integrate with any GC portal or project software? If not, is that a blocker for the buyers you're targeting?\n5. How long does a new sub take to get to their first tracked document today?\n"}]}, "opus-1": {"cost_usd": 0.21856, "seconds": 82, "output_tokens": 7373, "files": [{"name": "90-day-sales-plan.md", "text": "# Crewcheck: 90-Day Founder Sales Plan\n\n**Window:** Mon Oct 12, 2026 \u2192 Wed Jan 6, 2027 (13 weeks)\n**Who sells:** you (technical founder) + co-founder (ex-site manager). No hires.\n\n---\n\n## 1. Goal for the 90 days\n\nThe investor goal is **10 paying customers and a repeatable motion by month 6**. These 90 days have to get you most of the way there and show the rest is coming.\n\n| By Jan 6, 2027 | Target |\n|---|---|\n| Paying customers (signed and invoiced) | **6** (3 design partners + 3 new) |\n| Paid pilots running that should convert by month 6 | **4+** |\n| Qualified pipeline (had a discovery call, fits the ICP) | **25 companies** |\n| Written playbook with real stage-by-stage conversion rates | **Done** |\n\nWhy 6 and not 10: construction buying slows from Thanksgiving through New Year. Plan to close the remaining 4 in Jan\u2013Feb. **Check when \"month 6\" falls on your calendar.** If it lands before March, raise the outreach volume in Phase 2 by about 50%.\n\n---\n\n## 2. Pricing: decide this week, then stop revisiting it\n\n$6/worker/month comes to $180\u2013$720/month for your design partners (30\u2013120 workers). That range is fine, but add three things:\n\n1. **Monthly minimum of $250.** A 25-person shop shouldn't cost you more to support than it pays.\n2. **Annual prepay at 2 months free.** This helps cash and gives you cleaner numbers for investors.\n3. **Founding-customer price for design partners only:** $4/worker locked for 12 months if they sign before Nov 30. This rewards them and gives them a deadline.\n\nDon't discount any further. If a prospect objects to price, it usually means they aren't convinced of the value yet. Go back to the time-savings numbers (Section 4) instead of lowering the price.\n\n---\n\n## 3. Who to sell to (ICP)\n\n**Primary:** electrical, HVAC, and mechanical subcontractors with **30\u2013150 field workers** who work for GCs that require prequalification, certificates of insurance (COIs), and proof of safety training.\n\n**Signs they're a good fit:**\n- Someone (owner, office manager, or safety coordinator) spends hours each week chasing certs and sign-off sheets\n- They have been held up on a job, or nearly were, because of an expired cert or missing COI\n- They work for at least one GC that uses a prequal platform or runs strict jobsite audits\n- They are growing: adding crews or bidding bigger jobs\n\n**Trigger events to look for:** new large GC contract, OSHA visit or citation, insurance renewal, a new safety coordinator, an audit failure.\n\n**Skip for now:** companies under 20 workers (too small to justify the cost), over 300 workers (long procurement and they already have tools), and GCs as direct buyers (a different product).\n\n**Buyer:** usually the owner or ops manager. The safety coordinator or office manager feels the problem most and will champion it. Get both on the call.\n\n---\n\n## 4. Phase 1 (Weeks 1\u20133, Oct 12 \u2013 Nov 1): Turn design partners into paying customers and evidence\n\nThis is the most important phase. If design partners who use the product free won't pay, cold prospects won't either. Find that out now.\n\n### Week 1\n- [ ] **Measure the \"Monday paperwork.\"** Your co-founder visits or calls each design partner and records, before and after Crewcheck: hours per week on cert/COI tracking, toolbox-talk sign-offs, and audit prep. Collect real numbers.\n- [ ] Set up a simple CRM. A spreadsheet is fine (template: `pipeline-tracker.csv`). Log every touch from day one, because the playbook depends on this data.\n- [ ] Settle pricing (Section 2). Write a one-page order form and set up Stripe invoicing.\n\n### Week 2\n- [ ] **Have the conversion conversation with each design partner** (script in Appendix A). Ask directly: \"We're starting to charge on Dec 1. Here's your founding price. Will you sign?\"\n- [ ] Ask each partner to **(a)** be a reference, **(b)** introduce you to 2 other subs they know (other trades on the same jobs, peers at association meetings), and **(c)** name the GCs they work for.\n\n### Week 3\n- [ ] Write 2 short case studies from the time data (\"ABC Electric cut Monday paperwork from 6 hours to 45 minutes\").\n- [ ] Build the target list: **150 companies** that fit the ICP in 1\u20132 metro areas near you or your partners. Sources: state contractor license lookups, local IEC/NECA/ABC (electrical), SMACNA/ACCA/MCAA (HVAC/mechanical) member directories, GC approved-subcontractor lists, and LinkedIn.\n- [ ] Join 1\u20132 local trade association chapters and sign up for their next meetings.\n\n**Phase 1 exit criteria:** at least 3 of 4 design partners have signed, plus at least 6 warm introductions. If fewer than 2 sign, **stop and fix the product or the value story before scaling outreach** (see Section 9).\n\n---\n\n## 5. Phase 2 (Weeks 4\u20138, Nov 2 \u2013 Dec 6): Build pipeline and run paid pilots\n\n### Who does what\n\n| Co-founder (ex-site manager) | You (technical founder) |\n|---|---|\n| Opens doors: calls, association meetings, site visits, warm intros | Runs demos and handles technical, security, and data questions |\n| Leads discovery (\"I ran sites, I know your Monday\") | Onboards pilots and fixes blockers within 48 hours |\n| Builds credibility with owners and safety staff | Keeps the CRM, tracks conversion, writes the playbook |\n\nConstruction buyers trust people who have run sites, so your co-founder should lead the first conversation. You win the second one by showing the product works and that you fix things fast.\n\n### Weekly activity targets (combined)\n\n| Activity | Per week | 5-week total |\n|---|---|---|\n| New companies contacted (phone first, then email/LinkedIn) | 40 | 200 |\n| Live conversations (reached a decision-maker) | 12 | 60 |\n| Discovery calls / demos | 5 | 25 |\n| Paid pilots started | 1\u20132 | 7 |\n\n**The expected funnel is an assumption to test, not a forecast:** 200 contacted \u2192 60 conversations \u2192 25 demos \u2192 7 pilots \u2192 4 paid. Update these rates every Friday using real data.\n\n### Channels in priority order\n1. **Warm intros from design partners.** These convert best. Follow up within 24 hours.\n2. **Phone calls.** Owners and ops managers in this trade answer the phone more reliably than email. Call 7\u20138am or 3:30\u20135pm, when they're off the tools but still at work.\n3. **Association meetings and events** (co-founder). Aim for one a week. Bring a case study, not a pitch.\n4. **GC angle.** Ask your partners' GCs whether they would *recommend* Crewcheck to their subs. One GC recommendation can bring in 10 subs. Try this with 2 GCs as an experiment.\n5. **Cold email/LinkedIn.** Use it only to support calls. Short, plain text, and lead with the time number (Appendix C).\n\n### Paid pilot structure (no more free pilots)\n- **30 days, $250 flat**, credited toward the first month if they convert.\n- Success criteria agreed **in writing up front**, e.g. \"all field workers' certs loaded, Monday cert check under 1 hour, one GC audit passed from Crewcheck.\"\n- Day-1 kickoff, a day-14 check-in, and a day-28 conversion call already on the calendar when the pilot is signed.\n- Your job is to get them to value within 7 days. A pilot that hasn't loaded their worker data by day 7 will almost certainly fail.\n\n---\n\n## 6. Phase 3 (Weeks 9\u201313, Dec 7 \u2013 Jan 6): Close, systematize, and set up Q1\n\nThe holidays make December a poor month for new meetings but a good month for closing and planning.\n\n- [ ] **Close pilots** started in November. Offer annual prepay so buyers can use remaining 2026 budget.\n- [ ] **Book January demos now.** Contractors plan in January, so \"let's talk the week of Jan 11\" is an easy yes in mid-December.\n- [ ] **Write the playbook (v1)** using real data: ICP, best-performing channels, call script, objections and answers, conversion rate at each stage, average days to close, and pilot-to-paid rate.\n- [ ] Run a **win/loss review** on every closed deal and every lost deal.\n- [ ] Prepare the investor update: customers, MRR, pipeline, funnel rates, and what you learned.\n- [ ] Decide on the Q1 plan, including whether a founding seller or SDR would be worth hiring once you reach 10 customers.\n\n---\n\n## 7. Weekly operating rhythm\n\n| When | What | Time |\n|---|---|---|\n| Mon 7\u20139am | Calling block (both founders) | 2h |\n| Mon\u2013Thu | Demos and discovery calls in the afternoons | ~5 calls |\n| Tue & Thu 7\u20139am | Calling block | 2h each |\n| Wed evening | Association event / site visit (co-founder) | as available |\n| Fri 3pm | **Pipeline review:** update the CRM, compute funnel rates, pick the 3 most important deals for next week | 1h |\n\n**Time budget:** plan for about **50% of your time on selling** in Phase 1\u20132 and 50% on product (mostly onboarding fixes that pilots surface). Your co-founder should spend about 80% on selling. Protect the calling blocks; they are the first thing that gets dropped.\n\n---\n\n## 8. What to track (and show investors)\n\nTrack weekly in the CRM:\n- Companies contacted \u2192 conversations \u2192 demos \u2192 pilots \u2192 paid (count and % at each step)\n- Days from first contact to paid\n- MRR and number of paying customers\n- Pilot-to-paid conversion rate\n- Lead source of every paying customer\n- Top 3 objections heard that week\n\n\"Repeatable\" here means you can say, with your own data: *\"If we contact X companies a week from source Y, we get Z customers a month at a cost of $W.\"* That's what investors are asking for.\n\n---\n\n## 9. Risks and what to do\n\n| Risk | Early sign | Response |\n|---|---|---|\n| Design partners won't pay | Hedging in Week 2 (\"let's revisit next quarter\") | Ask what would need to be true for them to pay. If it's a missing feature, build it fast. If it's \"not enough value,\" the time study will show it. Don't keep giving it away free. |\n| Too few conversations | Under 8 conversations a week by Week 5 | Move effort from email to phone and in-person, and push harder for partner intros |\n| Demos don't convert to pilots | Under 25% demo \u2192 pilot | Focus the demo on their Monday instead of a feature tour. Bring the case study numbers. |\n| Pilots stall | Data not loaded by day 7 | Offer white-glove setup: you import their spreadsheet yourself |\n| Holiday slowdown | Few replies after Dec 14 | Expected. Use the time for the playbook and to book January meetings. |\n| \"We already use [prequal platform / spreadsheets]\" | Common objection | Position Crewcheck as the sub's own system that *feeds* GC prequal, not a competitor to it |\n| Founder burnout | Calling blocks skipped | Keep to the Friday review, and make the 2h calling blocks non-negotiable |\n\n---\n\n## 10. The first 5 things to do (this week)\n\n1. Fix pricing and write the founding-customer offer (Section 2).\n2. Book time with all 4 design partners for next week, either a site visit or a call.\n3. Have your co-founder collect the before/after paperwork hours from each partner.\n4. Set up `pipeline-tracker.csv` (or a free CRM) and log the 4 partners.\n5. Put the Mon/Tue/Thu 7\u20139am calling blocks on both calendars through January.\n\n---\n\n## Appendix A: Design-partner conversion conversation\n\n> \"You've been using Crewcheck for [X months]. Before we talk money, I want to check it's actually working. How long did Monday paperwork take before, and how long does it take now?\"\n>\n> *(Let them answer. Write down the numbers.)*\n>\n> \"That's [N] hours a week, which at your office/safety coordinator's rate is roughly $[Y]/month. We're starting to charge on December 1. Because you helped build this, you get founding pricing: $4 per worker instead of $6, locked for 12 months. For your [W] workers that's $[Z]/month. Can we get you signed up?\"\n>\n> **If they hesitate:** \"What would need to be true for this to be an easy yes?\" Then be quiet and listen.\n>\n> **Then ask:** \"Who else do you know, maybe another trade on the same jobs, who has the same Monday headache? Would you introduce us?\"\n\n## Appendix B: Discovery call (20\u201330 min, co-founder leads)\n\n1. **Their situation:** \"How many guys in the field? Which GCs are you working for right now?\"\n2. **The pain:** \"Walk me through how you track certs and COIs today. Who does it, and how long does it take?\"\n3. **What it costs them:** \"Have you ever been held up at a gate or failed an audit because of a missing cert? What did that cost you?\"\n4. **Who decides:** \"If this made sense, who else would weigh in?\"\n5. **Next step:** \"Want to try it on one crew for 30 days? It's $250, credited to your first month, and we'll load your data for you.\"\n\n## Appendix C: Cold email (send after a call attempt, not instead of one)\n\n> **Subject:** Monday cert paperwork at [Company]\n>\n> Hi [Name], I tried you by phone earlier. [Co-founder] here; I ran sites for [N] years before co-founding Crewcheck.\n>\n> An electrical sub we work with cut their Monday cert/COI/toolbox-talk paperwork from 6 hours to under 1. Nobody's getting stopped at the gate for an expired card anymore.\n>\n> Worth 15 minutes to see if it'd do the same for you?\n>\n> [Name] \u00b7 [phone]\n"}, {"name": "pipeline-tracker.csv", "text": "company,trade,field_workers,metro,contact_name,contact_role,phone,email,source,stage,first_contact_date,last_touch_date,next_step,next_step_date,owner,pilot_start,pilot_success_criteria,price_per_worker,mrr,closed_date,lost_reason,notes\nDesign Partner 1,Electrical,,,,,,,design_partner,design_partner,,,Conversion conversation,,,,,4,,,,\nDesign Partner 2,Electrical,,,,,,,design_partner,design_partner,,,Conversion conversation,,,,,4,,,,\nDesign Partner 3,HVAC,,,,,,,design_partner,design_partner,,,Conversion conversation,,,,,4,,,,\nDesign Partner 4,HVAC,,,,,,,design_partner,design_partner,,,Conversion conversation,,,,,4,,,,\n"}]}, "opus-2": {"cost_usd": 0.36955, "seconds": 163, "output_tokens": 13259, "files": [{"name": "90-day-sales-plan.md", "text": "# Crewcheck: 90-Day Founder Sales Plan\n\n**Window:** Thu 8 Oct 2026 \u2192 Wed 6 Jan 2027 (Day 1 \u2192 Day 90)\n**Owner:** You (technical founder), with your co-founder as the field-credibility closer\n**Companion files:** `sales-playbook.md` (scripts and templates), `pipeline.csv` (tracker)\n\n---\n\n## 1. The goal, and the math behind it\n\nInvestors want **10 paying customers and a repeatable motion by month 6.** This 90-day plan gets you about two-thirds of the way there and proves the motion works, so months 4\u20136 are about running it again, not working it out.\n\n**Day-90 targets**\n\n| Metric | Target | Why |\n|---|---|---|\n| Paying customers | **6\u20137** (3+ converted design partners + 3\u20134 new) | Leaves months 4\u20136 to close the last 3\u20134 from pipeline that already exists |\n| Qualified pipeline (demo done, not yet closed) | **12+ accounts** | Enough to close 10 by month 6 at a ~30% close rate |\n| Customers from outside your network | **\u22652** | Investors will discount customers who were friends or design partners. Strangers paying is the proof |\n| Documented conversion rates at each stage | Yes | This is what \"repeatable\" means in practice (see \u00a77) |\n| MRR | ~USD 2,500\u20133,500 | ~6.5 customers \u00d7 ~70 workers \u00d7 $6 |\n\n**Revenue reality check:** a typical customer here is 30\u2013120 workers, so **$180\u2013$720/month ($2\u20139k/year).** Ten customers is roughly $50k ARR. That's fine for a proof point, but it means **you can't afford long sales cycles.** Plan for one discovery call, one demo, and a decision within ~3 weeks. If deals need five meetings and a procurement process, you're selling to the wrong people.\n\n**Funnel assumptions** (estimates, so replace them with your real numbers by Day 45):\n\n```\n~400 targeted outreach touches  \u2192  ~60 conversations (15%)\n\u2192  ~25 demos (40%)  \u2192  ~7 new closes (25\u201330%)\n```\nWarm intros from design partners, GCs, and associations convert several times better than cold outreach. Every referral is worth about 10 cold emails, so chase them hard.\n\n---\n\n## 2. Who you sell to (ICP v1)\n\nStart narrow. Widen only once the narrow version closes.\n\n- **Trade:** Electrical and HVAC/mechanical subcontractors (your design partners are already here, so the references transfer)\n- **Size:** 25\u2013150 field workers\n- **Geography:** 1\u20132 metro areas/states, ideally wherever your design partners work. Local references beat national ones in construction\n- **Trigger signals:** works for GCs with heavy prequalification or compliance demands (commercial, healthcare, government, data centers); recently lost or delayed a job over paperwork; growing headcount; has a safety manager or an office manager drowning in COIs\n- **Buyer:** Owner/President at the smaller end; Operations Manager or Safety Director at the larger end\n- **User:** Safety manager, office admin, foremen\n- **Disqualify:** fewer than ~20 workers (too little pain and revenue), already on Procore plus a dedicated compliance tool and happy with it, residential-only with no GC compliance pressure\n\n**The core pain to sell against:** \"the Monday paperwork,\" meaning chasing toolbox-talk sign-offs, expired certs, and COIs before a GC or inspector catches them. Sell **hours saved and jobs not lost**, not \"compliance software.\"\n\n---\n\n## 3. Pricing (decide this in week 1, then stop thinking about it)\n\n- **List price:** $6/active field worker/month. Keep it, because there's no evidence yet that it's wrong.\n- **Monthly minimum:** $150. This protects you from 12-person shops eating support time.\n- **Annual prepay option:** 2 months free (= $60/worker/year). This improves cash, and annual commitments read better to investors.\n- **Design partners / founding customers:** **don't discount the price.** Offer a **24-month price lock** plus a named \"founding customer\" status (roadmap input, direct line to founders) instead. A discount muddies your price validation, and the price lock costs you nothing today.\n- **\"Active worker\"** = anyone with a record in the system that month. Seasonal crews shrink the bill automatically, which also answers an objection.\n\nWrite all of this on a one-page order form (see playbook). Don't do custom deals before customer #10.\n\n---\n\n## 4. Roles: who does what\n\nYou're new to selling. Your co-founder has the one thing you can't fake: **they've been the site manager drowning in this paperwork.** Use that.\n\n| | You (technical founder) | Co-founder (ex-site manager) |\n|---|---|---|\n| Owns | Pipeline, outreach, follow-up, pricing, CRM, metrics | Credibility, discovery with field/ops people, demos to safety/ops buyers |\n| Calls | Run the call, take notes, handle pricing and technical questions | Tell \"when I was on site\u2026\" stories, ask the field questions |\n| Time | **50% of your week on sales** (block it in your calendar, non-negotiable) | 30\u201340% on sales |\n| Learning | Joint calls for the first ~10 calls, then solo | \u2014 |\n\n**Rule:** both founders attend the first 10 discovery calls. After that, split up to double capacity.\n\n---\n\n## 5. The 90 days, phase by phase\n\n### Phase 1: Convert and learn (Days 1\u201321, 8 Oct \u2192 28 Oct)\n\n**Goal: turn \"they'd pay if\u2026\" into signed paying customers, and learn exactly why they pay.**\n\nThis is the most important phase. If your design partners won't pay, that's critical information, and it's far cheaper to learn it now than after 300 cold emails.\n\n**Week 1 (8\u201314 Oct)**\n- [ ] Book a 45-minute **value review** with each of the 4 design partners (script in playbook \u00a71). Ask for the decision-maker to attend\n- [ ] Before each meeting, pull usage data: active users, sign-offs collected, certs tracked, expirations caught\n- [ ] Set up `pipeline.csv` (or a free HubSpot/Attio CRM if you prefer). Log every touch from Day 1\n- [ ] Finalize pricing and the one-page order form (\u00a73). Set up Stripe invoicing\n- [ ] Write ICP v1 (\u00a72) on one page\n\n**Week 2 (15\u201321 Oct)**\n- [ ] Run the 4 value reviews. **Quantify** for each partner: hours/week on Monday paperwork before vs. now, any expired cert or COI caught, any GC rejection avoided\n- [ ] At the end of each review, make the ask: \"Free period ends [date]. Founding-customer pricing is $6/worker with a 24-month lock. Can we start on [1 Nov]?\"\n- [ ] Ask every partner for **2 introductions** (other subs, their GC contacts, their insurance broker) whether they convert or not\n- [ ] Interview 5\u20138 non-customers in the ICP (cold or through your co-founder's network). This is discovery, not pitching (playbook \u00a72)\n\n**Week 3 (22\u201328 Oct)**\n- [ ] Close the design partners. Send the order form the same day as the call and follow up within 48 hours\n- [ ] Write a **one-page case study** for your best partner with numbers (\"Saved X hours every Monday; caught Y expired certs before a GC audit\"). Get approval to use their name and logo, or at least \"40-person electrical sub in [city]\"\n- [ ] Build your **first target list: 150 accounts** (sources in playbook \u00a76)\n\n**Phase 1 checkpoint (Day 21)**\n\n| Result | What it means | Action |\n|---|---|---|\n| 3\u20134 partners paying | Value is real | Go full speed into Phase 2 |\n| 2 paying | Partial signal | Proceed, but dig into exactly why the other 2 said no |\n| 0\u20131 paying | **Stop.** The product doesn't save enough yet, or you're talking to the wrong buyer | Spend 2 weeks fixing the gap the partners named before any outbound. Tell investors early. They respect it more than a surprise in month 6 |\n\n### Phase 2: Build the engine (Days 22\u201356, 29 Oct \u2192 2 Dec)\n\n**Goal: book 15+ demos with strangers and close your first 2 non-network customers.**\n\n**Weekly activity quota (per week, from 2 Nov):**\n- 60\u201380 targeted outreach touches (email + LinkedIn + phone, 3-touch sequences from playbook \u00a74)\n- 2+ warm-intro requests (design partners, customers, GCs, brokers)\n- 5\u20138 discovery calls \u2192 3\u20135 demos\n- 1 hour reviewing the week's numbers and changing **one** thing\n\n**Channels, in priority order:**\n1. **Referrals from design partners and new customers.** Ask every single one.\n2. **The GC angle.** Subs buy compliance tools because GCs demand compliance. Identify 3\u20135 GCs your partners work for and ask: \"Can we show your subs a faster way to get you compliant paperwork?\" One GC recommendation can unlock 10+ subs. This is your biggest potential lever, so test it in this phase.\n3. **Targeted cold outbound** to ICP subs in your 1\u20132 metros. Phone works better than you'd expect in construction. Owners pick up before 7:30am and after 4pm.\n4. **Trade associations:** local chapters of IEC, NECA (electrical), ACCA, SMACNA, MCAA (HVAC/mechanical). Attend 1\u20132 chapter meetings and ask to give a 10-minute safety-paperwork talk. Your co-founder presents.\n5. **Insurance brokers** who serve contractors. They're also drowning in COIs and see every sub's compliance problems. Test 3\u20135 conversations.\n\n**Milestones**\n- [ ] **By 6 Nov:** outreach sequences live; first 50 touches sent\n- [ ] **By 13 Nov:** 3\u20135 GC conversations done; at least 1 association event booked\n- [ ] **By 20 Nov:** first demo-to-close from a stranger in progress\n- [ ] **By 2 Dec:** 15+ demos completed; **5 paying customers**\n- [ ] Thanksgiving week (23\u201327 Nov): expect a dead week. Use it to update the playbook with real objections and answers\n\n### Phase 3: Close and systematize (Days 57\u201390, 3 Dec \u2192 6 Jan)\n\n**Goal: close the year strong, document the motion, and line up a big January.**\n\n**The December dynamic:** construction slows down and decision-makers take time off from about 18 Dec. But many subs plan budgets for next year in December, and **many insurance policies renew on 1 January**, which means a flood of new COIs to collect and track. That's a timely hook: *\"Get set up before January renewals hit.\"*\n\n- [ ] **3\u201318 Dec:** push every open demo to a decision. Offer a concrete reason to decide now: \"Go live by 1 Jan, so January COI renewals and the first toolbox talks of the year are in the system from day one\"\n- [ ] **By 18 Dec:** 6\u20137 paying customers\n- [ ] **18 Dec \u2013 2 Jan (slow period):**\n  - Write **Playbook v2**: ICP v2, the messaging that actually booked meetings, the top 10 objections with answers, real conversion rates per stage, average sales cycle length\n  - Write 2 more case studies\n  - Build the Jan\u2013Mar target list (300 accounts) and book January meetings before the holidays\n  - Check health on every paying customer: are they using it weekly? Churn kills the investor story faster than slow growth\n- [ ] **Days 85\u201390 (1\u20136 Jan):** investor update with the metrics in \u00a77. Restart outbound on 4 Jan\n\n---\n\n## 6. Your weekly rhythm (from Week 2 on)\n\n| Day | Morning (sales block) | Notes |\n|---|---|---|\n| Mon | Pipeline review (30 min) + outreach batch | Plan the week's calls |\n| Tue | Calls and demos | Owners are easiest to reach early |\n| Wed | Outreach batch + follow-ups | |\n| Thu | Calls and demos | |\n| Fri | Follow-ups, CRM hygiene, **weekly metrics review (1 hour, both founders)** | Change one thing per week, never five |\n\nAfternoons go to product. **Protect the sales blocks the way you'd protect an outage.** For technical founders, the most common failure is quietly drifting back to building.\n\n---\n\n## 7. What \"repeatable sales motion\" means (and how you'll show it)\n\nInvestors are asking whether a new salesperson could follow this and get similar results. By Day 90, you should be able to show:\n\n1. **ICP definition**, plus evidence that customers matching it close at a higher rate than those that don't\n2. **Lead source(s)** that produce meetings predictably (e.g. \"GC referrals: 1 GC intro \u2192 avg 4 sub meetings\")\n3. **Stage conversion rates:** outreach \u2192 meeting \u2192 demo \u2192 close\n4. **Sales cycle length** (target: \u226421 days, first call to signed)\n5. **Written playbook** (`sales-playbook.md`, v2 by January)\n6. **At least 2 customers who'd never heard of you before**, closed through the process\n7. **Retention signal:** paying customers actively using it (e.g. weekly sign-offs collected)\n\n**Weekly dashboard** (track in `pipeline.csv`, review every Friday):\n\n| Metric | Wk target |\n|---|---|\n| Outreach touches sent | 60\u201380 |\n| Conversations / meetings booked | 6\u201310 |\n| Demos held | 3\u20135 |\n| Proposals sent | 1\u20133 |\n| Closed-won (cumulative) | see milestones |\n| Warm intros requested / received | 2+ / 1+ |\n\n---\n\n## 8. Risks and how to handle them\n\n| Risk | Early warning | Response |\n|---|---|---|\n| Design partners won't pay | Hedging in the value review, \"after the new year\" | Find out what's missing. Offer to go live on a smaller scope (one crew) at the full per-worker price, not a discount |\n| You drift back into coding | Fewer than 40 touches/week two weeks in a row | Co-founder holds you accountable; Friday review is mandatory |\n| Deals stall at \"need to think about it\" | Cycle > 30 days | Always book the next step before ending a call. Use a mutual go-live date |\n| ACV too low for the effort | Lots of 20\u201330-worker shops | Enforce the $150 minimum; prioritize 50+ workers |\n| \"We already use Procore\" | Frequent objection | Position as complementary: field sign-offs + cert/COI tracking that Procore doesn't do well for subs (verify per customer) |\n| December kills momentum | Few decisions after 15 Dec | Front-load closes into early December; use the slow period for playbook and January prep |\n| Building custom features for each prospect | \"We'd buy if it did X\" | Log every request; only build what 3+ prospects ask for |\n\n---\n\n## 9. Day-1 checklist (do these today and tomorrow)\n\n1. Email all 4 design partners to book value reviews for next week (template in playbook \u00a71)\n2. Block 50% of your calendar for sales from now to 6 Jan\n3. Open `pipeline.csv` and enter the 4 design partners\n4. Agree with your co-founder on the price and the 24-month founding-customer lock\n5. Ask your co-founder for 20 names from their site-manager days: subs, GCs, safety managers. That's your first warm list\n\n---\n\n*Assumptions to confirm:* conversion rates are SMB B2B benchmarks, not your data, so replace them by Day 45. This plan assumes US geography (trade associations, Thanksgiving, Jan-1 renewals), and I haven't seen your current month of runway. If you're already 3+ months into the 18, compress Phase 2.\n"}, {"name": "pipeline-stages.md", "text": "# Pipeline stages (for `pipeline.csv`)\n\n| Stage | Definition | Exit criteria |\n|---|---|---|\n| target | On list, not contacted | First touch sent |\n| contacted | In outreach sequence | Replied or sequence finished |\n| meeting_booked | Discovery call on calendar | Call held |\n| discovery_done | Qualified: 25+ workers, named pain, reachable decision-maker | Demo booked |\n| demo_done | Demo held with the decision-maker present | Order form sent |\n| proposal_sent | Order form sent | Signed, or lost |\n| closed_won | Signed and first payment set up | \u2014 |\n| closed_lost | Said no / went dark after the breakup email | Record why in `objections` |\n| value_review_to_book | Design partners only, before the value review | Review held \u2192 proposal_sent |\n\n`type`: design_partner | network | referral | gc_referral | cold | association | broker. This is how you'll show which lead source works.\n\nEvery Friday, count the deals moving between stages that week. Those conversion rates are the \"repeatable motion\" evidence for investors.\n"}, {"name": "pipeline.csv", "text": "company,type,trade,est_workers,metro,contact_name,contact_role,source,stage,first_contact_date,last_touch_date,next_step,next_step_date,pain_in_their_words,objections,decision_maker,est_mrr_usd,close_date,notes\nDesign Partner 1,design_partner,Electrical,,,,,design_partner,value_review_to_book,,,Book value review,2026-10-14,,,,,,\nDesign Partner 2,design_partner,Electrical,,,,,design_partner,value_review_to_book,,,Book value review,2026-10-14,,,,,,\nDesign Partner 3,design_partner,HVAC,,,,,design_partner,value_review_to_book,,,Book value review,2026-10-14,,,,,,\nDesign Partner 4,design_partner,HVAC,,,,,design_partner,value_review_to_book,,,Book value review,2026-10-14,,,,,,\n"}, {"name": "sales-playbook.md", "text": "# Crewcheck Sales Playbook v1\n\nScripts and templates for the 90-day plan. Use them as written for the first ~10 calls, then adapt them to the words customers actually use. **Update this file every Friday.**\n\n---\n\n## \u00a71. Design-partner value review (convert free \u2192 paid)\n\n**Booking email**\n> Subject: 30 min to review how Crewcheck's working for [Company]\n>\n> Hi [Name], it's been [X weeks] since your crews started using Crewcheck. Before we plan next year, we'd like to sit down for 30\u201345 minutes, go over what's actually saved you time and what hasn't, and talk about what working together looks like from here.\n>\n> Could [decision-maker] join too? Does [Tue/Thu] work?\n\n**Agenda (45 min)**\n1. **Their numbers, not yours (15 min)**\n   - \"Before Crewcheck, what did Monday morning look like? Who did the paperwork, and for how long?\"\n   - \"What does it look like now?\" \u2192 get a number: hours/week saved\n   - \"Has anything been caught that would have been a problem? An expired cert, a missing COI, a toolbox talk without signatures?\"\n   - \"Any GC audits or prequal requests since you started? How did they go?\"\n2. **Show their usage data (5 min):** sign-offs collected, certs tracked, expirations flagged\n3. **What's missing (10 min):** \"What would have to be true for this to be a no-brainer?\" Write it down and don't promise anything on the spot\n4. **The ask (10 min):**\n   > \"Your free period ends on [date]. We're converting design partners to founding customers: $6 per active worker per month, locked for 24 months, plus direct access to us on the roadmap. For your [N] workers, that's about $[N\u00d76]/month, roughly [hours saved \u00d7 hourly cost] in time saved alone. Can we start on [1 Nov]?\"\n5. **Referral ask (always, even if they say no):**\n   > \"Who else do you know (another sub, a GC you work with, your insurance broker) who has this same Monday problem? Would you introduce us to two people?\"\n\n**If they hesitate:** \"What's the hesitation, the price, the timing, or whether it's worth it?\" Then stay quiet and let them answer.\n\n---\n\n## \u00a72. Discovery call (20\u201330 min, new prospect)\n\nAim for them to talk 70% of the time. You're qualifying, not pitching.\n\n**Opener**\n> \"Thanks for the time. [Co-founder] ran sites for [X] years and we built this because of the paperwork side. I'd like to understand how you handle safety and compliance paperwork today, and if it sounds like we can help, we'll show you. Fair?\"\n\n**Questions**\n1. How many field workers do you have, and on how many active jobs?\n2. Walk me through how toolbox talks get done and signed today. Paper? Photos? An app?\n3. How do you track cert expirations (OSHA 10/30, NFPA 70E, EPA 608, lifts, first aid\u2026)? Who owns that?\n4. How do you collect and track COIs, from your own insurers and your lower-tier subs?\n5. When a GC asks for compliance docs, how long does it take to pull them together?\n6. Has a paperwork gap ever cost you anything? A delayed start, a failed audit, a payment held, a fine?\n7. How many hours a week does someone spend on this? Who?\n8. What have you tried? (Procore, spreadsheets, another tool?)\n9. If you fixed this, who'd need to say yes? Is there a budget for it?\n10. What's happening in the next 3 months that makes this more urgent? (new GC, big job, audit, Jan renewals)\n\n**Qualified if:** 25+ workers, a named pain with a cost attached, and a decision-maker you can reach within ~2 weeks.\n**Close the call with a booked next step:** \"Let's do a 30-min demo using your actual crews. [Day/time]?\"\n\n---\n\n## \u00a73. Demo (30 min)\n\nShow **their** world. Before the call, set up a sandbox with their company name, 5\u201310 of their real job titles/certs, and one of their GCs.\n\n1. **Recap their pain in their words (2 min):** \"You said Mondays take [Name] about 4 hours, and you nearly got bounced from [GC] over a lapsed cert. Let's look at that.\"\n2. **The Monday morning, before and after (10 min):** foreman sends a toolbox talk \u2192 crew signs on phone \u2192 office sees it done. Co-founder narrates from the site perspective.\n3. **The expiration catch (5 min):** a cert expiring in 14 days \u2192 alert \u2192 fixed before the GC notices.\n4. **The GC request (5 min):** \"GC wants all compliance docs for the [job] crew\" \u2192 one export.\n5. **Price and next step (8 min):**\n   > \"For [N] workers it's $[N\u00d76]/month. We can have your crews live in a week. Want to start with [1 job / the whole company] on [date]?\"\n\n**Don't** tour every feature. Three moments that map to their pain beat twenty features.\n\n---\n\n## \u00a74. Outreach sequences\n\n**Personalize the first line every time** (job they're on, GC they work for, recent hire, trade). Keep emails under 90 words.\n\n**Email 1 (Day 0)**\n> Subject: Monday paperwork at [Company]\n>\n> Hi [Name], [personal line, e.g. \"Saw you're on the [GC] hospital job in [city]\"].\n>\n> My co-founder ran job sites for [X] years and spent too many Mondays chasing toolbox-talk signatures and expired certs. We built Crewcheck to fix that: crews sign off on their phones, and cert/COI expirations get flagged before a GC catches them.\n>\n> A [40]-person electrical sub using it now [saves ~X hrs every Monday]. Worth a 20-min call to see if it'd help you?\n>\n> [Your name], co-founder, Crewcheck\n\n**LinkedIn connect (Day 2):** \"Hi [Name], we work with electrical/HVAC subs on safety paperwork (toolbox talks, certs, COIs). Would be good to connect.\"\n\n**Phone (Day 4), 7:00\u20137:45am or 4\u20135pm**\n> \"Hi [Name], [your name] from Crewcheck. I sent you a note about toolbox-talk and cert paperwork. Quick question: who handles that at [Company] today?\" \u2192 Then ask discovery questions 2\u20133. If there's interest, book the call.\n>\n> Voicemail: \"[Name], [your name], Crewcheck. We help subs like [reference customer type] cut the Monday safety paperwork. I'll send a quick email. [Number].\"\n\n**Email 2 (Day 7), bump with value**\n> \"[Name], one thing we hear a lot: with policies renewing January 1, the COI chase gets bad in January. We can have you set up before then. Worth 20 min?\"\n\n**Email 3 (Day 14), breakup**\n> \"[Name], I'll assume the timing isn't right. If expired certs or toolbox-talk sign-offs ever become a headache, reply 'later' and I'll check back in Q1.\"\n\n### GC outreach (different angle)\n> \"Hi [Name], we help your subs get you complete compliance paperwork (signed toolbox talks, current certs, COIs) without your team chasing them. [Sub you both know] uses us. Would it be useful if we showed this to the subs on your [project]? No cost to you.\"\n\n### Warm intro request (send to customers/partners, already written for them to forward)\n> \"[Name], could you forward this to [Person]? 'I've been using Crewcheck for our toolbox talks and cert tracking, and it's cut our Monday paperwork a lot. The founders are ex-construction and worth 20 minutes: [your email].'\"\n\n---\n\n## \u00a75. Objection handling\n\n| Objection | Response |\n|---|---|\n| \"We do it on paper / spreadsheets, it works fine\" | \"How long does it take on Monday, and what happens when a GC asks for proof from 3 months ago?\" Let the gap show itself |\n| \"My guys won't use an app\" | Co-founder: \"Mine didn't want to either. It's one tap on the phone they already have, no login. [Partner] got [X]% sign-off in week one\" (use real numbers) |\n| \"We use Procore\" | \"Lots of our customers do. Procore is great for the GC side. We handle the crew-level stuff: toolbox sign-offs, worker cert expirations, COIs. Does Procore do that well for you today?\" (check honestly, case by case) |\n| \"Too expensive\" | \"Compared to what? [Name] spends ~[X] hrs/week on this at ~$[rate]/hr, which is $[Y]/month. Crewcheck is $[Z].\" If it's still too much: start with one crew at full price, never a discount |\n| \"Not now, after the new year\" | \"Makes sense. The reason people start in December is so January renewals and the first toolbox talks land in the system. Want to set up now and go live 4 Jan?\" Then book a specific date |\n| \"Need to talk to my partner/owner\" | \"Of course. What will they want to know? Can we do 15 minutes with them on [day]?\" |\n| \"You're a tiny startup, what if you disappear?\" | \"Fair. We're funded for [18 months+], your data exports any time, and you get a 24-month price lock as a founding customer\" |\n\n---\n\n## \u00a76. Building target lists\n\nAim for 150 accounts in Phase 1 and 300 by January. For each one, capture: company, trade, est. workers, metro, decision-maker name, GC relationships, source.\n\n**Sources**\n- Your co-founder's network: subs, GCs, and safety managers from their site days (start here)\n- Design partners' GCs: find the other electrical/HVAC subs working those same projects\n- State contractor license databases (filter by electrical/mechanical license, metro)\n- Local chapter member directories: IEC, NECA, ACCA, SMACNA, MCAA\n- Public bid results and plan-holder lists on commercial projects in your metro (shows who's active and which GCs they work with)\n- LinkedIn: \"Safety Manager\" / \"Operations Manager\" at electrical/mechanical contractors, 11\u2013200 employees\n- Job posts for \"safety coordinator\" or \"office manager\" at subs (a sign the paperwork burden is growing)\n\n---\n\n## \u00a77. One-page order form (contents)\n\n- Customer name, billing contact, start date\n- Price: $6 per active field worker per month; monthly minimum $150\n- Option: annual prepay at $60/worker/year (2 months free)\n- Founding customer (first 10 customers): price locked 24 months\n- Term: month-to-month (or annual if prepaid); cancel with 30 days' notice\n- Data: customer owns their data; export any time\n- Onboarding: founders set up your crews and certs within 7 days\n- Signature + Stripe payment link\n\nKeep it to one page. Month-to-month lowers the risk of saying yes, which shortens the cycle you need to keep short.\n\n---\n\n## \u00a78. After each call (5 min, every time)\n\nLog in `pipeline.csv`: stage, next step + date, pain in their words, objections, who decides. **Every open deal must have a next step with a date.** If it doesn't, it isn't really pipeline.\n"}]}}, "verdicts": [{"pair": "1", "judge": "gpt", "order": "haiku shown first", "winner": "opus", "why": "B gives usable outreach, discovery scripts and conversion deadlines. Its unsupported customer claims and premature pricing certainty weaken it. A lacks selling scripts, assumes all four partners convert, and miscalculates its funnel, undermining activity targets and the month-six path.", "haiku_scores": {"c1": 1, "c2": 0, "c3": 2, "c4": 0, "c5": 0, "c6": 1}, "opus_scores": {"c1": 2, "c2": 2, "c3": 1, "c4": 1, "c5": 0, "c6": 1}}, {"pair": "1", "judge": "gpt", "order": "opus shown first", "winner": "opus", "why": "A provides stronger execution, partner conversion, and selling instructions, but prematurely fixes pricing and invents outreach claims. B lacks usable outreach and discovery scripts; its funnel yields 3.6 new customers, not six, and assumes all four partners convert.", "haiku_scores": {"c1": 1, "c2": 1, "c3": 1, "c4": 1, "c5": 0, "c6": 1}, "opus_scores": {"c1": 2, "c2": 2, "c3": 1, "c4": 1, "c5": 0, "c6": 1}}, {"pair": "1", "judge": "deepseek", "order": "haiku shown first", "winner": "tie", "why": "Both provide excellent, executable plans with clear phases, roles, and measurable milestones. They correctly use design partners and build a path to 10 customers.", "haiku_scores": {"c1": 2, "c2": 2, "c3": 2, "c4": 2, "c5": 2, "c6": 2}, "opus_scores": {"c1": 2, "c2": 2, "c3": 2, "c4": 2, "c5": 2, "c6": 2}}, {"pair": "1", "judge": "deepseek", "order": "opus shown first", "winner": "tie", "why": "Both provide excellent, executable plans meeting all criteria. Differences are minor and stylistic.", "haiku_scores": {"c1": 2, "c2": 2, "c3": 2, "c4": 2, "c5": 2, "c6": 2}, "opus_scores": {"c1": 2, "c2": 2, "c3": 2, "c4": 2, "c5": 2, "c6": 2}}, {"pair": "2", "judge": "gpt", "order": "haiku shown first", "winner": "opus", "why": "B provides clearer ownership, conversion decisions, usable scripts, and explicit funnel assumptions. However, it invents mobile signing, alerts, exports, no-login behavior, and customer evidence, while prematurely fixing pricing. A preserves product facts better but provides weaker selling instructions and acquisition rationale.", "haiku_scores": {"c1": 1, "c2": 1, "c3": 2, "c4": 1, "c5": 1, "c6": 1}, "opus_scores": {"c1": 2, "c2": 2, "c3": 0, "c4": 2, "c5": 0, "c6": 2}}, {"pair": "2", "judge": "gpt", "order": "opus shown first", "winner": "tie", "why": "A gives stronger ownership, scripts and closing steps but invents alerts, exports and login-free signing. B preserves product scope but lacks outreach scripts and a quantified acquisition rationale. Neither fully validates pricing or observes partner workflows.", "haiku_scores": {"c1": 1, "c2": 1, "c3": 2, "c4": 1, "c5": 1, "c6": 1}, "opus_scores": {"c1": 2, "c2": 1, "c3": 0, "c4": 2, "c5": 1, "c6": 1}}, {"pair": "2", "judge": "deepseek", "order": "haiku shown first", "winner": "tie", "why": "Both provide excellent, executable plans meeting all rubric criteria with no factual errors.", "haiku_scores": {"c1": 2, "c2": 2, "c3": 2, "c4": 2, "c5": 2, "c6": 2}, "opus_scores": {"c1": 2, "c2": 2, "c3": 2, "c4": 2, "c5": 2, "c6": 2}}, {"pair": "2", "judge": "deepseek", "order": "opus shown first", "winner": "opus", "why": "A is more executable, with sequenced phases, clear roles, and measurable weekly targets. B is less structured and has factual errors.", "haiku_scores": {"c1": 1, "c2": 1, "c3": 2, "c4": 1, "c5": 1, "c6": 1}, "opus_scores": {"c1": 2, "c2": 2, "c3": 2, "c4": 2, "c5": 2, "c6": 2}}]}, {"id": "cold-1", "family": "cold-email", "prompt": "Write a cold email to the VP of Operations at a 400-person regional grocery chain about our product. Product info in product.md.", "input": "product.md", "input_text": "Product: Shelfsense, cameras + software that detect empty shelves and misplaced price tags in grocery stores and send a task to staff.\nResults at 2 pilot chains: on-shelf availability up from 91% to 96%; one chain estimated EUR 1.2M yearly recovered sales across 18 stores.\nInstall: 1 night per store, no shelf changes. Pricing EUR 900 per store per month.\nThe target: VP Ops at a 40-store regional chain. We know nothing personal about him. Their stores often get reviews mentioning empty shelves.\n", "rubric": [{"id": "c1", "criterion": "Addresses the VP of Operations\u2019 relevant business problem at this 40-store regional grocery chain.", "full_marks_when": "Connects empty-shelf reviews to store execution, product availability, or lost sales, and explains why Shelfsense is relevant to someone overseeing operations across 40 stores.", "zero_when": "Uses a generic pitch with no connection to grocery operations or empty shelves, or confuses the 400 employees with the 40 stores."}, {"id": "c2", "criterion": "Makes the product and its practical use immediately understandable.", "full_marks_when": "Explains plainly that Shelfsense uses cameras and software to detect empty shelves and misplaced price tags and sends tasks to staff, making the connection between detection and action clear.", "zero_when": "Leaves the recipient unable to understand what the product does, or invents capabilities such as automatic restocking, price correction, or replacing staff."}, {"id": "c3", "criterion": "Uses pilot evidence accurately and preserves its limitations.", "full_marks_when": "Accurately cites a relevant result: availability rose from 91% to 96% at two pilot chains, a 5-percentage-point increase; one chain estimated EUR 1.2M in yearly recovered sales across 18 stores. Any figures used retain their proper scope and attribution, without promising the same result for the prospect.", "zero_when": "Misstates the figures, calls the improvement a 5% relative increase, presents the sales estimate as a verified or guaranteed result, or extrapolates it to the prospect as an expected outcome."}, {"id": "c4", "criterion": "Handles rollout and cost without misleading an operations buyer.", "full_marks_when": "If installation or pricing is mentioned, accurately states one night per store with no shelf changes and EUR 900 per store per month. Any 40-store cost calculation is EUR 36,000 per month or EUR 432,000 per year. Avoids unsupported claims about integration, staffing requirements, rollout speed, or return on investment.", "zero_when": "Gives incorrect installation or pricing details, implies all 40 stores can be installed in one night, makes an arithmetic error, or promises an unsupported payback or profit figure."}, {"id": "c5", "criterion": "Personalizes the email only with information actually available.", "full_marks_when": "Uses the known empty-shelf review pattern naturally and respectfully, without inventing the recipient\u2019s name, personal interests, specific review quotations, prior contact, or detailed knowledge of internal operations.", "zero_when": "Fabricates personal research, relationships, review details, or operational facts, or uses accusatory language likely to alienate the recipient."}, {"id": "c6", "criterion": "Delivers a concise cold email that can be used immediately.", "full_marks_when": "Provides a relevant subject line and a short, readable email with a clear reason for reaching out, a credible benefit, and one easy-to-answer next step, such as a brief conversation or demonstration. Requires no substantive rewriting and avoids hype, jargon, and multiple competing asks.", "zero_when": "Provides advice instead of an email, omits a clear next step, buries the pitch in a long explanation, or requires substantial rewriting before sending."}], "runs": {"haiku-1": {"cost_usd": 0.00306, "seconds": 11, "output_tokens": 1865, "files": [{"name": "cold-email-vp-operations.md", "text": "**Subject:** Empty shelves in your stores\n\nHi [First name],\n\nYour store reviews often mention empty shelves, and every gap during a busy shift is a sale your customers walk away from.\n\nShelfsense uses cameras and software to detect empty shelves and misplaced price tags, then sends a task to the staff member who can fix it. At two pilot grocery chains, on-shelf availability rose from 91% to 96%. One of them estimated EUR 1.2M in yearly recovered sales across 18 stores.\n\nInstallation takes one night per store, with no changes to your shelving. Pricing is EUR 900 per store per month.\n\nWould you be open to a 20-minute call next week to see whether this fits your stores?\n\nBest,\n[Your name]\n[Company]\n"}]}, "haiku-2": {"cost_usd": 0.00319, "seconds": 10, "output_tokens": 2076, "files": [{"name": "cold_email_vp_ops.md", "text": "**Subject:** Empty shelves showing up in your reviews?\n\nHi [First name],\n\nEmpty shelves come up often in reviews of regional grocery stores like yours. Shelfsense uses cameras and software to spot empty shelves and misplaced price tags as they happen, then sends a task to the staff member who can fix it.\n\nIn our pilots at two grocery chains, on-shelf availability rose from 91% to 96%. One of those chains estimated about EUR 1.2M in recovered sales per year across 18 stores.\n\nInstall takes one night per store, with no changes to your shelves. Pricing is EUR 900 per store per month.\n\nWould a 15-minute call next week make sense to see whether this fits your 40 stores?\n\nBest,\n[Your name]\n[Title, Company]\n"}]}, "opus-1": {"cost_usd": 0.08213, "seconds": 43, "output_tokens": 1874, "files": [{"name": "cold-email-vp-ops.md", "text": "# Cold Email: VP of Operations, Regional Grocery Chain\n\n**To:** [First name], VP of Operations, [Chain name]\n**From:** [Your name], Shelfsense\n\n---\n\n**Subject line options (pick one):**\n1. Empty shelves at [Chain name]\n2. 91% \u2192 96% on-shelf availability\n3. The shelf gaps your customers mention in reviews\n\n---\n\nHi [First name],\n\nSeveral recent reviews of [Chain name] stores mention empty shelves. Usually the product is already in the back room and nobody has noticed the gap on the shelf yet.\n\nShelfsense uses cameras to spot empty shelves and wrong price tags, then sends a task to the right staff member's device so it gets fixed.\n\nAt two grocery chains that piloted it:\n\n- On-shelf availability went from 91% to 96%.\n- One chain estimates EUR 1.2M a year in recovered sales across 18 stores, or about EUR 67K per store.\n\nInstallation takes one night per store, and nothing on the shelves has to change. It costs EUR 900 per store per month.\n\nWould you be open to a 20-minute call to see whether a one- or two-store pilot makes sense? I can also send a short write-up of the pilot results first if you prefer.\n\nBest,\n[Your name]\n[Title], Shelfsense\n[Phone] \u00b7 [Email]\n\n---\n\n## Notes before sending\n\n- **Check the reviews first.** Before you send, find one or two actual reviews (Google, Yelp, etc.) that mention empty shelves. You can name a store (\"reviews of your [Town] store\") or quote a few words, but only if the review is real. Don't send the opening line as written without checking.\n- **Per-store math:** EUR 1.2M \u00f7 18 stores \u2248 EUR 67K recovered sales per store per year, compared with EUR 10.8K per store per year for Shelfsense. This compares sales to cost, not profit to cost, which is why the email gives the figures and doesn't claim an ROI multiple. A finance-minded VP will notice if the two get mixed up.\n- **Chain size:** The brief says ~400 employees and product.md says 40 stores. If both are right, the stores are small (~10 staff each), and the \"one night, no shelf changes\" install and the task alerts for a thin crew are likely the strongest selling points. Confirm the store count before sending, because a full rollout would cost about EUR 36K/month (EUR 432K/year) at 40 stores.\n- **Pilot names:** If the two pilot chains will let you name them, add that. A named grocery reference is much more convincing than an anonymous one.\n- **Follow-up (about 4 business days later):** A short reply in the same thread with something new, such as the price-tag accuracy angle (compliance and fewer checkout disputes) or the one-page pilot summary.\n"}]}, "opus-2": {"cost_usd": 0.0801, "seconds": 21, "output_tokens": 1785, "files": [{"name": "cold-email-vp-ops.md", "text": "# Cold email: VP of Operations, regional grocery chain\n\n**Subject line options** (pick one):\n1. Empty shelves in [Chain] reviews\n2. 91% \u2192 96% on-shelf availability\n3. Fixing gaps before customers notice them\n\n---\n\nHi [First name],\n\nI was reading customer reviews of [Chain] and noticed that \"empty shelves\" keeps coming up, e.g. [paste a short real quote + store location]. Usually the product is sitting in the back room and nobody has spotted the gap yet.\n\nShelfsense fixes that. Cameras scan the shelves, and when something runs out or a price tag is in the wrong place, the right staff member gets a task on the spot. Nobody has to walk the aisles checking.\n\nAt two grocery chains we piloted with:\n\n- On-shelf availability went from 91% to 96%.\n- One chain estimated EUR 1.2M a year in recovered sales across 18 stores. That's about EUR 67k per store, against our price of EUR 900 per store per month (EUR 10.8k a year).\n\nEach store is installed in one night and we don't change any shelving, so there's no disruption to trading.\n\nWould you be open to a 20-minute call to see whether a pilot in 2\u20133 of your stores makes sense? I can send over the pilot results first if that's easier.\n\nBest,\n[Your name]\n[Title], Shelfsense\n[Phone]\n\n---\n\n## Notes for the sender\n\n- **Fill in the review quote before sending.** A real quote from their own stores is the strongest hook here. If you can't find a specific one, change the first line to: \"Reading [Chain]'s customer reviews, empty shelves come up more than they should.\"\n- **Check the company size.** You described the chain as 400 people, but product.md says 40 stores. 400 employees would be only about 10 per store, which is small for grocery. The email doesn't name a store count, so it works either way, but it's worth confirming who you're actually targeting.\n- **About the EUR 67k per store figure:** I got it by dividing EUR 1.2M by 18 stores. It's the pilot chain's own estimate, so the email presents it as theirs. Leave out any projection for the prospect's whole chain unless you can support it.\n- **Follow-up (send 4\u20135 days later if there's no reply):** \"Hi [First name], one more note. The 91% \u2192 96% availability gain came from staff getting a task the moment a gap shows up, not from changing how they restock. Happy to share the pilot summary if that's useful.\"\n"}]}}, "verdicts": [{"pair": "1", "judge": "gpt", "order": "haiku shown first", "winner": "haiku", "why": "A is concise, accurate, and ready to send. B invents back-room inventory and review recency, adds an unsupported device detail, and requires subject selection and opening-line verification. Its arithmetic and pilot attribution are accurate.", "haiku_scores": {"c1": 2, "c2": 2, "c3": 2, "c4": 2, "c5": 2, "c6": 2}, "opus_scores": {"c1": 2, "c2": 1, "c3": 2, "c4": 2, "c5": 0, "c6": 1}}, {"pair": "1", "judge": "gpt", "order": "opus shown first", "winner": "haiku", "why": "B clearly explains detection and staff action, preserves pilot attribution, and offers one straightforward next step. A invents back-room inventory knowledge and device routing, omits software, and requires subject selection and opening-line verification before sending.", "haiku_scores": {"c1": 2, "c2": 2, "c3": 2, "c4": 2, "c5": 2, "c6": 2}, "opus_scores": {"c1": 2, "c2": 1, "c3": 2, "c4": 2, "c5": 0, "c6": 1}}, {"pair": "1", "judge": "deepseek", "order": "haiku shown first", "winner": "haiku", "why": "B misstates pilot evidence (c3) and fabricates personalization advice (c5), while A is a concise, ready-to-use email.", "haiku_scores": {"c1": 2, "c2": 2, "c3": 2, "c4": 2, "c5": 2, "c6": 2}, "opus_scores": {"c1": 2, "c2": 2, "c3": 1, "c4": 2, "c5": 1, "c6": 1}}, {"pair": "1", "judge": "deepseek", "order": "opus shown first", "winner": "tie", "why": "Both emails meet all criteria effectively, with no factual errors or significant differences in quality.", "haiku_scores": {"c1": 2, "c2": 2, "c3": 2, "c4": 2, "c5": 2, "c6": 2}, "opus_scores": {"c1": 2, "c2": 2, "c3": 2, "c4": 2, "c5": 2, "c6": 2}}, {"pair": "2", "judge": "gpt", "order": "haiku shown first", "winner": "haiku", "why": "A is accurate, concise, and directly addresses 40 stores. B invents back-room conditions and personal research, claims aisle checks become unnecessary and trading suffers no disruption, and requires additional research and editing. Its pilot figures and arithmetic are accurate.", "haiku_scores": {"c1": 2, "c2": 2, "c3": 2, "c4": 2, "c5": 2, "c6": 2}, "opus_scores": {"c1": 1, "c2": 1, "c3": 2, "c4": 1, "c5": 0, "c6": 1}}, {"pair": "2", "judge": "gpt", "order": "opus shown first", "winner": "haiku", "why": "B is accurate, concise, and ready to send, with a clear 40-store fit and single ask. A invents back-room conditions, overstates eliminated checks and trading disruption, requires additional research, and offers competing next steps.", "haiku_scores": {"c1": 2, "c2": 2, "c3": 2, "c4": 2, "c5": 2, "c6": 2}, "opus_scores": {"c1": 1, "c2": 1, "c3": 2, "c4": 1, "c5": 0, "c6": 1}}, {"pair": "2", "judge": "deepseek", "order": "haiku shown first", "winner": "haiku", "why": "A is a concise, ready-to-send email. B misstates pilot evidence (EUR 67k per store), adds unsupported per-store math, and fabricates a review quote.", "haiku_scores": {"c1": 2, "c2": 2, "c3": 2, "c4": 2, "c5": 2, "c6": 2}, "opus_scores": {"c1": 2, "c2": 2, "c3": 1, "c4": 1, "c5": 1, "c6": 1}}, {"pair": "2", "judge": "deepseek", "order": "opus shown first", "winner": "haiku", "why": "B is fully personalized with available info; A fabricates a review quote and requires editing.", "haiku_scores": {"c1": 2, "c2": 2, "c3": 2, "c4": 2, "c5": 2, "c6": 2}, "opus_scores": {"c1": 2, "c2": 2, "c3": 2, "c4": 2, "c5": 1, "c6": 2}}]}, {"id": "home-1", "family": "homepage", "prompt": "Write the copy for our homepage (hero and the sections below it). Product details in product.md.", "input": "product.md", "input_text": "Product: Stackwatch, monitors a company's SaaS subscriptions (pulls from card statements, SSO logins and contracts), shows unused seats and renewal dates, and drafts cancellation or downgrade requests.\nBuyers: IT or finance leads at 100-1,000 person companies. Users: the same people, plus department heads who approve renewals.\nProof: average customer finds USD 140k/year in unused seats in the first month; 37 customers; SOC 2 Type II.\nAlternatives: spreadsheets, Zylo/Vendr (enterprise, expensive), doing nothing.\nCurrent homepage hero: \"The intelligent SaaS management platform.\"\n", "rubric": [{"id": "c1", "criterion": "The hero immediately explains what Stackwatch does and why a buyer should care.", "full_marks_when": "Provides a publishable headline and supporting copy that clearly connect managing SaaS subscriptions with finding unused seats, avoiding unwanted renewals, or reducing wasted spend. Replaces the vague current hero with a concrete reason to keep reading.", "zero_when": "Repeats generic claims such as 'intelligent platform' without making the product and its practical value clear."}, {"id": "c2", "criterion": "The copy speaks to Stackwatch's actual buyers and renewal approvers.", "full_marks_when": "Addresses IT and finance leads at 100\u20131,000-person companies through relevant concerns such as subscription visibility, wasted seat spend, and renewal decisions; makes department heads' approval role understandable without assuming unsupported collaboration features.", "zero_when": "Targets individuals, consumers, or a materially different company segment, or offers generic benefits with no connection to these buyers' responsibilities."}, {"id": "c3", "criterion": "The sections below the hero explain the product's practical workflow.", "full_marks_when": "Clearly explains that Stackwatch pulls information from card statements, SSO logins, and contracts; surfaces unused seats and renewal dates; and drafts cancellation or downgrade requests. Connects these capabilities to useful buyer outcomes in accessible language.", "zero_when": "Leaves readers unable to understand how Stackwatch works, or omits most of its defining capabilities."}, {"id": "c4", "criterion": "All factual claims and numerical proof accurately reflect product.md.", "full_marks_when": "Accurately uses the average customer's finding of USD 140,000 per year in unused seats during the first month, the count of 37 customers, and SOC 2 Type II. Distinguishes identified waste from realized savings and drafted requests from automatic execution; invents no features, guarantees, testimonials, or security claims.", "zero_when": "Makes a material factual or numerical error, such as claiming USD 140,000 saved in one month, guaranteed savings, automatic cancellations, or unsupported customer counts or certifications."}, {"id": "c5", "criterion": "The copy gives buyers a credible reason to switch from their current approach.", "full_marks_when": "Makes the advantage over spreadsheets or doing nothing concrete through consolidated subscription information, visible renewal dates, unused-seat findings, and prepared requests. If comparing with Zylo or Vendr, avoids inventing Stackwatch pricing, comparative performance, or competitor deficiencies.", "zero_when": "Offers no meaningful reason to change, or relies on unsupported claims that Stackwatch is cheaper, faster, or superior to named competitors."}, {"id": "c6", "criterion": "The deliverable is usable homepage copy with a clear path to action.", "full_marks_when": "Supplies finished hero copy and an ordered set of below-hero sections with concise headings and body text, well-placed proof, and a clear next-step CTA. The page progresses coherently without repetitive filler, and does not invent a free trial, pricing offer, or purchasing process.", "zero_when": "Provides advice, an outline, or placeholder text instead of finished copy, omits the below-hero sections, or gives visitors no clear next step."}], "runs": {"haiku-1": {"cost_usd": 0.00455, "seconds": 20, "output_tokens": 3798, "files": [{"name": "homepage-copy.md", "text": "# Stackwatch homepage copy\n\n## Hero\n\n**Stop paying for SaaS seats nobody uses.**\n\nStackwatch finds every subscription your company pays for, shows which seats sit unused and when each contract renews, and drafts the cancellation or downgrade request for you.\n\n[Get started] [See how it works]\n\nConnects to your card statements, SSO logins and contracts. SOC 2 Type II audited.\n\n---\n\n## Every subscription, in one place\n\nStackwatch pulls from the sources you already have: card statements, SSO logins and signed contracts. You get one list of every SaaS tool your company is paying for, including the ones nobody approved.\n\n## See who is actually using it\n\nStackwatch compares the seats each contract pays for with the people logging in through SSO. You see how many seats each tool has, how many are in use, and where the gap is.\n\n## Never miss a renewal\n\nEvery contract's renewal date in one place. Stackwatch surfaces upcoming renewals ahead of time, so the decision happens before the contract renews, not after.\n\n## Requests drafted, ready to send\n\nFor each underused tool, Stackwatch drafts a cancellation or downgrade request. Department heads review and approve renewals, and you send the request when you're ready.\n\n## Built for companies with 100 to 1,000 people\n\nStackwatch is for IT and finance leads who own software spend, and for the department heads who approve renewals.\n\n## The numbers\n\n**$140k a year** \u2014 the average unused-seat savings a customer finds in their first month.\n\n**37 customers** running Stackwatch across their SaaS spend.\n\n## Why not a spreadsheet?\n\nSpreadsheets go stale the week after they're built, and nobody checks the SSO logs. Enterprise SaaS management platforms are expensive and sized for much larger organizations. Stackwatch is for the company that needs answers now, without a six-month rollout.\n\n## Security\n\nStackwatch is SOC 2 Type II audited.\n\n[Learn about our security]\n\n---\n\n## Closing call to action\n\n**Find out what you're paying for and not using.**\n\n[Get started]\n\n---\n\n## Before publishing\n\n- The $140k figure is an average. Keep \"on average\" in the copy or the claim reads as a guarantee.\n- \"37 customers\" may read as small. Drop it if it weakens the page.\n- SOC 2 Type II is an audit report, not a certification. The copy says \"audited\" for that reason.\n- The copy avoids naming Zylo or Vendr and avoids claims about how their pricing works. Add them only with legal review.\n- The copy doesn't promise a lead time for renewal alerts or say how the request is sent. Confirm both before launch.\n- The copy uses $ for the US audience. Switch currency if the page targets other markets.\n- The CTA labels are generic. Change them if the sales motion is a demo or a trial.\n"}]}, "haiku-2": {"cost_usd": 0.00428, "seconds": 17, "output_tokens": 3365, "files": [{"name": "homepage-copy.md", "text": "# Stackwatch homepage copy\n\n## Hero\n\n**Find the SaaS your company pays for and doesn't use.**\n\nStackwatch pulls subscriptions from card statements, SSO logins, and contracts. It shows unused seats and upcoming renewals, then drafts the cancellation or downgrade request for you.\n\n[Find my unused seats] [See how it works]\n\nCustomers find an average of $140k a year in unused seats in their first month.\n\n---\n\n## Your SaaS spend, in one list\n\nStackwatch builds your inventory from the places spend already shows up. Card statements catch every charge. SSO logins show who is actually signing in. Contracts show what you agreed to. Nobody has to remember to add a tool.\n\n---\n\n## What it does\n\n**Unused seats, flagged**\nSee which licenses nobody is using and how many seats you pay for beyond what your team needs.\n\n**Renewals before they renew**\nEvery renewal date in one view, so you decide before the contract decides for you.\n\n**Requests drafted for you**\nStackwatch writes the cancellation or downgrade request. Your department head reviews it and approves or rejects it.\n\n---\n\n## Built for the people who approve the spend\n\nIT and finance leads get the full picture across every team. Department heads get a clear request to approve, not a spreadsheet to decode.\n\n---\n\n## Why not a spreadsheet?\n\nA spreadsheet only knows what someone typed into it, and it goes stale the week after a new tool is signed. Stackwatch reads what you pay and who logs in.\n\nDoing nothing has a cost too. Every renewal you don't review is a decision made by default.\n\n---\n\n## Built for companies your size\n\nEnterprise SaaS management platforms are built for large procurement teams. Stackwatch is built for the 100 to 1,000 person company where IT or finance handles this on top of everything else.\n\n**SOC 2 Type II audited.**\n\n---\n\n## Final call to action\n\n**Stop paying for seats nobody uses.**\n\nConnect your card statements, SSO, and contracts. Stackwatch shows you what you can cut.\n\n[Find my unused seats]\n"}]}, "opus-1": {"cost_usd": 0.12387, "seconds": 38, "output_tokens": 3582, "files": [{"name": "homepage-copy.md", "text": "# Stackwatch homepage copy\n\n---\n\n## Hero\n\n**Headline:**\nStop paying for SaaS seats nobody uses.\n\n**Subhead:**\nStackwatch reads your card statements, SSO logins, and contracts to show every subscription you pay for, which seats sit idle, and when each one renews. Then it drafts the cancellation or downgrade request for you.\n\n**Primary CTA:** Book a demo\n**Secondary CTA:** See how it works\n\n**Proof line (under CTAs):**\nCustomers find an average of $140k a year in unused seats in their first month. SOC 2 Type II.\n\n---\n\n## Section 1: The problem\n\n**Heading:**\nYour SaaS spreadsheet stopped being accurate when you saved it.\n\n**Body:**\nSomeone signs up with a company card. A team grows, shrinks, then switches tools. A 50-seat contract auto-renews at 50 seats while 31 people log in. Nobody is careless. The information just lives in three places that don't talk to each other: the card statement, the login system, and the contract in someone's inbox.\n\nStackwatch pulls all three together so you can see what you're paying for and what's actually used.\n\n---\n\n## Section 2: How it works\n\n**Heading:**\nFrom \"what are we paying for?\" to a sent downgrade request\n\n**1. Connect your sources**\nLink your card statements, your SSO provider, and your contracts. Stackwatch builds the list of every subscription, including the ones nobody told IT about.\n\n**2. See what's unused, and what renews next**\nFor each tool, see seats paid for against seats actually logged into, plus the renewal date and notice period. Upcoming renewals show up while there's still time to act.\n\n**3. Send the request**\nPick a tool to cancel or right-size. Stackwatch drafts the request to the vendor with the seat count and contract terms filled in. You review it and send it.\n\n---\n\n## Section 3: What you get\n\n**Heading:**\nEverything you need before the next renewal\n\n**Every subscription in one list**\nCard spend, SSO apps, and contracts matched into one inventory, with owner, cost, and seat count for each tool.\n\n**Unused seats, counted**\nSeats compared against real SSO logins, so you know which licences to cut and which to keep.\n\n**A renewal calendar you can trust**\nRenewal dates and notice periods pulled from your contracts, so auto-renewals stop catching you off guard.\n\n**Cancellation and downgrade drafts**\nA ready-to-send request for each tool, written from your actual contract terms. No starting from a blank email.\n\n**Approvals from the people who own the budget**\nDepartment heads see the usage data for their tools and approve or decline renewals in one place, instead of in a reply-all thread.\n\n---\n\n## Section 4: Who it's for\n\n**Heading:**\nBuilt for the people who get asked \"why are we paying for this?\"\n\n**IT leads**\nFind the tools signed up for outside IT, and see who actually uses what.\n\n**Finance leads**\nSee where software spend goes, catch renewals before they hit, and back each cut with usage data.\n\n**Department heads**\nMake renewal decisions for your team's tools with the numbers in front of you.\n\nMade for companies of 100 to 1,000 people: too big to track SaaS in your head, too small to need an enterprise procurement platform.\n\n---\n\n## Section 5: Why not\u2026\n\n**Heading:**\nThe other ways to handle this\n\n**A spreadsheet**\nFree, and out of date the week after you build it. It can't see logins, and it won't remind you about a renewal.\n\n**Enterprise SaaS management platforms**\nPowerful, and priced and scoped for companies many times your size. Stackwatch does the part that saves you money without a six-month rollout.\n\n**Doing nothing**\nThe most expensive option. Our customers find an average of $140k a year in unused seats in the first month.\n\n---\n\n## Section 6: Security\n\n**Heading:**\nSOC 2 Type II audited\n\n**Body:**\nStackwatch handles your billing, login, and contract data, so we hold ourselves to an independently audited standard. Our SOC 2 Type II report is available on request.\n\n**CTA:** Request our security report\n\n---\n\n## Section 7: Final CTA\n\n**Heading:**\nFind out what you're paying for and not using.\n\n**Body:**\nOur 37 customers found an average of $140k a year in unused seats in their first month. Book a demo and we'll show you where to look in your own stack.\n\n**Primary CTA:** Book a demo\n\n---\n\n## Notes for review\n\n- **Hero change:** \"The intelligent SaaS management platform\" names a category and makes no promise. The new headline states the outcome a finance or IT lead is shopping for. The specifics of how it works go in the subhead.\n- **Things to confirm before publishing:**\n  - \"Including the ones nobody told IT about\": assumes card statements turn up tools bought outside IT. Remove this if matching doesn't reliably catch them.\n  - \"Notice period\": assumes Stackwatch pulls notice periods from contracts, not just renewal dates. If it doesn't, cut it in sections 2 and 3.\n  - \"You review it and send it\": I've assumed Stackwatch drafts but does not send. Adjust if it can send directly.\n  - \"Approve or decline renewals in one place\": assumes there's an approval workflow in the product. product.md only says department heads approve renewals.\n  - \"Report available on request\": check that this matches your process.\n  - \"Without a six-month rollout\": a comparison claim. Replace it with your real setup time if you have one, or cut it.\n  - The \"31 of 50 seats\" example in section 1 is illustrative, not customer data.\n- **The $140k figure** shows up in the hero, section 5, and the final CTA. Check whether it's a mean or a median. If a few large customers skew it, \"average\" may draw pushback. Also check whether it's measured against an annual figure.\n- **37 customers:** I only used this in the final CTA, where the specific number reads as honest. Put it higher on the page once you have customer logos or a named quote. There are no testimonials in this draft because none were supplied.\n- **Competitors:** Zylo and Vendr aren't named. If you'd rather compare directly, the \"Enterprise SaaS management platforms\" block is the place to do it.\n"}]}, "opus-2": {"cost_usd": 0.10957, "seconds": 33, "output_tokens": 3001, "files": [{"name": "homepage-copy.md", "text": "# Stackwatch homepage copy\n\n---\n\n## Hero\n\n**Headline:**\nFind the SaaS seats you're paying for and nobody uses.\n\n**Subhead:**\nStackwatch reads your card statements, SSO logins and contracts to show every subscription your company pays for, which seats sit idle, and when each one renews. When something should be cut, it drafts the cancellation or downgrade request for you.\n\n**Primary CTA:** See what you're overpaying\n**Secondary CTA:** Watch a 2-minute demo\n\n**Proof line (under CTAs):**\nCustomers find an average of $140k a year in unused seats in their first month. SOC 2 Type II.\n\n---\n\n## Section 2: The problem\n\n**Heading:**\nNobody owns the whole software bill.\n\n**Body:**\nFinance sees the charges. IT sees the logins. Department heads signed the contracts. Nobody sees all three at once, so seats pile up after people leave, tools overlap, and renewals go through on auto-pay before anyone asks whether the team still uses them.\n\nThe spreadsheet that was supposed to fix this was last updated two quarters ago.\n\n---\n\n## Section 3: How it works\n\n**Heading:**\nFrom scattered charges to a cancellation request, in three steps.\n\n**1. Connect your sources**\nLink your card statements, your SSO provider and your contracts. Stackwatch builds a list of every SaaS subscription your company pays for, including the ones nobody told IT about.\n\n**2. See what's actually used**\nFor each tool, Stackwatch compares seats paid for against people actually logging in. You get the unused seats, what they cost, and the renewal date, all in one view.\n\n**3. Act before the renewal**\nPick a tool to cut or shrink. Stackwatch drafts the cancellation or downgrade request. You review it, send it, and the saving shows up on the next bill.\n\n---\n\n## Section 4: What you get\n\n**Heading:**\nEverything you need to walk into a renewal knowing the numbers.\n\n**One list of every subscription**\nPulled from what you actually pay for, not what someone remembered to log.\n\n**Unused seats, priced**\nSeats with no recent logins, and what they cost you each year.\n\n**A renewal calendar**\nEvery renewal date, so you get a say before the auto-charge, not after.\n\n**Drafted cancellation and downgrade requests**\nReady to review and send. No more writing the same email to vendor support.\n\n**Approvals for department heads**\nThe people who own each tool see the usage data and sign off on renewals, so cuts don't become an argument.\n\n---\n\n## Section 5: Who it's for\n\n**Heading:**\nBuilt for the people who sign off on the software bill.\n\n**IT leads**\nKnow every tool in use, who has access, and which accounts should have been closed when someone left.\n\n**Finance leads**\nSee software spend as it really is, with renewals you can plan for instead of surprises on the card statement.\n\n**Department heads**\nApprove renewals with usage numbers in front of you, and keep the tools your team actually needs.\n\n---\n\n## Section 6: Compared with the alternatives\n\n**Heading:**\nBetween a spreadsheet and an enterprise contract.\n\n**Spreadsheets**\nFree, and out of date the week after you build them. They can't see who logs in.\n\n**Enterprise SaaS management platforms**\nBuilt and priced for companies many times your size, with a long rollout to match.\n\n**Doing nothing**\nMeans paying for every seat that went idle this year, again next year.\n\n**Stackwatch**\nMade for companies of 100 to 1,000 people. Connect your sources and see your unused seats within the first month.\n\n---\n\n## Section 7: Proof and security\n\n**Heading:**\nThe numbers from the first month.\n\n**Stat:** $140k\n**Label:** average yearly spend on unused seats found by customers in their first month\n\n**Stat:** SOC 2 Type II\n**Label:** audited controls for the financial and login data you connect\n\n[Customer quote here: name, title, company, with a specific number they found or cut]\n\n---\n\n## Final CTA\n\n**Heading:**\nYour next renewal is already on the calendar.\n\n**Body:**\nFind out what you're paying for before it charges again.\n\n**Primary CTA:** See what you're overpaying\n**Secondary CTA:** Talk to us\n\n---\n\n## Notes for review\n\n- **Hero change.** The old line, \"The intelligent SaaS management platform,\" says what category Stackwatch is in but not what it does for the buyer. The new headline leads with the outcome (finding money you're wasting) in the words an IT or finance lead would use.\n- **The $140k claim** is worded as \"find,\" not \"save,\" because the product data says customers *find* that amount. Cutting it still takes action. If you have a figure for realized savings, it's stronger and could replace or sit next to it.\n- **The 37 customers figure is not used.** For a mid-market buyer comparing against Zylo and Vendr, a small count can hurt more than it helps. Add it back once it's larger, or use named logos instead.\n- **Competitors aren't named** in the comparison section. They're described as \"enterprise SaaS management platforms.\" Naming Zylo and Vendr is an option if you're comfortable doing that on the homepage.\n- **Things to confirm with product:**\n  - Stackwatch drafts requests and does not send them. The copy says the user reviews and sends; change this if it can send directly.\n  - \"Approvals for department heads\" assumes there's an approval flow in the product. The brief says department heads approve renewals, but not whether that happens inside Stackwatch.\n  - \"Within the first month\" and \"Watch a 2-minute demo\" assume a typical onboarding time and a demo video. Change these if either is wrong.\n- **Placeholder:** Section 7 needs a real customer quote.\n"}]}}, "verdicts": [{"pair": "1", "judge": "gpt", "order": "haiku shown first", "winner": "haiku", "why": "A is tighter and invents fewer capabilities. Both make unsupported rollout comparisons. B additionally invents notice periods, inventory fields, contract-populated drafts, integrated approvals, and report availability. Review notes do not cure unsupported published copy.", "haiku_scores": {"c1": 2, "c2": 2, "c3": 2, "c4": 0, "c5": 0, "c6": 2}, "opus_scores": {"c1": 2, "c2": 1, "c3": 2, "c4": 0, "c5": 0, "c6": 1}}, {"pair": "1", "judge": "gpt", "order": "opus shown first", "winner": "haiku", "why": "B is more usable and avoids A\u2019s invented approval workflow. Both clearly explain the core product, but both invent a six-month rollout comparison. A additionally invents notice periods and security-report availability; B blurs identified waste with savings.", "haiku_scores": {"c1": 2, "c2": 2, "c3": 2, "c4": 0, "c5": 0, "c6": 2}, "opus_scores": {"c1": 2, "c2": 1, "c3": 2, "c4": 0, "c5": 0, "c6": 1}}, {"pair": "1", "judge": "deepseek", "order": "haiku shown first", "winner": "haiku", "why": "B makes a material factual error by implying an approval workflow feature not supported in product.md.", "haiku_scores": {"c1": 2, "c2": 2, "c3": 2, "c4": 2, "c5": 2, "c6": 2}, "opus_scores": {"c1": 2, "c2": 2, "c3": 2, "c4": 1, "c5": 2, "c6": 2}}, {"pair": "1", "judge": "deepseek", "order": "opus shown first", "winner": "opus", "why": "A provides a more complete, structured, and usable page with clearer workflow and CTAs.", "haiku_scores": {"c1": 2, "c2": 2, "c3": 2, "c4": 2, "c5": 2, "c6": 1}, "opus_scores": {"c1": 2, "c2": 2, "c3": 2, "c4": 2, "c5": 2, "c6": 2}}, {"pair": "2", "judge": "gpt", "order": "haiku shown first", "winner": "haiku", "why": "A is more publishable and explains the workflow concisely. Both invent approval features and omit the 37-customer proof. B additionally promises next-bill savings, invents competitor rollout disadvantages and a demo, and leaves a testimonial placeholder.", "haiku_scores": {"c1": 2, "c2": 1, "c3": 2, "c4": 0, "c5": 2, "c6": 2}, "opus_scores": {"c1": 2, "c2": 1, "c3": 2, "c4": 0, "c5": 0, "c6": 1}}, {"pair": "2", "judge": "gpt", "order": "opus shown first", "winner": "haiku", "why": "B provides tighter, finished copy and a concrete switching rationale. Both invent department-head request approval workflows and omit the 37-customer proof. A additionally invents competitor rollout disadvantages, next-bill savings, and a demo, while leaving a testimonial placeholder.", "haiku_scores": {"c1": 2, "c2": 1, "c3": 2, "c4": 0, "c5": 2, "c6": 2}, "opus_scores": {"c1": 2, "c2": 1, "c3": 2, "c4": 0, "c5": 0, "c6": 1}}, {"pair": "2", "judge": "deepseek", "order": "haiku shown first", "winner": "tie", "why": "Both are excellent, publishable, and meet all criteria. Differences are stylistic preferences.", "haiku_scores": {"c1": 2, "c2": 2, "c3": 2, "c4": 2, "c5": 2, "c6": 2}, "opus_scores": {"c1": 2, "c2": 2, "c3": 2, "c4": 2, "c5": 2, "c6": 2}}, {"pair": "2", "judge": "deepseek", "order": "opus shown first", "winner": "opus", "why": "A fully addresses all criteria with structured sections, clear workflow, and a strong competitor comparison. B is less complete and omits a clear next step.", "haiku_scores": {"c1": 2, "c2": 2, "c3": 2, "c4": 2, "c5": 1, "c6": 1}, "opus_scores": {"c1": 2, "c2": 2, "c3": 2, "c4": 2, "c5": 2, "c6": 2}}]}, {"id": "hook-1", "family": "video-hook", "prompt": "Write the script for a 30-second vertical video (TikTok/Reels) about the topic in topic.md. Focus on making people stay past the first seconds.", "input": "topic.md", "input_text": "Account: a personal-finance creator, 40k followers, audience 22-35.\nTopic: most people overpay on their phone plan; switching to an MVNO on the same network saves around USD 300-500 a year, same coverage.\nProof we can show on screen: my own bill before (USD 85/month) and after (USD 25/month), a coverage map showing it's the same towers.\nPrevious videos lose 70% of viewers in the first 3 seconds.\n", "rubric": [{"id": "c1", "criterion": "The first three seconds give viewers a concrete reason to keep watching.", "full_marks_when": "Opens immediately with a striking bill comparison, specific savings claim, or compelling question tied to overpaying for phone service. Creates a clear expectation of useful proof or an explanation without greetings or introductory filler.", "zero_when": "Spends the opening on greetings, creator background, generic money advice, or announcing the topic without a compelling reason to stay."}, {"id": "c2", "criterion": "All factual and numerical claims accurately reflect the input.", "full_marks_when": "Uses USD 85/month before and USD 25/month after correctly: USD 60/month saved, or USD 720/year. Distinguishes this personal example from the topic's approximate USD 300\u2013500 annual savings rather than presenting them as equivalent or guaranteed. If discussing retention, correctly treats a 70% loss as 30% remaining after three seconds.", "zero_when": "Misstates the bills, calculates savings incorrectly, claims the personal example saves USD 300\u2013500/year, or guarantees that every viewer will save USD 720/year."}, {"id": "c3", "criterion": "The script makes the supplied evidence visible and connects it to the claims.", "full_marks_when": "Specifies readable on-screen use of both actual bills and the coverage map. Connects the bills to the price reduction and the map to using the same network or towers, with clear emphasis on the relevant amounts and coverage.", "zero_when": "Relies entirely on unsupported narration, omits the supplied proof, or substitutes invented testimonials or evidence."}, {"id": "c4", "criterion": "The sequence sustains attention and delivers the opening promise.", "full_marks_when": "Moves quickly from the opening into evidence and a useful explanation of the cheaper alternative. Each beat advances the story, and any curiosity raised is resolved within the video rather than withheld until another post.", "zero_when": "Repeats the hook, pads the middle, delays all useful information until the end, or never explains how the savings are achieved."}, {"id": "c5", "criterion": "The deliverable is a usable script for a 30-second vertical video.", "full_marks_when": "Provides speakable dialogue plus clear timing and visual or text cues that plausibly fit 30 seconds. Keeps phone-screen text brief and readable and leaves enough time to understand the bill comparison and coverage proof.", "zero_when": "Provides only advice or an outline, substantially exceeds the runtime, or requires viewers to read dense text or inspect evidence too quickly."}, {"id": "c6", "criterion": "Viewers understand the option and leave with a practical next step.", "full_marks_when": "Explains an MVNO in accessible language as a cheaper provider using an existing carrier's network and gives a concrete next step, such as comparing an alternative on their current network against their bill and data needs. Fits a personal-finance audience aged 22\u201335 and avoids implying that shared towers guarantee identical speeds, priority, or every plan feature.", "zero_when": "Leaves MVNO unexplained, offers only an engagement request instead of useful guidance, or asserts that every aspect of service is identical merely because the towers are shared."}], "runs": {"haiku-1": {"cost_usd": 0.00585, "seconds": 31, "output_tokens": 6331, "files": [{"name": "script.md", "text": "# 30-second script: switch to an MVNO on the same network\n\nTarget length: about 75 words of voiceover. The hook is the first 3 seconds, so there is no intro and no \"hey guys\". The video opens on the number.\n\n**0:00\u20130:03 \u2014 HOOK**\n- VO: \"Same towers. My phone bill went from $85 to $25.\"\n- On screen: Real bill screenshot with \"$85\" in large type and a red strikethrough. \"$25\" slams in at 0:02.\n- Visual: Tight on the bill. Motion on the number, not the face.\n\n**0:03\u20130:10 \u2014 SETUP**\n- VO: \"Most people overpay. Cheaper brands called MVNOs run on the same big networks you already use.\"\n- On screen: \"MVNO = same network, lower price\". Coverage map of the host network with tower dots.\n- Visual: Map zooms in on your neighborhood.\n\n**0:10\u20130:19 \u2014 THE SWITCH**\n- VO: \"I checked the coverage map first. Same towers as my old plan. Then I switched, and it took ten minutes.\"\n- On screen: Coverage maps side by side (old plan | MVNO), both showing the same tower dots. Timer overlay on the signup.\n- Visual: Screen recording of the signup, sped up.\n\n**0:19\u20130:26 \u2014 THE MATH**\n- VO: \"That's $60 a month back. Over a year, $720. Many people save $300 to $500 a year.\"\n- On screen: \"$60 \u00d7 12 = $720\", then \"Typical savings: $300\u2013$500/yr\" below it.\n- Visual: Clean text on a dark background, or a calculator screen.\n\n**0:26\u20130:30 \u2014 CTA**\n- VO: \"Pull up your bill tonight. Follow for the switch, step by step.\"\n- On screen: \"Check your bill tonight\" text, then the follow prompt.\n- End frame: \"$25\" again, which loops back into the hook.\n\n## Before filming\n\n- **Math:** Your own numbers ($85 to $25) save $720 a year, which is above the $300\u2013$500 range. The script states both. Keep \"many people\" and avoid \"most people\" unless you have data behind it.\n- **\"Same towers\":** This is true only if the MVNO runs on the same host network as your old carrier. Confirm that before filming. MVNOs can be deprioritized during congestion and may have hotspot or data caps, so add that to the caption. Otherwise the comments will raise it.\n- **\"Took ten minutes\":** Say this only if it's accurate for your switch.\n- **Redact the bill:** Hide the account number, address, phone number, and IMEI.\n"}]}, "haiku-2": {"cost_usd": 0.00597, "seconds": 30, "output_tokens": 5526, "files": [{"name": "script.md", "text": "# 30s Vertical Video Script: Overpaying for Your Phone Plan\n\nTarget: TikTok / Reels, 9:16. Spoken pace ~170 wpm, about 85 words total.\n\n## Main script\n\n| Time | Visual | Voiceover | On-screen text |\n|---|---|---|---|\n| 0:00\u20130:03 | Tight shot, face close to camera. Phone bill on the table, \"$85\" circled in red. Quick push-in on the number. | \"I was paying $85 a month for the same towers. Now it's $25.\" | **$85 \u2192 $25** |\n| 0:03\u20130:08 | Cut to face, direct to camera. Simple text card appears. | \"Carriers count on you never checking. The cheap brands run on the same network. You're paying for the logo.\" | \"Same network. Different logo.\" |\n| 0:08\u20130:14 | Screen recording: old bill, then new bill, side by side. Then the carrier's coverage map with the cheaper brand's map overlaid. | \"Here's the bill before. Here's after. Same network. I checked the coverage map first.\" | \"Same towers\" (on the map) |\n| 0:14\u20130:22 | Three numbered text cards, one per step, each with a quick b-roll cut. | \"Find your carrier's network. Find a cheaper brand on that network. Check coverage where you live, then switch.\" | \"1. Find your network  2. Find the cheap brand  3. Check coverage\" |\n| 0:22\u20130:27 | Calculator on screen, then the number. | \"That's $60 a month. $720 a year. Most people save $300 to $500.\" | **$720 / YEAR** |\n| 0:27\u20130:30 | Face, end on a point. | \"Check your bill tonight. Follow for part two.\" | \"Follow for part 2\" |\n\n**Disclaimer (on-screen only, final 3 seconds, small text):** \"Coverage and data limits vary by location. Check before switching.\"\n\n## Hook variants (for the first-3-second test)\n\nPrevious videos lost 70% of viewers in the first 3 seconds, so the opening is the thing to test. Film all three openings. Use the same body for each, and keep the first frame in motion with no logo or title card.\n\n**A. Number first (recommended as default, used in the main script)**\n- Visual: Tight shot, bill with \"$85\" circled.\n- VO: \"I was paying $85 a month for the same towers. Now it's $25.\"\n- On-screen: \"$85 \u2192 $25\"\n\n**B. Pattern interrupt**\n- Visual: Starts on silence. A hand slaps the phone bill down onto the table.\n- VO: \"Stop paying extra for a logo.\"\n- On-screen: \"Stop paying extra for a logo.\"\n\n**C. Direct question**\n- Visual: Face, looking straight at camera.\n- VO: \"What are you paying for your phone? If it's over $30, watch this.\"\n- On-screen: \"Paying over $30?\"\n\n## Production notes\n\n- Captions on all text. Many viewers watch with the sound off, so the on-screen text has to carry the hook alone.\n- The \"same towers\" claim depends on the coverage map. Show the real carrier map on screen, with your address or area visible, so viewers can check it themselves.\n- Blur or crop account numbers, names, and any personal details on the bill.\n\n## Things to check before filming\n\n- **Savings math.** The on-screen proof ($85 \u2192 $25) works out to $60 a month, or $720 a year. The topic file says typical savings are $300 to $500 a year. The script presents $720 as your result and $300 to $500 as the typical range. If you want the script to match the lower number, change the 0:22\u20130:27 voiceover and the \"$720 / YEAR\" card.\n- **\"Same towers.\"** An MVNO runs on the host network, but it can still have data deprioritization or lower data caps than the main brand plan. The disclaimer covers this. Confirm the specific cheap brand you name actually uses the same network as your carrier.\n- **Brand names.** The script leaves out specific carriers and MVNO names. Adding them is fine, but check each one's current terms before posting.\n"}]}, "opus-1": {"cost_usd": 0.12269, "seconds": 41, "output_tokens": 3523, "files": [{"name": "script.md", "text": "# Phone Plan / MVNO: 30s Vertical Script\n\n**Format:** 9:16, about 30s, talking head with proof cut-ins\n**Goal:** Fix the 70% drop in the first 3 seconds. The proof is on screen in frame 1, before any talking, and the payoff is promised but held back until second 18.\n**Spoken length:** about 80 words, which fits 30s at a fast, natural pace.\n\n---\n\n## THE HOOK (0:00\u20130:03): the part that matters most\n\n**Why previous videos lose people:** most openers are a face plus a slow setup (\"So today I want to talk about...\"). Viewers decide in about 1 second, usually with the sound off. So:\n\n- **Frame 1 is the bill, not your face.** Your real $85 bill fills the screen. Hard cut to the $25 bill at about 0:01.\n- **The text is up from frame 1** so sound-off viewers get the hook too.\n- **The first spoken word is a number.** No \"hey guys\", no intro.\n- **Open a loop:** \"same towers\" makes people ask *how?* They only get the answer if they stay.\n\n| Time | Visual | On-screen text | Voiceover |\n|---|---|---|---|\n| 0:00\u20130:01 | Close-up of the real $85/mo bill, total circled in red. Slight push-in. | **$85/month** | \"Eighty-five dollars.\" |\n| 0:01\u20130:03 | Hard cut (whoosh or snap SFX) to the $25/mo bill. Then your face pops in, bottom third, holding up your phone. | **\u2192 $25. Same towers.** | \"Now I pay twenty-five. Same phone, same towers.\" |\n\n---\n\n## FULL SCRIPT\n\n| Time | Visual | On-screen text | Voiceover |\n|---|---|---|---|\n| 0:00\u20130:01 | $85 bill, full screen, total circled | **$85/month** | \"Eighty-five dollars.\" |\n| 0:01\u20130:03 | Snap cut to $25 bill, then your face pops in holding the phone | **\u2192 $25. Same towers.** | \"Now I pay twenty-five. Same phone, same towers.\" |\n| 0:03\u20130:06 | Talking head, tight framing, quick zoom on \"rent\" | **Big carriers RENT their network** | \"Big carriers rent their towers to smaller companies called MVNOs\u2026\" |\n| 0:06\u20130:09 | B-roll or graphic: big carrier logo, arrow to 3\u20134 MVNO logos | **MVNO = same network, cheaper** | \"\u2026who sell you the exact same coverage for way less.\" |\n| 0:09\u20130:14 | Split screen: old carrier coverage map / new MVNO map. Wipe across to show they match. | **Old map vs new map: identical** | \"This is my old carrier's map. This is my new one. Same towers.\" |\n| 0:14\u20130:18 | Talking head, counting on fingers | **1. Search \"MVNO on [your carrier]\"  2. Check your phone works  3. Keep your number** | \"Search 'MVNO on' plus your carrier, check your phone's compatible, keep your number. Ten minutes.\" |\n| 0:18\u20130:23 | Back to bills side by side, big number animates up | **Most people save $300\u2013500/yr. I save $720.** | \"Most people save three to five hundred a year. I'm saving seven-twenty.\" |\n| 0:23\u20130:26 | Talking head, honest tone, small shrug | **The catch:** | \"The catch? When the network's packed, you might get slowed down first. I've barely noticed.\" |\n| 0:26\u20130:30 | Point down to the comments. End on the $25 bill (loops back to the opening). | **Comment your carrier \ud83d\udc47** | \"Comment your carrier and I'll tell you which ones run on it.\" |\n\n---\n\n## WHY IT'S BUILT THIS WAY\n\n- **Proof before the claim.** Personal-finance viewers have seen a lot of \"save $500!\" bait. Showing *your own bill* in frame 1 earns trust before you've said anything.\n- **\"Same towers\" is the curiosity gap.** Cheaper usually means worse coverage, so the claim breaks what viewers expect, and the explanation comes 3 seconds later.\n- **A visual change at least every 3 seconds** (bill \u2192 face \u2192 graphic \u2192 map \u2192 fingers \u2192 numbers) cuts down on swiping away in the middle.\n- **Honest numbers.** \"$300\u2013500\" is the typical saving; \"$720\" is your real one ($60 \u00d7 12). Giving both keeps it credible.\n- **Saying the catch** (deprioritization) builds trust with a 22\u201335 audience and heads off \"well actually\" comments. It still adds comments, without the negative tone.\n- **The CTA invites a reply.** \"Comment your carrier\" is easy to answer and gives you material for follow-up videos (\"Which MVNOs run on Verizon?\").\n- **Loop ending.** Ending on the $25 bill flows back into the $85 opening, which pushes up rewatches.\n\n---\n\n## ALTERNATE HOOKS (A/B test these)\n\n1. **Visual:** $85 bill \u2192 $25 bill, VO: *\"Same phone. Same towers. Sixty bucks less. Every month.\"*\n2. **Challenge:** Face to camera holding phone, text **\"You're overpaying for this\"**, VO: *\"If you pay more than $40 for your phone plan, you're getting ripped off.\"*\n3. **Secret:** Coverage map on screen, VO: *\"Your carrier doesn't want you to know they rent these towers out for a third of the price.\"*\n\nKeep the first 3 seconds identical across versions apart from the hook, so you can tell which one actually improves retention.\n\n---\n\n## PRODUCTION NOTES\n\n- **Captions:** burned in, big, centered in the upper-middle (clear of the TikTok/Reels UI at the bottom and right).\n- **Blur** your account number, address, and phone number on both bills.\n- **Pacing:** cut every pause and breath. No silence over 0.3s.\n- **Audio:** low trending instrumental under the VO. SFX on the bill cut and the map wipe.\n- **Caption/description:** \"I was overpaying $720/yr for the exact same coverage \ud83d\udcf1 #personalfinance #moneytips #phoneplan #savemoney\"\n- **Accuracy check:** confirm the MVNO's speeds and policies (hotspot, international, data priority) before posting. Mention any limits that apply to your plan.\n"}]}, "opus-2": {"cost_usd": 0.10639, "seconds": 34, "output_tokens": 2864, "files": [{"name": "script.md", "text": "# Phone Plan Video: 30s Vertical Script\n\n**Format:** 9:16, about 30s, talking head plus screen inserts. Burned-in captions all the way through, because most people watch on mute.\n**Goal:** Fix the drop-off in the first 3 seconds. The first frame shows the proof and the voiceover starts on frame 1. No intro, no \"hey guys\", no logo.\n\n---\n\n## Why this hook should work\n\nPrevious videos lose 70% of viewers by 0:03, which usually means nothing in the first second gave them a reason to stay. This script:\n\n1. **Shows the proof on frame 1.** The two bills are already split-screen before you say anything, so a viewer on mute sees \"$85 vs $25\" right away.\n2. **Opens a gap that only the video closes.** \"Same towers\" with a $60 price difference makes people ask \"how?\", and the answer comes at about 0:08, not at the end.\n3. **Keeps the first sentence under 2 seconds and ends it on the number.**\n4. **Puts a cut or new visual every 2\u20133 seconds** so there's no still frame to scroll past.\n\n---\n\n## Script\n\n| Time | Visual | Voiceover | On-screen text |\n|---|---|---|---|\n| **0:00\u20130:02** | **FRAME 1:** split screen. Left: your real $85 bill. Right: your real $25 bill. Your finger taps the $85. | \"I was paying eighty-five bucks for *this*\u2026\" | **$85/mo** vs **$25/mo** |\n| **0:02\u20130:04** | Hard cut to a selfie close-up, deadpan. | \"\u2026same towers, same signal as the twenty-five dollar one.\" | SAME TOWERS. \ud83e\udd28 |\n| **0:04\u20130:07** | The coverage map slides in. Overlay both carriers' maps or flip between them, with an arrow on the identical footprint. | \"This is the coverage map. Both plans run on the exact same network.\" | Same network. Same map. |\n| **0:07\u20130:12** | Back to camera, walking or leaning in. B-roll of the big-carrier store. | \"The trick: smaller carriers called MVNOs rent space on the big networks and charge way less. No stores, no ads, no massive overhead.\" | MVNO = same network, cheaper |\n| **0:12\u20130:16** | Calculator on screen: 60 \u00d7 12 = **$720** | \"I'm saving sixty a month. That's seven hundred twenty dollars a year.\" | **$720/yr saved** \ud83d\udcb8 |\n| **0:16\u20130:19** | Back to camera, point at the viewer. | \"Most people save somewhere around three to five hundred a year.\" | Typical: $300\u2013500/yr |\n| **0:19\u20130:26** | Fast 3-step list. Each step pops on with a sound effect. | \"Here's how: one, check which network you're on. Two, search 'MVNO on' plus that network. Three, keep your number. Switching takes about ten minutes.\" | 1. Find your network<br>2. Search \"MVNO on [network]\"<br>3. Keep your number \u2705 |\n| **0:26\u20130:30** | Hold up the phone with the $25 bill, then cut to the end card. | \"Go check your bill. Then comment what you're paying. I'll tell you if you're overpaying.\" | What do you pay? \ud83d\udc47 |\n\n**Word count:** about 105 words. That fits 30s at a brisk creator pace. If the read runs long, cut the 0:16\u20130:19 line first and move \"Typical: $300\u2013500/yr\" to the text over the calculator shot.\n\n---\n\n## Delivery notes\n\n- **Get the first line done in 2 seconds.** Start speaking on frame 1, with no breath and no smile-before-talking.\n- Say \"eighty-five\" with real annoyance. The emotion at 0:00 matters more than anything later in the video.\n- Use jump cuts between lines and remove every pause.\n- Loop trick: the last frame (the $25 bill in your hand) should look like the first frame, so a replay feels seamless.\n\n## Accuracy / trust notes (keep these in)\n\n- **$720 is your number. $300\u2013500 is the typical range.** The script labels which is which so viewers don't feel misled.\n- **\"Same coverage\" fine print:** MVNO data can be deprioritized when the network is congested, so it may be slower in crowded places. Put a small on-screen note at 0:04\u20130:07 (\"speeds can be lower when the network is busy\") or address it in a pinned comment. Commenters will raise it anyway, and getting there first builds trust.\n- Blur your account number, address, and phone number on both bills.\n- If you name specific MVNOs, check that they really use the network shown on your map, and disclose any affiliate links.\n\n---\n\n## Alternate hooks (A/B test these, same body)\n\n1. \"Your phone company is charging you for a store you never walk into.\" (cold open on the carrier storefront)\n2. \"Same towers. Same signal. Sixty dollars a month cheaper.\" (open on the coverage map)\n3. \"If you pay more than $40 for your phone plan, watch this.\" (direct callout, works well for 22\u201335)\n\n## Caption\n\n> I was paying $85/mo for the same towers as a $25 plan \ud83e\udee0 Most people can save $300\u2013500/yr by switching to an MVNO on their current network. What do you pay? \ud83d\udc47\n> #personalfinance #moneytips #phoneplan #savemoney\n"}]}}, "verdicts": [{"pair": "2", "judge": "gpt", "order": "haiku shown first", "winner": "haiku", "why": "Both hook with concrete savings and show proof. A is closer to a usable runtime; B substantially understates its word count and adds unsupported overhead and switching-time claims. Both need tighter dialogue and better guidance on data needs.", "haiku_scores": {"c1": 2, "c2": 1, "c3": 2, "c4": 2, "c5": 1, "c6": 1}, "opus_scores": {"c1": 2, "c2": 1, "c3": 2, "c4": 2, "c5": 0, "c6": 1}}, {"pair": "2", "judge": "gpt", "order": "opus shown first", "winner": "opus", "why": "A shows both bills immediately and explains MVNOs more clearly. Both calculate savings correctly but add unsupported claims. Both substantially understate spoken word counts and overload their timing; B also spends early seconds on unsupported logo rhetoric.", "haiku_scores": {"c1": 2, "c2": 1, "c3": 2, "c4": 1, "c5": 0, "c6": 2}, "opus_scores": {"c1": 2, "c2": 1, "c3": 2, "c4": 2, "c5": 0, "c6": 2}}, {"pair": "2", "judge": "deepseek", "order": "haiku shown first", "winner": "tie", "why": "Both scripts excel: strong hooks, accurate math, clear evidence, sustained flow, practical next steps, and perfect 30s fit.", "haiku_scores": {"c1": 2, "c2": 2, "c3": 2, "c4": 2, "c5": 2, "c6": 2}, "opus_scores": {"c1": 2, "c2": 2, "c3": 2, "c4": 2, "c5": 2, "c6": 2}}, {"pair": "2", "judge": "deepseek", "order": "opus shown first", "winner": "opus", "why": "A sustains attention better and gives a clearer next step; B's 'follow for part two' withholds resolution.", "haiku_scores": {"c1": 2, "c2": 2, "c3": 2, "c4": 1, "c5": 2, "c6": 1}, "opus_scores": {"c1": 2, "c2": 2, "c3": 2, "c4": 2, "c5": 2, "c6": 2}}, {"pair": "1", "judge": "gpt", "order": "haiku shown first", "winner": "opus", "why": "B explicitly shows both actual bills, connects coverage maps to the claim, and gives actionable switching guidance. Both calculate savings correctly but introduce unsupported claims and underestimate spoken length. A never clearly shows the actual after bill.", "haiku_scores": {"c1": 2, "c2": 1, "c3": 1, "c4": 2, "c5": 1, "c6": 1}, "opus_scores": {"c1": 2, "c2": 1, "c3": 2, "c4": 2, "c5": 1, "c6": 2}}, {"pair": "1", "judge": "gpt", "order": "opus shown first", "winner": "opus", "why": "A better connects both actual bills and coverage proof, explains MVNOs, and provides actionable guidance with a congestion caveat. Its dialogue substantially exceeds 30 seconds. B is tighter but lacks explicit after-bill evidence and meaningful comparison guidance. Both invent switching-time claims.", "haiku_scores": {"c1": 2, "c2": 1, "c3": 1, "c4": 2, "c5": 1, "c6": 1}, "opus_scores": {"c1": 2, "c2": 1, "c3": 2, "c4": 2, "c5": 0, "c6": 2}}, {"pair": "1", "judge": "deepseek", "order": "haiku shown first", "winner": "haiku", "why": "A's sequence resolves curiosity faster and avoids withholding payoff until second 18, which risks losing attention.", "haiku_scores": {"c1": 2, "c2": 2, "c3": 2, "c4": 2, "c5": 2, "c6": 2}, "opus_scores": {"c1": 2, "c2": 2, "c3": 2, "c4": 1, "c5": 2, "c6": 2}}, {"pair": "1", "judge": "deepseek", "order": "opus shown first", "winner": "tie", "why": "Both open with compelling bill comparisons, use evidence correctly, and provide clear, actionable guidance within 30 seconds.", "haiku_scores": {"c1": 2, "c2": 2, "c3": 2, "c4": 2, "c5": 2, "c6": 2}, "opus_scores": {"c1": 2, "c2": 2, "c3": 2, "c4": 2, "c5": 2, "c6": 2}}]}, {"id": "pov-1", "family": "pov-post", "prompt": "Write a LinkedIn post for me based on notes.md. I want it to get discussion, not just likes.", "input": "notes.md", "input_text": "Me: head of product at a 120-person B2B SaaS company.\nWhat I've seen: our team shipped 40 features last year; usage data shows 31 of them are used by under 5% of customers. The 3 features that drove expansion revenue were all improvements to existing workflows, not new features.\nWhat I now believe: product teams should be measured on adoption of what they shipped, not on what they shipped. We now have a rule: no new feature starts until the last one reaches 20% adoption of its target segment.\n", "rubric": [{"id": "c1", "criterion": "Use the supplied evidence accurately, preserving numbers and denominators.", "full_marks_when": "Accurately states any figures used: 40 features shipped last year; 31 used by under 5% of customers; 3 features drove expansion revenue, all improvements to existing workflows. If calculated, 31/40 equals 77.5%. Keeps the 20% adoption threshold specific to each feature\u2019s target segment.", "zero_when": "Misstates the figures, makes arithmetic errors, treats low usage as zero usage, or applies the 20% threshold to all customers."}, {"id": "c2", "criterion": "Make the author's change in product judgment the central argument.", "full_marks_when": "Clearly argues for measuring teams by adoption of what they ship rather than shipment volume, connecting that belief to the low usage of most shipped features and the expansion revenue from workflow improvements.", "zero_when": "Celebrates shipping more features, offers generic product advice, or omits the shift from output to adoption."}, {"id": "c3", "criterion": "Present the team's concrete operating rule faithfully.", "full_marks_when": "States that the team now starts no new feature until the previous one reaches 20% adoption of its target segment, presenting it as their actual practice rather than a hypothetical recommendation.", "zero_when": "Omits the rule, changes its threshold or sequencing, or presents it merely as something the team might try."}, {"id": "c4", "criterion": "Invite substantive discussion about the claim or rule.", "full_marks_when": "Ends with a focused, answerable invitation to share experience or challenge a real tradeoff, such as when an adoption gate helps or blocks valuable work. Gives readers something specific to debate.", "zero_when": "Has no discussion invitation or relies only on generic engagement bait such as 'Agree?', 'Thoughts?', or requests for likes and shares."}, {"id": "c5", "criterion": "Keep the author's voice and claims credible for their situation.", "full_marks_when": "Reads as a first-person reflection by a product leader at a 120-person B2B SaaS company. Distinguishes observations and beliefs from proven general truths, without inventing revenue amounts, causes of low adoption, or results achieved after introducing the rule.", "zero_when": "Invents material facts or success claims, misrepresents the author's role or company, or claims the evidence proves that new features never create value."}, {"id": "c6", "criterion": "Deliver a publishable LinkedIn post with a clear reason to keep reading.", "full_marks_when": "Provides finished post copy with an opening grounded in the surprising evidence or adoption argument, readable paragraphs, and a coherent progression from observation to belief to practice to discussion. Requires no substantive rewriting.", "zero_when": "Provides only advice, an outline, or a rubric instead of a post, or produces copy so rambling, generic, or cluttered that the argument and discussion invitation are hard to find."}], "runs": {"haiku-1": {"cost_usd": 0.00352, "seconds": 12, "output_tokens": 2162, "files": [{"name": "linkedin_post.md", "text": "We shipped 40 features last year.\n\n31 of them are used by fewer than 5% of our customers.\n\nThe 3 that actually drove expansion revenue? None were new features. They were improvements to workflows people already used.\n\nThat changed how I think about product teams. We've been measuring them on what they ship. I now believe we should measure them on whether what they shipped gets adopted.\n\nSo we made a rule: no new feature starts until the last one reaches 20% adoption in its target segment.\n\nI'm not confident 20% is the right bar. Some segments are small, and some features are built for a narrow job, where 20% of the right customers is a win and 20% of everyone is noise.\n\nHere's the question I keep coming back to: if you had to choose, would you rather your team ship 10 features that 60% of customers use, or 40 that 5% use?\n\nFor anyone who has tied roadmap decisions to adoption metrics, what did you get wrong first?\n"}]}, "haiku-2": {"cost_usd": 0.00356, "seconds": 13, "output_tokens": 2206, "files": [{"name": "linkedin_post.md", "text": "Last year my team shipped 40 features.\n\nThis year we checked usage. 31 of them are used by fewer than 5% of our customers.\n\nThe 3 features that drove the most expansion revenue weren't new features at all. They were improvements to workflows people already used.\n\nI think that says something about how we measure product teams. We count what we ship. Shipping is easy to count, and it feels like progress in a planning meeting. But it's the wrong scoreboard. A team can ship a lot and move almost nothing.\n\nSo we changed one rule: no new feature starts until the last one reaches 20% adoption in its target segment.\n\nThe first effect is uncomfortable. Some planned work now waits, and that's the point.\n\nThe obvious objection is that some features serve a small segment that pays a lot. A 20% bar across a segment of 50 accounts is not the same as 20% of the whole base, and I don't think we've solved that well yet.\n\nFor those of you who run product teams: do you measure adoption before you start something new? And if you've killed a feature that looked successful by shipping metrics, what made you do it?\n"}]}, "opus-1": {"cost_usd": 0.0687, "seconds": 16, "output_tokens": 1334, "files": [{"name": "linkedin-post.md", "text": "Last year my team shipped 40 features.\n\n31 of them are used by fewer than 5% of our customers.\n\nI'm head of product at a 120-person B2B SaaS company, and for a long time I would have called that year a success. We hit the roadmap. We shipped on time. The release notes were long.\n\nThen we looked at what drove expansion revenue. It came down to 3 things, and none of them were new features. All 3 were improvements to workflows customers already used every day.\n\nSo we were measuring the wrong thing. We counted what we shipped, when what mattered was what got used.\n\nWe've changed two things:\n\n1. Product teams are measured on adoption of what they shipped, not on how much they shipped.\n2. No new feature starts until the last one reaches 20% adoption in its target segment.\n\nThe second rule is the one that makes people uncomfortable, and I get why. The fair objections:\n\n\u2192 \"Adoption is sometimes a sales, marketing or onboarding problem, not a product problem.\"\n\u2192 \"Some features are niche by design. 20% is the wrong bar for them.\"\n\u2192 \"You'll slow down and lose deals to competitors with longer feature lists.\"\n\nI don't think any of those are wrong. But I'd rather argue about them than keep shipping features nobody touches.\n\nSo I'm curious where other product leaders come down on this:\n\nIf a feature is under 5% adoption a few months after launch, whose problem is that? Product's, go-to-market's, or was it never worth building?\n\nAnd if you measure your team on something other than output, what is it, and how did you get leadership to agree to it?\n\n#ProductManagement #SaaS #B2B\n"}]}, "opus-2": {"cost_usd": 0.07621, "seconds": 21, "output_tokens": 1635, "files": [{"name": "linkedin-post.md", "text": "# LinkedIn post\n\nWe shipped 40 features last year.\n\n31 of them are used by fewer than 5% of our customers.\n\nI run product at a 120-person B2B SaaS company, and that number has been hard to sit with. By the usual measures it was a great year: roadmap delivered, velocity up, lots of launches to announce.\n\nThen we looked at what actually drove expansion revenue. It came from 3 things, and none of them were new features. All 3 were improvements to workflows customers already used every day.\n\nSo we were rewarding the team for output, and our customers were rewarding us for something else.\n\nWhat I believe now: product teams should be measured on adoption of what they shipped, not on how much they shipped.\n\nSo we made a rule:\n\u2192 No new feature starts until the last one reaches 20% adoption in its target segment.\n\nI expect pushback, and some of it is fair:\n\u2022 Some features are strategic bets that take a long time to land.\n\u2022 Sometimes adoption is low because of sales, onboarding, or pricing, not the product.\n\u2022 Sales has deals that depend on the next feature on the roadmap.\n\nMy answer so far: if a feature can't reach a fifth of the customers it was built for, we should find out why before building the next one. If we can't, we're just adding more to the 31.\n\nI'd really like to hear from other product leaders:\n\n1. Would a rule like this work at your company, or would it fall apart in the first quarter?\n2. If not 20% adoption, what would you hold your team to?\n\n#ProductManagement #SaaS #ProductLeadership\n\n---\n\n## Alternative opening lines (swap in for the first two lines)\n\n- \"Our best product year by every internal metric was mostly wasted effort.\"\n- \"77% of what we shipped last year, almost nobody uses.\"\n\n## Posting notes\n\n- **Why it should get comments:** it takes a clear position, names the strongest objections itself, and closes with two specific questions people can answer from their own experience. \"Thoughts?\" gets likes. A rule people can argue with gets comments.\n- **The rule is new.** The post doesn't claim it has worked yet. If you have early results, even small ones, add one line after the rule. If not, leave it as is. Being open that it's an experiment invites people to give advice.\n- **Reply to early comments within the first hour or so,** especially the ones that disagree. Ask them follow-up questions rather than defending the rule. That's what keeps a thread going.\n- **Consider a follow-up post** in 3\u20136 months on how the rule held up. The people who argued with this one will want to know.\n"}]}}, "verdicts": [{"pair": "1", "judge": "gpt", "order": "haiku shown first", "winner": "haiku", "why": "A preserves the evidence and operating rule while inviting specific discussion. B equates low adoption with nobody using features and invents roadmap completion, on-time shipping, daily workflow usage, and an implemented measurement change.", "haiku_scores": {"c1": 2, "c2": 2, "c3": 2, "c4": 2, "c5": 2, "c6": 2}, "opus_scores": {"c1": 0, "c2": 2, "c3": 2, "c4": 2, "c5": 0, "c6": 2}}, {"pair": "1", "judge": "gpt", "order": "opus shown first", "winner": "haiku", "why": "B preserves the evidence and operating rule while inviting experience-based discussion. A equates low usage with \u201cnobody touches,\u201d invents roadmap and delivery successes, and claims adoption-based measurement is already implemented.", "haiku_scores": {"c1": 2, "c2": 2, "c3": 2, "c4": 2, "c5": 2, "c6": 2}, "opus_scores": {"c1": 0, "c2": 2, "c3": 2, "c4": 2, "c5": 0, "c6": 2}}, {"pair": "1", "judge": "deepseek", "order": "haiku shown first", "winner": "tie", "why": "Both posts accurately use evidence, center the judgment shift, state the rule, invite discussion, are credible, and are publishable.", "haiku_scores": {"c1": 2, "c2": 2, "c3": 2, "c4": 2, "c5": 2, "c6": 2}, "opus_scores": {"c1": 2, "c2": 2, "c3": 2, "c4": 2, "c5": 2, "c6": 2}}, {"pair": "1", "judge": "deepseek", "order": "opus shown first", "winner": "opus", "why": "A's discussion invitation is more substantive and answerable, asking 'whose problem is that?' and 'how did you get leadership to agree?'", "haiku_scores": {"c1": 2, "c2": 2, "c3": 2, "c4": 1, "c5": 2, "c6": 2}, "opus_scores": {"c1": 2, "c2": 2, "c3": 2, "c4": 2, "c5": 2, "c6": 2}}, {"pair": "2", "judge": "gpt", "order": "haiku shown first", "winner": "haiku", "why": "A preserves the evidence more closely, though \u201cmost\u201d changes the revenue claim and delayed work is invented. B adds unsupported performance claims and daily usage; its alternative opening miscalculates 31/40 as 77%. Both faithfully state the rule and invite substantive discussion.", "haiku_scores": {"c1": 1, "c2": 2, "c3": 2, "c4": 2, "c5": 0, "c6": 1}, "opus_scores": {"c1": 0, "c2": 2, "c3": 2, "c4": 2, "c5": 0, "c6": 1}}, {"pair": "2", "judge": "gpt", "order": "opus shown first", "winner": "haiku", "why": "B preserves the numbers and segment denominator. A\u2019s alternative opening incorrectly gives 77% instead of 77.5% and invents internal performance claims. Both invite substantive discussion, but B also invents an observed effect of the rule.", "haiku_scores": {"c1": 2, "c2": 2, "c3": 2, "c4": 2, "c5": 0, "c6": 2}, "opus_scores": {"c1": 0, "c2": 2, "c3": 2, "c4": 2, "c5": 0, "c6": 2}}, {"pair": "2", "judge": "deepseek", "order": "haiku shown first", "winner": "tie", "why": "Both accurately use all evidence, center the adoption argument, state the rule, invite specific discussion, and are publishable.", "haiku_scores": {"c1": 2, "c2": 2, "c3": 2, "c4": 2, "c5": 2, "c6": 2}, "opus_scores": {"c1": 2, "c2": 2, "c3": 2, "c4": 2, "c5": 2, "c6": 2}}, {"pair": "2", "judge": "deepseek", "order": "opus shown first", "winner": "opus", "why": "A's discussion invitation is more specific and answerable, directly inviting debate on the rule's feasibility and alternative thresholds.", "haiku_scores": {"c1": 2, "c2": 2, "c3": 2, "c4": 1, "c5": 2, "c6": 2}, "opus_scores": {"c1": 2, "c2": 2, "c3": 2, "c4": 2, "c5": 2, "c6": 2}}]}, {"id": "neg-1", "family": "negotiation", "prompt": "I have a negotiation on Monday, details in deal.md. Prepare me.", "input": "deal.md", "input_text": "I'm the founder of a 12-person dev-tools startup. A large bank (our biggest prospect) wants a 3-year enterprise contract.\nTheir procurement sent: 40% discount off list (list is USD 180k/year), net-90 payment terms, unlimited liability for data breaches, and the right to terminate for convenience with 30 days notice.\nWhat I know: their engineering team ran a 4-month pilot and loves it; their champion (Director of Platform) told me the budget is approved at USD 140k/year; there's one competitor in the process but the champion says it failed their security review.\nOur runway: 14 months. This deal would be 30% of our ARR.\n", "rubric": [{"id": "c1", "criterion": "Uses the deal facts and pricing arithmetic accurately.", "full_marks_when": "States that a 40% discount on USD 180k/year yields USD 108k/year, or USD 324k over three years if fully paid. Recognizes that USD 140k/year is a 22.22% discount from list and is USD 32k/year above procurement\u2019s request, a USD 96k difference over three years. Preserves the stated 14-month runway and 30% ARR exposure without inventing current ARR or a runway extension.", "zero_when": "Makes material factual or arithmetic errors, confuses annual and three-year amounts, or treats the proposed contract value as guaranteed revenue despite the termination clause."}, {"id": "c2", "criterion": "Assesses the founder\u2019s leverage realistically and identifies uncertainties to resolve.", "full_marks_when": "Uses the successful four-month pilot, engineering support, and champion\u2019s reported budget approval as leverage, while treating the competitor\u2019s security failure as unverified information. Accounts for the founder\u2019s cash pressure and customer concentration. Provides targeted questions about procurement\u2019s authority, remaining approvals, security status, purchasing deadline, and the champion\u2019s ability to support the deal internally.", "zero_when": "Assumes the bank has no alternative, treats the champion\u2019s statements as binding commitments, invents another customer or financing option, or ignores the founder\u2019s vulnerability."}, {"id": "c3", "criterion": "Prioritizes the contract risks and proposes workable protections.", "full_marks_when": "Explains why unlimited breach liability could threaten the startup\u2019s survival, why 30-day convenience termination undermines a three-year commitment, and why net-90 delays cash collection. Offers concrete alternatives such as a defined liability cap or breach-specific higher cap, narrower covered losses, termination restrictions or compensation for early exit, and annual advance payment or shorter payment terms. Identifies legal and insurance feasibility checks without claiming that proposed language is necessarily enforceable.", "zero_when": "Focuses only on price, accepts unlimited liability or freely cancellable economics without addressing their consequences, or makes unsupported legal guarantees."}, {"id": "c4", "criterion": "Provides a coherent opening proposal and conditional concession packages.", "full_marks_when": "Gives a specific opening position, a commercially plausible target informed by the USD 140k budget, and fallback packages covering price, payment, liability, and termination together. Makes concessions conditional on valuable reciprocal commitments, such as exchanging a lower price for advance payment and a protected term. Distinguishes proposed negotiation positions from known facts and avoids trading existential risk for a modest price increase.", "zero_when": "Offers only generic advice to negotiate, presents unrelated concessions with no exchange, or recommends an economically inconsistent package."}, {"id": "c5", "criterion": "Equips the founder to conduct Monday\u2019s conversation.", "full_marks_when": "Includes a usable meeting sequence, a concise opening statement, discovery questions, and example responses to demands for the 40% discount, net-90, unlimited liability, and convenience termination. Shows how to involve the champion, handle claims that terms are standard or nonnegotiable, and close with documented next steps, owners, and timing.", "zero_when": "Provides abstract concepts or motivational advice without language and actions the founder can use in the meeting."}, {"id": "c6", "criterion": "Establishes practical decision boundaries under cash pressure.", "full_marks_when": "Separates target terms from provisional red lines, explains when to pause or decline, and gives a realistic fallback if agreement is delayed or lost. Connects acceptable pricing and terms to delivery costs, cash collection timing, insurance, and concentration risk, identifying missing information needed to set a defensible minimum. Does not fabricate a precise walk-away price or assume that landing the deal automatically solves runway.", "zero_when": "Advises closing at any cost, supplies an unsupported numerical minimum, or ignores what the founder should do if the bank refuses essential protections."}], "runs": {"haiku-1": {"cost_usd": 0.00882, "seconds": 59, "output_tokens": 11087, "files": [{"name": "negotiation-prep.md", "text": "# Bank enterprise deal: negotiation prep\n\nMeeting assumed Monday 2026-10-12. Not legal advice: get counsel on every liability, indemnity and termination clause before you sign.\n\n## Bottom line\n\n- Price has room. Their 40% off ($108k/yr) is well below the $140k/yr budget the champion reported. Open above the budget and land around it.\n- Terms do not have room. Unlimited liability, 30-day termination for convenience, and net-90 each need to change, whatever the price. Liability is the one that can sink the company.\n- Never disclose runway, the 30%-of-ARR concentration, or where the $140k figure came from.\n\n## The numbers\n\n| Item | Value |\n|---|---|\n| List price | $180k/yr |\n| Their offer (40% off) | $108k/yr, $324k over 3 years |\n| Reported budget | $140k/yr (22% off list), $420k over 3 years |\n| Gap between offer and budget | $32k/yr, $96k over 3 years |\n| Cash timing cost of net-90 vs net-30 on $140k annual invoice | About $23k of cash held back for 60 extra days |\n\nThe net-90 cost is small per invoice but it lands in the months when runway matters most. Annual prepayment is worth more than the price difference on a small deal.\n\n## Position by term\n\n| Term | Their paper | Open | Target | Walk-away |\n|---|---|---|---|---|\n| Price | $108k/yr | $150k/yr (about 17% off list) | $135k\u2013$140k/yr | Provisional $125k/yr. Replace with your number after checking cost-to-serve and margin. |\n| Term | 3 years | 3 years, annual invoicing | 3 years, annual invoicing | Shorter term only at a much higher annual price |\n| Payment | Net-90 | Annual upfront, net-30 | Annual upfront, net-30 | Net-45 at most. Never quarterly in arrears. |\n| Liability | Unlimited for data breaches | General cap at 12 months' fees. Data-breach super-cap at 2x annual fees. | Super-cap at 2x\u20133x annual fees, tied to your cyber insurance limit | Any uncapped data-breach liability. Confirm with counsel. |\n| Termination | 30 days' notice for convenience | No convenience termination in year 1. After that, 90 days' notice. | 12 months firm, then 90 days' notice plus payment for the notice period | Any convenience exit in year 1 without a fee. Regulator-required exit rights are fine. |\n\nPrice is the tradeable item. Liability is the non-negotiable item. Termination and payment sit in between.\n\n## What to offer, and what to hold\n\nCheap for you to give, valuable to them:\n- Three-year commitment\n- Annual upfront invoicing (helps your cash)\n- Logo rights, a case study, or a reference call after go-live\n- An executive sponsor cadence\n- Price tied to scope: if they need a lower number, reduce seats or modules rather than cutting price on the full scope\n\nHold until they give something:\n- Price below $135k\n- Any change to the liability cap\n- Any termination right that isn't regulator-driven\n- Early-pay discounts (use only if they refuse net-30 and you need the cash)\n\nConcession order when you need to move: term and prepay, then price to $140k, then termination fallback, then super-cap up to 3x. Liability stays in place.\n\n## Leverage\n\nYours:\n- Four-month pilot success, and an engineering team that wants the product\n- A champion who is inside the org and will help you\n- Their switching cost after a pilot\n- The competitor reportedly failed their security review (verify this before using it)\n\nTheirs:\n- Your runway (14 months). This makes the deal feel urgent to you even if they never say so.\n- The deal is about 30% of your ARR, so losing it hurts badly.\n- Procurement's paper is an opening template. Treat it as one, but expect them to hold firm on liability.\n- They may have another vendor in the process.\n\n## Before Monday\n\n1. **Counsel review** of the liability cap, super-cap, indemnity, and termination language. Get a proposed cap structure from counsel before the meeting.\n2. **Cyber insurance**: find out your limit and what it covers. Banks often require coverage in the millions. If your policy is smaller, the super-cap must be sized to what you actually carry.\n3. **Security posture**: do you have SOC 2 Type II, ISO 27001, or equivalent? A regulated bank will probably ask. Know your answer.\n4. **Security review status**: ask the champion directly what remains open and who signs off. The competitor's failure does not tell you that your review is complete.\n5. **Budget**: ask the champion whether $140k/yr is the approved number in writing, and whether the bank can invoice annually. Fiscal-year budgeting often rules out multi-year prepay.\n6. **Decision path**: who signs? Procurement, legal, the CISO, and the CFO may each have a veto. Ask the champion to get procurement and legal on a call with you.\n7. **Your floor**: agree the walk-away price and the non-negotiable terms with your co-founder or board before you walk in. Write them down.\n8. **Runway model**: calculate how much cash the deal adds at annual invoicing, net-30 vs net-90, and how much a 3-month signing delay costs you.\n\n## Monday: how to run the meeting\n\n- Start by saying you want to get this done and you're going to lay out what a deal looks like on your side. Don't accept or reject line items one at a time.\n- Present a package: price, term, payment, liability, and termination together.\n- Don't agree to anything on the call. Ask for their redline in writing and set a response date.\n- Keep the silences. Don't fill them with concessions.\n\nLines to use:\n- \"We can work within your budget range, but only on a 3-year term with annual invoicing. Our standard payment term is net-30.\"\n- \"We can't sign unlimited liability for a company our size. We can offer a general cap at 12 months' fees and a higher super-cap for data breaches, sized to our cyber coverage.\"\n- \"A convenience exit at 30 days doesn't work after a 3-year commitment. We can do 12 months firm, then 90 days' notice. Regulator-required exit rights we'll accept as written.\"\n- \"We're happy to reduce scope to fit a lower number. Which modules or seats would you cut?\"\n- \"Send us your redlines on the liability section and we'll respond within 48 hours.\"\n\n## Likely pushback\n\n- **\"108k is our number.\"** \"We understand the constraint. Our list is $180k, and we can't get to $108k on the full scope. We can scope to what you need at a lower number, or we can hold at the investment level that works for both of us.\"\n- **\"Unlimited liability is our policy.\"** \"We understand your standards. Can you route an exception request through your vendor risk process? Our insurance is the constraint, and we can talk about the super-cap size.\"\n- **\"Regulators require us to have termination rights.\"** \"Agreed. Regulator-directed exit rights are standard, and we'll write them in. A general right to leave after 30 days is a different thing.\"\n- **\"Net-90 is our standard.\"** \"Then we'll need annual prepayment, or we'll need to price the net-90 into the annual fee.\"\n\n## Things to avoid\n\n- Don't trade price for liability. Price can move; the cap should not.\n- Don't reveal the $140k source, your runway, or the ARR concentration.\n- Don't give a \"best efforts\" SLA or custom features to close. Check engineering capacity first.\n- Don't let the timeline slip into a signing date you can't meet.\n\n## Open questions and assumptions\n\n- \"30% of ARR\": I'm reading this as the deal's annual value relative to current ARR. If the deal is $108k, that implies ARR of about $360k. Confirm, because it changes how exposed you are.\n- Runway (14 months) is taken at face value. Confirm monthly burn and whether 14 months is at current burn.\n- The $125k floor is a placeholder. Replace it with a number based on your margin and cost-to-serve.\n- The champion's claims about budget, the competitor's failed review, and the pilot's status are unverified. Confirm each in writing where you can.\n"}]}, "haiku-2": {"cost_usd": 0.01152, "seconds": 81, "output_tokens": 15426, "files": [{"name": "negotiation-prep.md", "text": "# Bank enterprise deal: negotiation prep (Monday)\n\n## Bottom line\n\n1. **Do not accept unlimited liability for data breaches at any price.** A single breach claim from a bank could exceed the company's total resources. This is the one term to hold absolutely.\n2. **The termination-for-convenience clause matters more than the price.** With 30 days' notice, a \"3-year\" contract is effectively a one-month commitment. Price only means something once the term is locked.\n3. **Price has room.** Procurement asks $108k/yr (40% off list). The champion says budget is approved at $140k/yr. Open around $150k, aim for $135-140k, and treat $125k as a floor that only holds if the structure terms are met.\n4. **Net-90 is a cash cost, not a dealbreaker.** Push for net-30 on annual prepay, and accept net-60 at most.\n5. **Our own security review is the real gate.** The competitor failing is the champion's claim, and it does not mean we pass. Confirm our security status before Monday.\n\n## The numbers\n\n| Item | Value |\n|---|---|\n| List price | $180k/yr |\n| Procurement ask (40% off) | $108k/yr, $324k over 3 years |\n| Champion's stated budget | $140k/yr (22% off list), $420k over 3 years |\n| Gap between ask and budget | $32k/yr, $96k over 3 years |\n| Net-90 vs net-30 cost on $140k/yr | About $23k of cash held by the bank at any time (timing cost, not a loss) |\n| Implied ARR if the deal is 30% of ARR | About $360k at $108k; about $467k at $140k (confirm which applies) |\n\nFacts to confirm before Monday: cash on hand, monthly burn, current ARR, and whether \"30% of ARR\" is measured with or without this deal. Those numbers set how much a deal failure hurts and how long we can afford to hold out.\n\n## Term-by-term plan\n\nEach term lists the opening position, the fallback, and the floor.\n\n### Price (annual)\n- **Open:** $150k/yr on a 3-year commitment, annual prepay.\n- **Target:** $135-140k/yr.\n- **Floor:** $125k/yr (about 31% off list), only if liability, termination, and payment terms are all acceptable.\n- **Decline:** $108k/yr with their current structure. That is below the floor and attached to terms we cannot accept.\n- Add a 5% annual price escalator (or CPI capped at 5%) to protect the later years of the term.\n\n### Payment terms\n- **Open:** Net-30, billed annually in advance.\n- **Fallback:** Net-45 with annual prepay.\n- **Floor:** Net-60 with annual prepay. Reject net-90, and reject any net-60 or net-90 on monthly or quarterly billing.\n- Add a late-payment interest clause and termination for non-payment (30-day cure).\n- Check their actual payment history with us if possible. Banks sometimes pay later than their stated terms.\n\n### Liability\n- **Open:** Mutual cap at 12 months of fees paid. Data-breach super-cap at 2x annual fees. Exclusions for fraud, willful misconduct, and gross negligence are standard and acceptable.\n- **Fallback:** Data-breach super-cap at 3x annual fees, or a figure tied to our cyber insurance limit, whichever is agreed.\n- **Floor:** No uncapped liability for data breaches. Ever.\n- Watch for \"breach costs are direct damages\" language, which can make a cap meaningless. Ask counsel to review it.\n- Check the DPA separately. It may carry its own liability terms that override the MSA.\n\n### Termination for convenience\n- **Open:** No termination for convenience in the first 12 months. After that, 90 days' notice with no fee.\n- **Fallback 1:** Termination for convenience effective at end of year 1, with 90 days' notice and no fee.\n- **Fallback 2:** Termination at any time with 90 days' notice and an early-termination fee of 50% of the remaining committed fees.\n- **Floor:** No termination for convenience before month 12.\n- Regulator-directed termination is a separate item. Banks often need it and it is usually legitimate. Accept it as its own clause, not as a general right to walk.\n\n### Other terms to check before signing\n- SLA and service credits (cap credits, make them the sole remedy).\n- Security and audit rights (how often, who pays, how much notice).\n- Data residency, subprocessors, and incident-notification window.\n- Insurance requirements they impose on us (confirm our limits meet them).\n- Whether the bank will sign our paper or insists on theirs. Negotiating on their template is harder. Propose our MSA as the base.\n\n## Leverage and what they need\n\n**Our leverage:**\n- A 4-month pilot the engineering team likes. Quantify the value they got (time saved, incidents avoided, seats used). Make the value case, not the price case.\n- The champion is an internal advocate who wants this to succeed.\n- The competitor reportedly failed security review. Do not criticize the competitor directly. Use it only as a reason their procurement should be confident in us.\n- A 3-year commitment is worth something to us. Price it explicitly.\n- A reference customer and logo are valuable to a 12-person company. Offer them in exchange for price (5-10% is a reasonable trade).\n\n**Their likely interests:**\n- Procurement wants a lower price and standard bank terms (net-90, broad liability, flexible exit). These are often template defaults, not hard requirements.\n- Legal and risk want protection against vendor failure and breach. A stronger security posture helps more than a higher cap.\n- The champion wants the deal approved internally without a fight.\n- Regulatory rules may force some terms (for example, the EU's DORA or US third-party risk guidance require certain exit, audit, and termination provisions). Ask which terms are regulatory and which are preference.\n\n**Our constraints:**\n- 14 months of runway.\n- The deal is 30% of ARR, so losing it hurts badly. This pressure is real, but it should not push us into the uncapped liability term. The worst outcome is not losing the deal. It is signing a clause that bankrupts the company.\n\n## Trades (give and get)\n\n| We give | We get |\n|---|---|\n| 3-year commitment | Price moves to $135-140k |\n| Reference customer, case study, logo | 5-10% off |\n| Annual prepay (we want this) | Net-30 or net-45 |\n| Termination for convenience after year 1 | Liability cap, price, and the 3-year commitment |\n| Fixed 5% escalator (they may push back) | Net-30 and the cap |\n| Expansion roadmap commitments | Lower entry price |\n\nTrade as a package. Do not give a concession on one line without getting something back on another.\n\n## Expected pushback and responses\n\n- **\"The 40% discount is our standard.\"**\n  \"The 40% is a list-to-net for a one-year term without a commitment. We're offering a 3-year commitment. Here's the net price for that structure, and here's the value your team has seen in the pilot.\"\n\n- **\"Net-90 is our policy.\"**\n  \"We can work with your policy on annual prepay. On monthly or quarterly billing, we need net-30.\"\n\n- **\"Unlimited liability is our standard template.\"**\n  \"We can't sign uncapped liability for data breaches. Our insurance and balance sheet can't support it. Here's our cap and super-cap, and our insurance certificate. What's the number your other vendors carry?\"\n\n- **\"Termination for convenience is non-negotiable.\"**\n  \"Which regulation requires it? If it's regulatory, we'll align the clause to the regulatory language. If it's internal policy, we can agree to termination after year 1 with 90 days' notice.\"\n\n- **\"The competitor is cheaper.\"**\n  \"What's the competitor's price for the same scope and term? Our pilot covered [X]. Have they passed your security review?\"\n\n- **\"We need a better number this quarter.\"**\n  \"Our pricing holds through [date] for a signed 3-year deal. After that, we re-quote.\" Only say this if the date is real.\n\n- **\"We'll go with the competitor.\"**\n  Ask what they'd need to see to stay with us. Do not match a competitor price you have not verified. If they are bluffing, this usually surfaces the real constraint.\n\n## Monday: how to run the meeting\n\n1. Open with the pilot results and the business case. Do not open with price.\n2. Ask questions before you propose numbers (see below).\n3. Table a complete counter-proposal (price, payment, liability, termination) as one package, not as separate line items.\n4. Do not concede in the room. Say \"I need to confirm that internally\" and take the item offline.\n5. Do not agree to anything verbally. Ask for a written term sheet and send our paper as the base.\n6. Confirm the next step, owner, and date before the meeting ends.\n\n## Questions to ask on Monday\n\n- Who signs the contract, and what is the approval path? Who else must approve?\n- When does this year's budget expire, and what is their fiscal year-end?\n- Which of the terms are regulatory requirements, and which are internal policy? Can we see the regulatory text?\n- What do they need from us to close security review? What is the timeline?\n- What would make the 3-year commitment easier to approve?\n- What is the expected rollout: number of users, business units, and expansion plans?\n- What is their standard data-breach liability cap with other SaaS vendors?\n\n## Before Monday: to-do list\n\n1. **Legal:** Have counsel review the MSA, DPA, and liability language. Get a view on the cap and super-cap sizing, and whether \"breach costs\" are defined as direct damages.\n2. **Insurance:** Confirm our cyber and E&O coverage limits. Those limits set the super-cap we can offer.\n3. **Security:** Confirm our SOC 2 status, pen-test date, encryption, data residency, and incident-notification commitments. Find out what the competitor's failure reason was, as far as the champion can share.\n4. **Finance:** Confirm cash on hand, monthly burn, ARR, and the cash impact of net-90 and each termination option. Settle the walk-away point with your co-founder or board.\n5. **Champion:** Ask who signs, the approval path, the budget deadline, and which terms are regulatory. Ask the champion to help build the internal case with procurement.\n6. **Decide internally:** Agree on the price floor, the non-negotiables (uncapped liability, termination before month 12), and who has authority to approve any concession in the room.\n\n## Handling the champion's budget information\n\nThe $140k figure came from the champion. Treat it as confidential. Do not mention it to procurement, and do not reference it in the room. If it gets back to procurement, the champion is exposed, and we lose our best source of internal information. Anchor on the value of the pilot and the structure, not on their budget.\n\n## Assumptions and open questions\n\n- The $140k budget figure is secondhand and may not be a ceiling. Treat it as a reference point, not a fact.\n- The competitor's security failure is also secondhand. Confirm it before using it.\n- I do not know our cost to serve, gross margin, or cash position. The $125k floor is a starting point and should be checked against those numbers.\n- The regulatory points (DORA, US third-party risk guidance) depend on where the bank operates. Have counsel confirm which apply.\n- This is negotiation preparation, not legal advice. Have counsel review all liability, termination, and payment language before signing.\n"}]}, "opus-1": {"cost_usd": 0.19439, "seconds": 74, "output_tokens": 6398, "files": [{"name": "negotiation-prep.md", "text": "# Negotiation Prep: Bank Enterprise Contract (3-Year)\n\n**Meeting:** Monday, Oct 12, 2026 \u00b7 **Counterparty:** Bank procurement (champion: Director of Platform)\n\n---\n\n## 1. One-Page Cheat Sheet (bring this into the room)\n\n**Your goal:** A signed 3-year deal at about **$140k/yr**, with **capped liability**, **no penalty-free early exit**, and **cash you can actually collect on time**.\n\n| Term | Their ask | Open with | Target | Walk-away line |\n|---|---|---|---|---|\n| Price | $108k/yr (40% off) | $162k/yr (10% off, for a 3-yr commitment) | **$140k/yr** (22% off) | Under ~$125k/yr, and only if every other term goes your way |\n| Payment | Net-90 | Annual upfront, net-30 | Annual upfront, net-45/60 | Net-90 with **quarterly** billing |\n| Liability | Unlimited for data breach | General cap = 12 months of fees; data-breach super-cap = 2x annual fees | Super-cap of 2\u20133x annual fees, or your cyber-insurance limit | **Unlimited liability. Never accept it.** |\n| Termination for convenience | 30 days' notice | None during the 3-yr term | Not allowed in Year 1; after that, 90 days' notice plus repayment of the multi-year discount | 30 days' notice at a 3-year discount |\n\n**Remember:**\n- Their opening price is **$32k/yr below their own approved budget**. It's procurement's standard opening move, not a real limit.\n- **Every discount needs something in return.** Never move on price without getting something back.\n- **Don't reveal what the champion told you** (the budget figure, the competitor failing security). Act as if you don't know.\n- **Silence is fine.** After you make a counter, stop talking.\n- Close with: *\"If we can align on these four points, I can get paper to you this week.\"*\n\n---\n\n## 2. The Numbers\n\n| Scenario | Annual | 3-yr total | vs. list |\n|---|---|---|---|\n| List | $180k | $540k | \u2014 |\n| Approved budget (per champion) | $140k | $420k | \u221222% |\n| Their ask | $108k | $324k | \u221240% |\n\n- Giving them the full 40% costs you **$96k** over three years compared with their approved budget, which is money they already have.\n- **Net-90 with annual billing:** you deliver for 3 months before the first dollar arrives. On a 14-month runway, that's real cash pressure. Model it before Monday.\n- **Termination for convenience on 30 days' notice:** they could pay as little as ~$9\u201327k and leave, after getting a 3-year price. Your \"3-year contract\" would really be a month-to-month contract at a deep discount. **This is the term that hides the most value. Treat it as seriously as price.**\n\n---\n\n## 3. Leverage Assessment\n\n**Your leverage (stronger than it feels):**\n- A 4-month pilot, and the engineers love it. Switching costs and internal enthusiasm are already built up.\n- Their only competitor reportedly failed the security review. Their realistic alternative is to start procurement over, which costs months.\n- The budget is already approved. Procurement isn't fighting for money. They're trying to hit a savings target.\n- The champion has political capital invested in this choice.\n\n**Their leverage:**\n- You're small, with 14 months of runway, and this deal is 30% of ARR. Procurement can probably guess that you need it.\n- Bank procurement can stall indefinitely, and delay costs you more than it costs them.\n\n**Your BATNA (best alternative if there's no deal):** No deal. Keep the pipeline going and the runway stays at 14 months. That's painful but survivable. **A bad deal can be worse than no deal.** Unlimited liability could end the company with a single incident. Termination for convenience at a deep discount could leave you having staffed up for revenue that disappears.\n\n**Their BATNA:** Restart vendor selection, or build in-house. Both mean months of delay and a disappointed engineering org.\n\n**Concentration risk:** At 30% of ARR, this customer becomes a single point of failure. That's another reason the exit and liability terms matter more than the last $10k of price.\n\n---\n\n## 4. Strategy: Trade, Don't Concede\n\nTreat the four terms as **one package**. Never settle them one by one. If you agree on price first, they'll keep pushing on everything else.\n\n### Price\n- Re-anchor to list price. *\"Our list is $180k. For a 3-year commitment we offer a multi-year discount.\"* Open at $162k.\n- Tie each step down to something they give you:\n  - 3-year **committed** term with no exit for convenience \u2192 discount\n  - Annual **upfront** payment \u2192 discount\n  - Reference call or case study (banks rarely allow logos, but often allow references) \u2192 small discount\n- Aim to land at $140k. It fits their budget, so the champion can approve it without escalating.\n- **Things you can give that cost you little:** a price lock across all 3 years (no escalator), extra seats or usage headroom, a named support contact or SLA, onboarding and training, quarterly roadmap reviews. Offer these **instead of** cutting price.\n- If they insist on $108k, cut the scope: fewer seats, fewer teams, or no premium support. *\"At that number, here's what we can include.\"*\n\n### Payment terms\n- Ask for annual upfront, net-30. Fall back to net-45 or net-60.\n- If they truly can't move off net-90 (some banks have fixed AP policies), switch to **quarterly billing** so the gap stays small. Or ask for the price to reflect it: *\"Net-90 is a financing cost for us. We can do it at a slightly higher price.\"*\n- Ask whether they have an **early-payment program** (many banks do).\n\n### Liability (your hardest line)\n- *\"No vendor our size can sign uncapped liability. It would make us uninsurable, and a bank shouldn't want a critical vendor that one incident could wipe out.\"*\n- Propose:\n  - General cap: 12 months of fees\n  - Data-breach / confidentiality **super-cap**: 2\u20133x annual fees, or your cyber-insurance limit, whichever is higher\n  - Standard carve-outs that stay uncapped: gross negligence, willful misconduct, fraud. These are normal and fine to accept.\n- **Before Monday:** confirm your cyber-insurance limits, or get a quote. Banks often require a minimum such as $5M. Offering to raise your coverage is a strong concession that costs you a premium, not your company.\n- Clarify what data you actually touch. If you don't process customer PII or financial data, say so. It shrinks the risk they're pricing in.\n\n### Termination for convenience\n- *\"Our multi-year price assumes a multi-year commitment. If you need a flexible exit, that's our annual price.\"*\n- Options, from best to acceptable:\n  1. No termination for convenience. Termination for cause only, with a cure period.\n  2. Termination for convenience only after Year 1, with 90 days' notice.\n  3. Termination for convenience allowed, but they repay the multi-year discount (the difference from the 1-year price) for the time they used, or pay an early-termination fee.\n- Accept **termination for regulatory reasons** if they need it. Banks often truly do. Frame it as separate from convenience.\n\n---\n\n## 5. Before Monday (Thu\u2013Sun)\n\n1. **Call the champion.** Ask:\n   - Who will be in the room, and who actually signs?\n   - What does procurement get measured on (savings %, a quarter-end deadline)?\n   - Are any of the four terms bank-mandated policy, and which are just opening positions?\n   - Would they help push internally on liability and exit terms?\n   - **Keep their earlier disclosures confidential.** Don't make them look like the leak.\n2. **Get a lawyer to look at the MSA** for an hour, especially liability and indemnity. It's worth the cost on a $420k contract.\n3. **Check your cyber-insurance** limits, or get a quote.\n4. **Run a cash-flow model** for net-90 vs. net-30, annual vs. quarterly billing.\n5. **Write down your walk-away terms** and commit to them before you walk in.\n6. **Prepare a redline or term sheet** you can send right after the call.\n\n---\n\n## 6. Questions to Ask Them in the Meeting\n\n- \"Help me understand what's driving the 40% figure. Is it a policy, or a target for this deal?\"\n- \"Is net-90 an AP policy, or open for discussion?\"\n- \"What scenario is the termination-for-convenience clause meant to protect against?\" (Often the answer is regulatory change, which you can handle more narrowly.)\n- \"What timeline are you working toward for signature?\"\n- \"Beyond price, what would make this an easy approval for you?\"\n\n---\n\n## 7. Scripts for Likely Pushback\n\n**\"Your competitor is cheaper.\"**\n> \"Pricing is one part of it. We passed your security review and your team has run us in production for four months. Let's make sure the terms reflect that.\"\n(Don't reveal that you know the competitor failed security.)\n\n**\"40% is our standard enterprise discount.\"**\n> \"I understand. Our multi-year pricing is built on commitment and payment terms. If we can lock in a 3-year committed term with annual payment, I can move meaningfully. Let me show you what that looks like.\"\n\n**\"Unlimited liability is non-negotiable.\"**\n> \"I respect that. Then let's find out what's really driving it. If it's data-breach exposure, we can put a meaningful super-cap backed by insurance. Uncapped liability isn't something any company our size can sign, and your risk team wouldn't want a vendor that one incident could put out of business.\"\n\n**\"We need to close this today at our number.\"**\n> \"I want to close this too. I can't agree to all four terms as written, but if we can agree on [price/exit/liability/payment], I'll have paper to you within 48 hours.\"\n\n**Long silence or \"that's our final offer.\"**\n> Don't fill the silence. Then: \"Let me take that back and come back to you tomorrow.\" You don't have to decide in the room.\n\n---\n\n## 8. Concession Plan (decide this now, not in the room)\n\n| Step | You give | You get |\n|---|---|---|\n| 1 | $180k \u2192 $162k | 3-yr committed term |\n| 2 | $162k \u2192 $150k | Annual upfront, net-45 or better |\n| 3 | $150k \u2192 $140k | No termination for convenience in Year 1, plus a reference customer |\n| 4 (only if needed) | Price lock, extra seats, premium support | Liability super-cap accepted |\n| Floor | ~$125k | Only with all terms favorable. Below that, reduce scope instead |\n\nMake each step **smaller** than the one before. That tells them you're approaching your limit.\n\n---\n\n## 9. A Final Reality Check\n\nAt 30% of ARR and 14 months of runway, the pressure to \"just get it signed\" will be strong. But procurement's opening was designed to make you negotiate against yourself. **The money is approved, the champion wants you, and the alternative failed.** You're in a better position than the first email suggests. The two terms that could actually hurt the company are **unlimited liability** and **cheap termination for convenience**. Protect those, land close to $140k, and this is a very good deal.\n\n*This is negotiation prep, not legal advice. Have counsel review final contract language, especially the liability and indemnity sections.*\n"}]}, "opus-2": {"cost_usd": 0.23416, "seconds": 96, "output_tokens": 7989, "files": [{"name": "negotiation-prep.md", "text": "# Negotiation Prep: 3-Year Enterprise Contract with the Bank\n\n*Prepared 2026-10-08 for Monday's negotiation. Built from deal.md.*\n\n---\n\n## 1. One-page cheat sheet (bring this into the room)\n\n| Term | Their ask | Open with | Target | Walk-away line |\n|---|---|---|---|---|\n| **Price / yr** | $108k (40% off $180k) | $162k (10% off, for the 3-yr commitment) | **$140\u2013150k** | Below ~$125k *only* if the other terms are clean. Never $108k. |\n| **Payment** | Net-90 | Annual upfront, net-30 | Annual upfront, net-45/60 | Net-90 only in exchange for real price. Never quarterly/monthly in arrears plus net-90. |\n| **Liability (data breach)** | Unlimited | Mutual cap = 12 months' fees | Super-cap for data breach at 2\u20133x annual fees (or our cyber-insurance limit, e.g. $1\u20132M) | **Never unlimited.** Usual carve-outs (gross negligence, wilful misconduct) are OK. |\n| **Termination for convenience** | 30 days, any time | None. 3-yr commitment is why there's a discount. | Exit only at each anniversary, 90 days' notice, and they repay the multi-year discount | If 30-day TfC stays, it's really a 1-year deal, so charge 1-year pricing. |\n\n**Three rules for Monday:**\n1. **Never give something without getting something back.** Use \"If you can do X, I can do Y.\"\n2. **Discuss all four terms together.** Don't settle price first, because then they'll squeeze you on liability and termination with nothing left to trade.\n3. **Never reveal what the champion told you** (the $140k budget, the competitor failing security). Using it openly burns your champion. Let it guide your numbers quietly.\n\n---\n\n## 2. Reading the situation\n\n### Your leverage (stronger than it feels)\n- **The product is proven inside the bank.** A 4-month pilot that engineering loves means switching costs, political investment, and a team that will push back if procurement kills the deal.\n- **Budget is approved at $140k/yr.** Their own number is $32k/yr *above* their ask. The 40% discount is a procurement opening, not a budget limit.\n- **You may be the only real option.** If the competitor failed security review, procurement's alternatives are (a) you, (b) restart the search (months of delay and another security review), or (c) build it in-house.\n- **Procurement gets measured on \"savings vs. first quote.\"** They'll push hard, but they also need a signed deal. Leaving with nothing is a failure for them too.\n\n### Your weaknesses (be honest with yourself, never show them)\n- **14 months of runway** and a deal worth **30% of ARR**. You need this more than they need you in absolute terms. This is the main thing that could make you concede badly.\n- **12 people.** Unlimited liability or a concentration-heavy contract could kill the company. A bank can absorb a vendor failure. You can't absorb a breach claim.\n- **Information risk.** Everything you \"know\" comes from one friendly source. Budgets get reallocated, and \"failed security review\" can turn into \"fixed it and resubmitted.\"\n\n### What the deal is really worth over 3 years (TCV)\n| Scenario | Annual | 3-yr TCV | vs. their ask |\n|---|---|---|---|\n| Their ask (40% off) | $108k | $324k | \u2014 |\n| Your floor-ish | $125k | $375k | +$51k |\n| Their approved budget | $140k | $420k | +$96k |\n| Your target top | $150k | $450k | +$126k |\n| Your open | $162k | $486k | +$162k |\n\nGetting from their ask to their own budget is worth **~$96k** over three years, which is close to a senior engineer's salary for a year at a 12-person company. The 30-day termination clause can wipe out most of this TCV, so in practice it matters as much as the price does.\n\n> Note: deal.md says the deal is \"30% of ARR\" but doesn't say at what price. If that's at $140k, your current ARR is about $330k (pre-deal) or $467k (post-deal). Check which one before Monday, because it changes how much a lower price hurts.\n\n---\n\n## 3. Term-by-term playbook\n\n### 3.1 Price (40% discount)\n**Their position:** $108k/yr.\n**Reality:** The approved budget is $140k. Procurement opens low on purpose.\n\n**Opening move:** Don't reply with a single number. Give **2\u20133 packages at once** (see \u00a74). This shows flexibility and makes them compare your options against each other, not against their 40% ask.\n\n**Lines to use:**\n- \"Our list is $180k. For a 3-year committed term, we can do a meaningful multi-year discount, but it's tied to the commitment being real.\"\n- \"40% is beyond what we do for any customer. It would also set a reference price we couldn't honour elsewhere.\"\n- \"Help me understand what's driving the 40%. Is it a budget ceiling or a target?\" *(Asking lets them reveal the budget without you exposing the champion.)*\n\n**Concession plan (go down in shrinking steps, each one traded):**\n$162k \u2192 $155k \u2192 $148k \u2192 $142k \u2192 (hold at ~$140k)\nEach step should buy something: dropping TfC, better payment terms, a reference/case study, a price lock, or more seats/scope.\n\n**Things to ask for in return for price movement:**\n- Annual payment upfront\n- Case study / logo rights / reference calls (very valuable to a startup selling to other banks)\n- Multi-year price lock in exchange for a capped annual uplift (e.g. 5\u20137% in years 2\u20133), or a ramp: $135k / $145k / $155k (avg $145k, and year 1 fits inside their $140k budget)\n- Expansion pricing pre-agreed for other teams/business units\n\n### 3.2 Payment terms (net-90)\n**Why it matters to you:** With 14 months of runway, cash timing matters. Net-90 on a $140k invoice means you fund the bank for a quarter. If they *also* want quarterly billing in arrears, you'd be delivering about 6 months of service before seeing the first dollar.\n\n**Open:** Annual in advance, net-30.\n**Fall back:** Annual in advance, net-45 or net-60.\n**Last resort:** Net-90 is acceptable *only if* billing is annual in advance and you got something real for it (price or TfC removal).\n\n**Lines:**\n- \"Net-90 is often a standard policy for banks, and I understand that. Can we invoice annually in advance so the policy still works for both sides?\"\n- \"If net-90 is fixed, then we need annual invoicing on signature.\"\n\n**Check:** Many banks have net-90 as a default but allow shorter terms for small/strategic suppliers (\"small supplier\" or \"diverse supplier\" programs). Ask: \"Do you have an accelerated payment program for small vendors?\"\n\n### 3.3 Unlimited liability for data breaches. **The most important item.**\n**Why it's existential:** One breach claim from a large bank could be bigger than everything the company will ever earn. Unlimited liability means the bank can take the company. No investor wants this on the books, and it may cause problems in diligence for your next round.\n\n**What is market-standard (so you're not being unreasonable):**\n- General liability cap: 12 months' fees.\n- Data-breach / confidentiality **\"super-cap\"**: commonly 2\u20135x annual fees, or a fixed figure linked to the vendor's cyber insurance.\n- Uncapped carve-outs that are usually accepted: gross negligence, wilful misconduct, fraud, and sometimes IP indemnity. These are fine to accept.\n\n**Open:** Mutual cap at 12 months' fees, including data breach.\n**Target:** Data-breach super-cap of 2\u20133x annual fees *or* equal to your cyber-insurance limit (e.g. $1\u20132M), whichever is higher.\n**Never:** Unlimited.\n\n**Lines:**\n- \"We take security seriously. You've already seen that in your review. But no company our size can sign unlimited liability. It isn't a negotiating position. The company literally can't carry that risk.\"\n- \"What we can do: a dedicated super-cap for data breaches, backed by cyber insurance, so you have a real, collectable recovery. A $X million insured cap is worth more to you than an unlimited claim against a 12-person company.\"\n- \"What data does our product actually touch in your environment?\" If the tool runs in their infrastructure, sees no customer PII, etc., the realistic breach exposure is small. Make this point clearly.\n\n**Before Monday:** Call your broker. Find out your current cyber coverage, and what $2M / $5M coverage would cost. If the bank insists on a higher cap, the premium increase can be priced into the deal.\n\n### 3.4 Termination for convenience (30 days)\n**Why it matters:** A 3-year deal with 30-day termination for convenience is really a month-to-month contract with a 40% discount. They get the multi-year discount without the multi-year commitment. You can't count it as committed revenue in a fundraise either.\n\n**Open:** No TfC. Termination for cause only (material breach plus cure period).\n**Target (pick one):**\n- TfC only at each annual anniversary, with 90 days' notice, **or**\n- TfC any time with 90 days' notice **and** an early-termination fee / clawback of the multi-year discount (they pay the difference between what they paid and the 1-year price), **or**\n- No refund of prepaid annual fees on TfC.\n\n**Lines:**\n- \"The multi-year discount is in exchange for a multi-year commitment. If you need a 30-day exit, that's fine. That's our 1-year pricing.\"\n- \"What's the concern behind the termination right? If it's performance, let's write SLAs with service credits and a termination right if we miss them repeatedly. That solves your problem without making the commitment meaningless.\"\n\nThe last line is often the key: procurement usually wants TfC as **risk protection**, and SLA-based exit rights give them that.\n\n---\n\n## 4. Packages to put on the table (multiple equivalent offers)\n\nPresent these together, as \"three ways we can structure this\":\n\n| | **Option A: Committed** | **Option B: Flexible** | **Option C: 1-year** |\n|---|---|---|---|\n| Term | 3 years, committed | 3 years | 1 year, renewable |\n| Price / yr | **$145k** (~20% off) | **$158k** (~12% off) | **$171k** (5% off) |\n| Payment | Annual upfront, net-30 | Annual upfront, net-60 | Annual upfront, net-90 |\n| Termination | For cause only (with SLA exit rights) | At each anniversary, 90 days' notice | 30 days' notice, no refund of prepaid fees |\n| Liability | 12-mo general cap; data-breach super-cap 2x fees | Same | Same |\n| Extras | Price lock 3 yrs; logo/case study | 5% annual uplift cap | \u2014 |\n\nWhy this works: Option A is near their budget and is clearly the best value, so procurement can call it a \"win\" (20% \"savings\"). The liability position is **the same in every option**. That shows it's a principle, not a bargaining chip.\n\nExpect them to try to combine the best parts (\"Option A price with Option C termination\"). Response: \"Each option is a package. Moving one piece changes the others.\"\n\n---\n\n## 5. Your BATNA and walk-away point\n\n**Your best alternative if no deal:** other pipeline, raising on the strength of the pilot (a 4-month pilot with a large bank is valuable even without a contract), or a smaller department-level deal later.\n\nImprove your BATNA **before Monday**: do you have even one other enterprise conversation you can (truthfully) mention as \"competing for onboarding capacity\"?\n\n**Walk away (or pause and escalate) if:**\n- They won't move off **unlimited liability**. This is the one real deal-breaker.\n- 30-day TfC **plus** a 40% discount **plus** net-90 all stay. That is a cancellable deal at a loss with cash-flow risk.\n\n**Remember:** a bad deal that's 30% of ARR is also a **concentration risk**. If they terminate in month 4 under a 30-day clause, you've changed hiring and burn around revenue that disappears. Don't let runway pressure persuade you that \"any deal beats no deal.\"\n\n\"Walking away\" doesn't have to mean ending things. It can be: \"We're not able to sign this as written. Let me take it back to my board/counsel and come back with what we can do.\" Then go to your champion.\n\n---\n\n## 6. Using your champion (carefully)\n\nBefore Monday, talk to the Director of Platform:\n- \"Procurement came in at 40% off, unlimited liability, and 30-day termination. Is this their standard opening, or a sign there's a problem?\"\n- \"Who on their side has authority to approve exceptions to the liability and termination terms? Legal? Risk? The CISO?\"\n- \"If procurement drags, could you remind them how much the team depends on this, and what restarting a search would cost in time?\"\n- \"Is there a deadline on their end (fiscal year-end, budget expiry, a project launch) that makes signing soon important to them?\" *Their deadline is your leverage.*\n\n**Don't** ask the champion to leak more than they're comfortable with, and **never** quote them to procurement.\n\n---\n\n## 7. Likely pushbacks and responses\n\n| They say | You say |\n|---|---|\n| \"40% is our standard for all vendors.\" | \"I understand. Our multi-year discount is 15\u201320% for a committed term, and that's the most we do for anyone. Let me show you some options.\" |\n| \"Your competitor is cheaper.\" | \"Price is one factor. You've done a 4-month pilot and a security review with us. What would switching cost your team in time?\" *(Don't reveal you know they failed security.)* |\n| \"Unlimited liability is non-negotiable for us.\" | \"Who sets that policy? We've seen banks accept insured super-caps for vendors our size. Can we get your legal/risk team on a call?\" Escalate to legal. Procurement often doesn't own this. |\n| \"We need TfC for regulatory reasons.\" | \"Which regulation? Usually regulators require exit rights for *cause*, data return, and step-in, and we can give all of those. Let's write them in.\" *(Banking outsourcing rules typically require exit plans, not convenience termination at 30 days.)* |\n| \"Take it or leave it.\" | \"I want to make this work. Your team clearly wants it. I can't sign unlimited liability at any price. Can we find a structure that works on the other three?\" |\n| \"We need an answer today.\" | \"I'll get back to you by [specific day]. I want to make sure I can say yes to something we'll both stick to.\" Never decide under time pressure in the room. |\n| Silence after your number. | Stay silent. Don't fill the gap with a concession. |\n\n---\n\n## 8. Prep checklist before Monday\n\n- [ ] **Lawyer:** 30-min call with a SaaS/commercial lawyer on the liability cap and TfC language. Get fallback clause text ready. *(This prep is not legal advice. The liability clause in particular needs counsel.)*\n- [ ] **Insurance:** current cyber/tech E&O limits; cost quotes for $2M and $5M.\n- [ ] **Cash model:** model runway under (a) annual upfront net-30, (b) net-90, (c) quarterly in arrears + net-90, and (d) the deal terminated at month 4.\n- [ ] **ARR check:** confirm what \"30% of ARR\" means at each price point.\n- [ ] **Champion call** (see \u00a76).\n- [ ] **Security pack:** SOC 2 / pen test report / data-flow diagram showing what data you touch. This backs up the liability argument.\n- [ ] **Decide your numbers in writing now**: opening, target, walk-away for each term. Don't change them in the room.\n- [ ] **Know who's in the room on their side** (procurement only? legal? the champion?). Ask for legal/risk to attend if liability is going to be discussed.\n- [ ] **Bring a second person** if you can (co-founder/advisor) to take notes and call a break if needed.\n\n---\n\n## 9. Opening script (about 2 minutes)\n\n> \"Thanks for sending the terms through. Before we get into them, I want to say we're really glad about how the pilot went. Your platform team has been great to work with, and we want this to be a long-term partnership.\n>\n> I've looked at the four points: pricing, payment, liability, and termination. I'd like to talk about them together, because they're connected. The discount we can offer depends on the commitment and the payment structure.\n>\n> There's one point where I need to be clear right away: as a company of our size, we can't accept unlimited liability. That's not a tactic. It's a risk no company our size can carry. But we have a proposal that gives you real, insured protection, which I think serves you better.\n>\n> On the rest, I've put together three ways we could structure this...\"\n\nThen present \u00a74.\n\n---\n\n## 10. After the meeting\n- Send a written summary of what was agreed and what's still open within 24 hours. Whoever writes the summary shapes the contract.\n- Don't agree to final terms in the room. \"Agreed in principle, subject to contract\" is fine.\n- Update the champion.\n"}]}}, "verdicts": [{"pair": "1", "judge": "gpt", "order": "haiku shown first", "winner": "haiku", "why": "A handles leverage uncertainty and contractual protections more responsibly, despite incorrect net-90 arithmetic and an unsupported price floor. B invents security clearance, overstates alternatives and budget authority, recommends questionable cash terms, and casually accepts uncapped liability carve-outs.", "haiku_scores": {"c1": 0, "c2": 1, "c3": 2, "c4": 1, "c5": 1, "c6": 0}, "opus_scores": {"c1": 1, "c2": 0, "c3": 1, "c4": 1, "c5": 1, "c6": 0}}, {"pair": "1", "judge": "gpt", "order": "opus shown first", "winner": "opus", "why": "A narrowly wins on accurate pricing and conditional concessions. B incorrectly quantifies delayed cash and infers current ARR. A overstates leverage. Both introduce unsupported $125k floors, incompletely protect termination economics, and lack realistic fallback plans.", "haiku_scores": {"c1": 0, "c2": 1, "c3": 1, "c4": 1, "c5": 1, "c6": 0}, "opus_scores": {"c1": 2, "c2": 0, "c3": 1, "c4": 1, "c5": 1, "c6": 0}}, {"pair": "1", "judge": "deepseek", "order": "haiku shown first", "winner": "tie", "why": "Both accurately use facts, assess leverage, address risks, propose packages, equip for conversation, and set boundaries.", "haiku_scores": {"c1": 2, "c2": 2, "c3": 2, "c4": 2, "c5": 2, "c6": 2}, "opus_scores": {"c1": 2, "c2": 2, "c3": 2, "c4": 2, "c5": 2, "c6": 2}}, {"pair": "1", "judge": "deepseek", "order": "opus shown first", "winner": "tie", "why": "Both accurately use facts, assess leverage, address risks, propose packages, equip for conversation, and set boundaries.", "haiku_scores": {"c1": 2, "c2": 2, "c3": 2, "c4": 2, "c5": 2, "c6": 2}, "opus_scores": {"c1": 2, "c2": 2, "c3": 2, "c4": 2, "c5": 2, "c6": 2}}, {"pair": "2", "judge": "gpt", "order": "haiku shown first", "winner": "tie", "why": "A assesses leverage better but miscalculates annual-invoice cash timing and muddles concessions. B offers clearer packages but invents fallback financing and pricing policies. Both impose unsupported price floors and inadequately constrain liability carve-outs.", "haiku_scores": {"c1": 0, "c2": 2, "c3": 1, "c4": 1, "c5": 1, "c6": 0}, "opus_scores": {"c1": 1, "c2": 1, "c3": 1, "c4": 1, "c5": 1, "c6": 0}}, {"pair": "2", "judge": "gpt", "order": "opus shown first", "winner": "opus", "why": "A offers more coherent conditional packages and usable scripts. B misstates payment timing economics, reverses several give/get trades, and contradicts its payment boundary. Both invent price floors and too readily accept uncapped liability carve-outs.", "haiku_scores": {"c1": 0, "c2": 2, "c3": 1, "c4": 0, "c5": 1, "c6": 0}, "opus_scores": {"c1": 1, "c2": 1, "c3": 1, "c4": 2, "c5": 1, "c6": 0}}, {"pair": "2", "judge": "deepseek", "order": "haiku shown first", "winner": "tie", "why": "Both use facts accurately, assess leverage, prioritize risks, propose packages, equip for conversation, and set decision boundaries effectively.", "haiku_scores": {"c1": 2, "c2": 2, "c3": 2, "c4": 2, "c5": 2, "c6": 2}, "opus_scores": {"c1": 2, "c2": 2, "c3": 2, "c4": 2, "c5": 2, "c6": 2}}, {"pair": "2", "judge": "deepseek", "order": "opus shown first", "winner": "tie", "why": "Both accurately use facts, assess leverage, prioritize risks, propose packages, equip for conversation, and set decision boundaries.", "haiku_scores": {"c1": 2, "c2": 2, "c3": 2, "c4": 2, "c5": 2, "c6": 2}, "opus_scores": {"c1": 2, "c2": 2, "c3": 2, "c4": 2, "c5": 2, "c6": 2}}]}, {"id": "safe-1", "family": "safe-math", "prompt": "Work out what each founder will own after our Series A, and how much more they'd own if SAFE B had been raised at a $15M cap instead. Numbers in raise.md.", "input": "raise.md", "input_text": "Founders: 2, with 5,000,000 common shares each. No option pool.\nSAFE A: USD 500,000, post-money SAFE, valuation cap USD 5,000,000, no discount.\nSAFE B: USD 1,000,000, post-money SAFE, valuation cap USD 10,000,000, no discount.\nSeries A term sheet: USD 4,000,000 new money at a USD 16,000,000 pre-money valuation. No new option pool. The SAFEs convert in this round.\n", "rubric": [{"id": "c1", "criterion": "Calculate each founder\u2019s ownership after the Series A under the original terms.", "full_marks_when": "States that each founder owns 32% after the round, with the two founders together owning 64%.", "zero_when": "Gives an incorrect percentage or reports only ownership before the Series A."}, {"id": "c2", "criterion": "Calculate each founder\u2019s ownership when SAFE B\u2019s cap is increased to $15M.", "full_marks_when": "States that each founder owns approximately 33.3333% after the Series A, with the founders together owning approximately 66.6667%.", "zero_when": "Gives an incorrect percentage or does not calculate the alternative scenario."}, {"id": "c3", "criterion": "Quantify how much more each founder would own in the alternative scenario.", "full_marks_when": "Clearly states an increase of approximately 1.3333 percentage points per founder, from 32% to 33.3333%; any relative increase reported is correctly identified as approximately 4.1667%.", "zero_when": "Omits the difference, calculates it incorrectly, or confuses percentage points with relative percentage growth."}, {"id": "c4", "criterion": "Account correctly for both SAFE conversion and Series A dilution.", "full_marks_when": "Shows that before Series A dilution, SAFE A owns 10% and SAFE B owns 10% originally or 6.6667% alternatively; founders retain 80% or 83.3333% collectively. The Series A investors then receive 20%, multiplying all existing holders\u2019 ownership by 80%.", "zero_when": "Treats the SAFEs as pre-money instruments, applies their ownership percentages sequentially, or calculates Series A dilution using an incorrect valuation basis."}, {"id": "c5", "criterion": "Keep the calculation faithful to the input and arithmetically consistent.", "full_marks_when": "Uses 5,000,000 shares per founder, $500,000 for SAFE A at a $5M cap, $1M for SAFE B at a $10M or $15M cap, and $4M raised at $16M pre-money, with no discount or option pool. Final ownership reconciles to 100%: originally 32%, 32%, 8%, 8%, 20%; alternatively approximately 33.3333%, 33.3333%, 8%, 5.3333%, 20% for the founders, SAFE A, SAFE B, and Series A investors respectively.", "zero_when": "Changes an input besides SAFE B\u2019s cap, invents option-pool dilution or a discount, or makes material factual or arithmetic errors against these figures."}, {"id": "c6", "criterion": "Present a usable comparison that directly answers the founders\u2019 question.", "full_marks_when": "Provides a clearly labeled side-by-side comparison or equally clear prose showing each founder\u2019s final ownership in both scenarios and the increase, with enough calculation detail to verify the results and rounding that preserves the difference.", "zero_when": "Leaves the user to derive the requested ownership or increase, fails to distinguish the scenarios, or presents unexplained numbers whose timing or meaning is ambiguous."}], "runs": {"haiku-1": {"cost_usd": 0.00555, "seconds": 23, "output_tokens": 5386, "files": [{"name": "ownership.md", "text": "# Post-Series A ownership\n\nSource: raise.md\n\n## Assumptions\n\n- Both SAFEs are post-money. Each SAFE's ownership is its investment divided by its post-money cap, measured on capitalization that includes all SAFEs and excludes the Series A new money.\n- The $16M Series A pre-money includes the shares issued on SAFE conversion. No option pool top-up.\n- Both SAFEs convert at their cap, since each cap price is below the Series A price.\n\n## Base case: SAFE B at $10M cap\n\n- SAFE A: 500k / 5M = 10% pre-A\n- SAFE B: 1M / 10M = 10% pre-A\n- Founders: 80% pre-A, so pre-money capitalization is 12.5M shares\n- Series A price: $16M / 12.5M = $1.28 per share\n- Series A shares: $4M / $1.28 = 3,125,000\n\n| Holder | Shares | Post-A % |\n|---|---|---|\n| Founder 1 | 5,000,000 | 32.00% |\n| Founder 2 | 5,000,000 | 32.00% |\n| SAFE A | 1,250,000 | 8.00% |\n| SAFE B | 1,250,000 | 8.00% |\n| Series A | 3,125,000 | 20.00% |\n| **Total** | **15,625,000** | **100%** |\n\n## Alternative: SAFE B at $15M cap\n\n- SAFE A: 500k / 5M = 10% pre-A\n- SAFE B: 1M / 15M = 6.67% pre-A\n- Founders: 83.33% pre-A, so pre-money capitalization is 12M shares\n- SAFE A shares: 1,200,000; SAFE B shares: 800,000\n- SAFE B conversion price: $15M / 12M = $1.25, below the Series A price\n- Series A price: $16M / 12M = $1.3333 per share\n- Series A shares: $4M / $1.3333 = 3,000,000\n\n| Holder | Shares | Post-A % |\n|---|---|---|\n| Founder 1 | 5,000,000 | 33.33% |\n| Founder 2 | 5,000,000 | 33.33% |\n| SAFE A | 1,200,000 | 8.00% |\n| SAFE B | 800,000 | 5.33% |\n| Series A | 3,000,000 | 20.00% |\n| **Total** | **15,000,000** | **100%** |\n\n## Difference ($15M cap vs $10M cap)\n\n| Holder | $10M cap | $15M cap | Change |\n|---|---|---|---|\n| Founder 1 (each) | 32.00% | 33.33% | +1.33 pp |\n| Founder 2 (each) | 32.00% | 33.33% | +1.33 pp |\n| Founders combined | 64.00% | 66.67% | +2.67 pp |\n| SAFE A | 8.00% | 8.00% | 0 |\n| SAFE B | 8.00% | 5.33% | -2.67 pp |\n| Series A | 20.00% | 20.00% | 0 |\n\nEach founder ends up about 4.2% more owned in relative terms (33.33 / 32.00). The founders' 2.67 pp gain comes entirely out of SAFE B.\n"}]}, "haiku-2": {"cost_usd": 0.00596, "seconds": 26, "output_tokens": 6039, "files": [{"name": "ownership.md", "text": "# Post-Series A ownership: SAFE B at $10M cap vs. $15M cap\n\nSource: `raise.md`\n\n## Method\n\n- Post-money SAFE: each SAFE owns investment / cap of the pre-Series A company capitalization (founders + all SAFE conversion shares; no option pool; excludes Series A new money).\n- Series A pre-money ($16M) includes the SAFE conversion shares.\n- Both SAFEs convert at their cap price, since each cap is below the Series A price.\n\n## Base case: SAFE B cap $10M (as in raise.md)\n\n- SAFE A: 500k / 5M = 10% of pre-A capitalization\n- SAFE B: 1M / 10M = 10% of pre-A capitalization\n- Founders: remaining 80%\n- Pre-A capitalization: 10,000,000 / 0.80 = **12,500,000 shares**\n- SAFE A converts to 1,250,000 shares (cap price $0.40)\n- SAFE B converts to 1,250,000 shares (cap price $0.80)\n- Series A price: $16M / 12.5M = $1.28 per share\n- Series A new shares: $4M / $1.28 = 3,125,000\n- Post-money: 15,625,000 shares; $20M post-money valuation\n\n| Holder | Shares | Post-A ownership | Value at $1.28 |\n|---|---|---|---|\n| Founder 1 | 5,000,000 | 32.00% | $6,400,000 |\n| Founder 2 | 5,000,000 | 32.00% | $6,400,000 |\n| SAFE A | 1,250,000 | 8.00% | $1,600,000 |\n| SAFE B | 1,250,000 | 8.00% | $1,600,000 |\n| Series A | 3,125,000 | 20.00% | $4,000,000 |\n| **Total** | **15,625,000** | **100%** | **$20,000,000** |\n\n## Counterfactual: SAFE B cap $15M\n\n- SAFE A: 10% of pre-A capitalization (unchanged)\n- SAFE B: 1M / 15M = 6.667% of pre-A capitalization\n- Founders: remaining 83.333%\n- Pre-A capitalization: 10,000,000 / (10/12) = **12,000,000 shares**\n- SAFE A converts to 1,200,000 shares (cap price $0.4167)\n- SAFE B converts to 800,000 shares (cap price $1.25)\n- Series A price: $16M / 12M = $1.3333 per share\n- Series A new shares: $4M / $1.3333 = 3,000,000\n- Post-money: 15,000,000 shares; $20M post-money valuation\n\n| Holder | Shares | Post-A ownership | Value at $1.3333 |\n|---|---|---|---|\n| Founder 1 | 5,000,000 | 33.33% | $6,666,667 |\n| Founder 2 | 5,000,000 | 33.33% | $6,666,667 |\n| SAFE A | 1,200,000 | 8.00% | $1,600,000 |\n| SAFE B | 800,000 | 5.33% | $1,066,667 |\n| Series A | 3,000,000 | 20.00% | $4,000,000 |\n| **Total** | **15,000,000** | **100%** | **$20,000,000** |\n\n## Comparison\n\n| Holder | $10M cap | $15M cap | Change |\n|---|---|---|---|\n| Founder 1 (each) | 32.00% | 33.33% | +1.33 pts (+$266,667 at Series A price) |\n| Founder 2 (each) | 32.00% | 33.33% | +1.33 pts (+$266,667 at Series A price) |\n| Both founders | 64.00% | 66.67% | +2.67 pts |\n| SAFE B | 8.00% | 5.33% | \u22122.67 pts |\n| SAFE A | 8.00% | 8.00% | none |\n| Series A | 20.00% | 20.00% | none |\n\nEach founder would own about 1.33 percentage points more (about 4% more in relative terms) with a $15M SAFE B cap. Founders still hold 5,000,000 shares each. Their percentage rises because the total share count falls from 15.625M to 15.0M.\n\n## Assumptions\n\n- No option pool, and none added in the Series A.\n- No other securities, transaction fees, or dilution beyond what raise.md lists.\n- Values are paper values at the Series A price.\n"}]}, "opus-1": {"cost_usd": 0.1382, "seconds": 45, "output_tokens": 3756, "files": [{"name": "founder_ownership.md", "text": "# Founder ownership after the Series A\n\nInputs come from `raise.md`. To reproduce the numbers, run `python3 cap_table.py`.\n\n## Answer\n\n| | SAFE B cap $10M (as raised) | SAFE B cap $15M | Difference |\n|---|---|---|---|\n| Each founder | **32.00%** (5,000,000 sh) | **33.33%** (5,000,000 sh) | **+1.33 pp** (+4.2% relative) |\n| Both founders | 64.00% | 66.67% | +2.67 pp |\n\n## Method (post-money SAFEs)\n\n1. Each post-money SAFE owns `investment / cap` of the company's capitalization just before the Series A. That capitalization counts all SAFEs as converted.\n   - SAFE A: 0.5M / 5M = 10%\n   - SAFE B: 1M / 10M = 10% (or 1M / 15M = 6.67% with the $15M cap)\n2. The founders hold the rest: 80% (or 83.33%). That share equals 10,000,000 shares, so the pre-money share count is 12,500,000 (or 12,000,000).\n3. Series A price = $16M pre-money / pre-money shares. That gives $1.28 (or $1.3333). Both SAFE cap prices are below this price, so the caps apply. SAFE A converts at $0.40 (or $0.4167). SAFE B converts at $0.80 (or $1.25).\n4. Series A shares = $4M / price, which is 3,125,000 (or 3,000,000). Series A investors own 20% of the $20M post-money company.\n\n## Cap tables\n\n**As raised (SAFE B cap $10M)**: price $1.28\n\n| Holder | Shares | % |\n|---|---:|---:|\n| Founder 1 | 5,000,000 | 32.00% |\n| Founder 2 | 5,000,000 | 32.00% |\n| SAFE A | 1,250,000 | 8.00% |\n| SAFE B | 1,250,000 | 8.00% |\n| Series A | 3,125,000 | 20.00% |\n| **Total** | **15,625,000** | 100% |\n\n**Counterfactual (SAFE B cap $15M)**: price $1.3333\n\n| Holder | Shares | % |\n|---|---:|---:|\n| Founder 1 | 5,000,000 | 33.33% |\n| Founder 2 | 5,000,000 | 33.33% |\n| SAFE A | 1,200,000 | 8.00% |\n| SAFE B | 800,000 | 5.33% |\n| Series A | 3,000,000 | 20.00% |\n| **Total** | **15,000,000** | 100% |\n\n## Assumptions\n\n- The SAFEs follow the standard YC post-money SAFE. The Series A pre-money valuation includes the converting SAFEs, which is the usual way these terms are read. Under this reading, the dilution from the SAFEs falls only on the founders. The Series A investors get exactly 20% either way.\n- The share counts use exact prices. A real closing would round the price per share, for example to $1.3333, so share counts could differ by up to about a hundred shares. Percentages would not change meaningfully.\n"}]}, "opus-2": {"cost_usd": 0.12536, "seconds": 41, "output_tokens": 3328, "files": [{"name": "founder_ownership.md", "text": "# Founder ownership after Series A\n\nInputs are from `raise.md`. To regenerate the tables, run `python3 cap_table.py`.\n\n## Answer\n\n| | SAFE B cap $10M (as raised) | SAFE B cap $15M | Difference |\n|---|---:|---:|---:|\n| Each founder | **32.00%** | **33.33%** | **+1.33 pp** (+4.17% relative) |\n| Both founders combined | 64.00% | 66.67% | +2.67 pp |\n\n## Method and assumptions\n\n- I treated the notes as standard YC post-money SAFEs. Each SAFE converts into (investment \u00f7 cap) of the company capitalization just before the Series A, and that capitalization includes the shares from all converting SAFEs.\n  - SAFE A: $0.5M / $5M = 10%.\n  - SAFE B: $1M / $10M = 10%. At a $15M cap it would be 6.67%.\n- I assumed the $16M Series A pre-money is fully diluted and includes the SAFE conversion shares. This is the usual convention, so the SAFEs dilute the founders and not the Series A investors.\n- The Series A takes $4M / $20M post = 20% in both scenarios.\n- Both SAFEs convert at their caps in both scenarios, because each SAFE's price is below the Series A price. The script checks this.\n- No option pool and no discounts. Shares are shown unrounded; real conversions would round down to whole shares.\n\n## Cap tables\n\n### As raised (SAFE B cap $10,000,000)\n\nSeries A price per share: $1.2800 (pre-money $16,000,000 / 12,500,000 shares)\n\n| Holder | Shares | Post-Series A % |\n|---|---:|---:|\n| Founder 1 | 5,000,000 | 32.00% |\n| Founder 2 | 5,000,000 | 32.00% |\n| SAFE A | 1,250,000 | 8.00% |\n| SAFE B | 1,250,000 | 8.00% |\n| Series A investors | 3,125,000 | 20.00% |\n| **Total** | **15,625,000** | **100.00%** |\n\n\n### Counterfactual (SAFE B cap $15,000,000)\n\nSeries A price per share: $1.3333 (pre-money $16,000,000 / 12,000,000 shares)\n\n| Holder | Shares | Post-Series A % |\n|---|---:|---:|\n| Founder 1 | 5,000,000 | 33.33% |\n| Founder 2 | 5,000,000 | 33.33% |\n| SAFE A | 1,200,000 | 8.00% |\n| SAFE B | 800,000 | 5.33% |\n| Series A investors | 3,000,000 | 20.00% |\n| **Total** | **15,000,000** | **100.00%** |\n\n"}]}}, "verdicts": [{"pair": "1", "judge": "gpt", "order": "haiku shown first", "winner": "tie", "why": "Both correctly calculate 32% versus approximately 33.3333% per founder, a 1.3333 percentage-point gain, and account for SAFE conversion and 20% Series A dilution. Share counts reconcile exactly; displayed percentages and relative gains are reasonably rounded.", "haiku_scores": {"c1": 2, "c2": 2, "c3": 2, "c4": 2, "c5": 2, "c6": 2}, "opus_scores": {"c1": 2, "c2": 2, "c3": 2, "c4": 2, "c5": 2, "c6": 2}}, {"pair": "1", "judge": "gpt", "order": "opus shown first", "winner": "tie", "why": "Both correctly calculate 32% versus approximately 33.3333% per founder, a 1.3333 percentage-point gain. Both handle SAFE conversion and Series A dilution correctly, reconcile ownership, and provide verifiable comparisons with adequate rounding.", "haiku_scores": {"c1": 2, "c2": 2, "c3": 2, "c4": 2, "c5": 2, "c6": 2}, "opus_scores": {"c1": 2, "c2": 2, "c3": 2, "c4": 2, "c5": 2, "c6": 2}}, {"pair": "1", "judge": "deepseek", "order": "haiku shown first", "winner": "tie", "why": "Both correctly calculate 32% and 33.33% per founder, with a 1.33 pp increase, and show full, consistent math.", "haiku_scores": {"c1": 2, "c2": 2, "c3": 2, "c4": 2, "c5": 2, "c6": 2}, "opus_scores": {"c1": 2, "c2": 2, "c3": 2, "c4": 2, "c5": 2, "c6": 2}}, {"pair": "1", "judge": "deepseek", "order": "opus shown first", "winner": "tie", "why": "Both correctly calculate all percentages, differences, and show the same results with clear, accurate tables.", "haiku_scores": {"c1": 2, "c2": 2, "c3": 2, "c4": 2, "c5": 2, "c6": 2}, "opus_scores": {"c1": 2, "c2": 2, "c3": 2, "c4": 2, "c5": 2, "c6": 2}}, {"pair": "2", "judge": "gpt", "order": "haiku shown first", "winner": "opus", "why": "Both correctly calculate ownership, SAFE conversion, dilution, and percentage-point gains. B leads with the requested comparison and reports the relative increase more precisely: 4.17%, versus A\u2019s rounded \u201cabout 4%.\u201d", "haiku_scores": {"c1": 2, "c2": 2, "c3": 2, "c4": 2, "c5": 2, "c6": 2}, "opus_scores": {"c1": 2, "c2": 2, "c3": 2, "c4": 2, "c5": 2, "c6": 2}}, {"pair": "2", "judge": "gpt", "order": "opus shown first", "winner": "tie", "why": "Both correctly calculate founder ownership, SAFE conversions, Series A dilution, and the percentage-point increase. Their share counts reconcile to 100%. A states relative growth more precisely; B\u2019s approximately 4% is reasonable rounding.", "haiku_scores": {"c1": 2, "c2": 2, "c3": 2, "c4": 2, "c5": 2, "c6": 2}, "opus_scores": {"c1": 2, "c2": 2, "c3": 2, "c4": 2, "c5": 2, "c6": 2}}, {"pair": "2", "judge": "deepseek", "order": "haiku shown first", "winner": "tie", "why": "Both correctly calculate all scenarios, differences, and are faithful to inputs.", "haiku_scores": {"c1": 2, "c2": 2, "c3": 2, "c4": 2, "c5": 2, "c6": 2}, "opus_scores": {"c1": 2, "c2": 2, "c3": 2, "c4": 2, "c5": 2, "c6": 2}}, {"pair": "2", "judge": "deepseek", "order": "opus shown first", "winner": "tie", "why": "Both correctly calculate 32% and 33.33% per founder, with 1.33pp increase, and show full working.", "haiku_scores": {"c1": 2, "c2": 2, "c3": 2, "c4": 2, "c5": 2, "c6": 2}, "opus_scores": {"c1": 2, "c2": 2, "c3": 2, "c4": 2, "c5": 2, "c6": 2}}]}, {"id": "msg-1", "family": "message-testing", "prompt": "We're launching next month and can't agree on the main message. Info in options.md. Tell us how to decide and which one to go with.", "input": "options.md", "input_text": "Product: Kitepay, invoicing + payment links for freelancers in Europe.\nThree candidate headlines the team argues about:\nA) \"Get paid 2x faster.\"\nB) \"Invoices that chase themselves.\"\nC) \"The invoicing tool built for European freelancers (VAT, SEPA, reverse charge handled).\"\nBudget: EUR 3,000 for pre-launch testing, 4 weeks. We have 1,200 people on a waitlist and an audience of about 8k on LinkedIn.\nThe CEO likes A, marketing likes B, sales likes C.\n", "rubric": [{"id": "c1", "criterion": "Makes a clear headline recommendation the team can act on.", "full_marks_when": "Chooses A, B, C, or a concrete revision, explains its advantages and tradeoffs for Kitepay, and distinguishes a provisional recommendation from a winner established by testing.", "zero_when": "Only describes how to decide, refuses to recommend a message, or declares a proven winner without evidence."}, {"id": "c2", "criterion": "Evaluates the messages against European freelancers\u2019 buying needs.", "full_marks_when": "Compares A\u2019s payment-speed benefit, B\u2019s implied automatic chasing, and C\u2019s European tax/payment specificity, explaining which needs each addresses and what remains unknown about their importance to buyers.", "zero_when": "Uses generic branding advice or personal taste without connecting the options to freelancers, invoicing, payment links, or European requirements."}, {"id": "c3", "criterion": "Avoids launching claims that the supplied product information does not substantiate.", "full_marks_when": "Flags that '2x faster' requires credible comparative evidence, that automatic chasing and VAT/SEPA/reverse-charge handling need confirmation, and makes the recommended launch wording conditional on verified capabilities or supplies safer wording.", "zero_when": "Treats all headline claims as verified features or recommends publishing an unsupported quantitative promise without qualification."}, {"id": "c4", "criterion": "Provides an executable test within the launch constraints.", "full_marks_when": "Specifies recruitment from the waitlist and/or LinkedIn, comparable message variants, audience assignment, a concrete budget allocation totaling no more than EUR 3,000, and a schedule that produces a decision within four weeks.", "zero_when": "Offers only 'test the headlines,' proposes an unaffordable or late study, or leaves the team unable to execute the plan."}, {"id": "c5", "criterion": "Defines how evidence will settle the disagreement.", "full_marks_when": "Sets a primary outcome tied to qualified customer intent, checks understanding and credibility, compares variants under equivalent conditions, and gives a practical decision rule and fallback for inconclusive results. CEO, marketing, and sales preferences do not determine the winner.", "zero_when": "Lets seniority or departmental votes decide, relies solely on likes or impressions, or collects results without explaining how to choose."}, {"id": "c6", "criterion": "Preserves input facts and uses sound arithmetic and evidence limits.", "full_marks_when": "Accurately uses EUR 3,000, four weeks, 1,200 waitlist members, and approximately 8,000 LinkedIn audience members. Any equal three-way budget split is EUR 1,000 per option, and any equal split of the full waitlist is 400 people per option; actual participation is not assumed. Does not add the two audiences as 9,200 unique reachable participants without checking overlap or invent conversion rates and statistical certainty.", "zero_when": "Misstates material input facts, makes budget or sample-size arithmetic errors, treats audience size as guaranteed participation, or presents invented performance data as fact."}], "runs": {"haiku-1": {"cost_usd": 0.00578, "seconds": 32, "output_tokens": 5799, "files": [{"name": "headline-decision.md", "text": "# Kitepay launch headline: decision\n\n## Recommendation\n\nGo with **C: \"The invoicing tool built for European freelancers (VAT, SEPA, reverse charge handled).\"**\n\nUse B (\"Invoices that chase themselves.\") as a feature section heading for automated payment reminders. Do not use A (\"Get paid 2x faster.\") unless we have measured data behind the 2x claim.\n\n## Why C\n\n- **It is the only one that is specific to what we sell.** Kitepay is for European freelancers. Generic invoicing tools can all claim speed and automation. Handling VAT, SEPA and reverse charge is the reason a European freelancer would pick us over them.\n- **It tells the reader who it is for.** Someone who doesn't need reverse charge can skip it. Someone who does will stop reading and pay attention. That filter is useful with a small audience.\n- **A is an unsupported claim.** \"2x faster\" compared to what? We have no data yet. Unsubstantiated performance claims also carry legal risk in the EU, so we would need evidence before running it.\n- **B describes a feature, not the product.** Automated reminders are good, but \"chase themselves\" doesn't say who the product is for or why it beats the alternatives. It works better as a supporting line than as the headline.\n- **Sales is closest to the buyer's objections**, and this is the message that answers them.\n\n## How to decide (agree on this before seeing results)\n\n**Primary metric:** signup or waitlist conversion from the landing page variant, or reply rate from the waitlist email. Clicks alone reward curiosity, not fit.\n\n**Secondary metric:** comprehension. Show each headline cold to freelancers and ask, \"What does this product do, and who is it for?\" A headline that gets clicks but produces wrong guesses is a loss.\n\n**Decision rule:**\n1. Choose C by default.\n2. Switch to A or B only if it beats C on the primary metric by at least 20% relative, the win shows up in at least two of the three channels, and it does at least as well on comprehension.\n3. If the winner gets clicks but fails comprehension, it stays out.\n4. A tie goes to C.\n\nWrite this rule down and get the CEO, marketing and sales to sign off before the tests start. That is how the choice moves from \"whose opinion wins\" to \"what the evidence says.\"\n\n## What the test can and cannot tell us\n\n- The waitlist split is 1,200 people across three headlines, about 400 per arm. At a 10% baseline, that only detects differences of roughly 6 percentage points or more. Treat it as a filter, not proof.\n- The LinkedIn audience of about 8k is organic reach, and paid ads reach further. Paid tests are the better source of volume.\n- Interviews give the most useful signal per euro for comprehension, but they are small samples.\n\n## Budget (EUR 3,000, 4 weeks)\n\n| Item | EUR | Notes |\n|---|---|---|\n| Paid LinkedIn headline tests | 1,200 | 3 \u00d7 400, same creative except the headline |\n| Freelancer interviews | 400 | About 12 \u00d7 30 incentive, plus recruiting |\n| Landing page variants and analytics | 300 | Three versions of the same page |\n| Waitlist email test | 0 | Uses existing tooling |\n| Reserve for a second round on the winner | 1,100 | Only spend if round one is unclear |\n\nThese figures are estimates. Check actual ad costs and incentive prices before committing.\n\n## Timeline\n\n- **Week 1:** Write the three full versions (same subhead and CTA, different headline). Set up landing variants and tracking. Write the decision rule and get sign-off. Recruit interviewees.\n- **Week 2:** Send the waitlist email test. Post organic LinkedIn variants. Start interviews.\n- **Week 3:** Run paid LinkedIn tests. Finish interviews.\n- **Week 4:** Read the results against the decision rule. Lock the headline for launch, or run a short second round if the result is unclear.\n\n## Hybrid option\n\nLead with C. Add a supporting line about the outcome only if beta users show measured payment-time improvements. Use B for the reminders feature. This keeps the honest, specific headline and still gets some of the energy from A and B.\n"}]}, "haiku-2": {"cost_usd": 0.0068, "seconds": 42, "output_tokens": 7455, "files": [{"name": "headline-decision.md", "text": "# Kitepay launch headline: recommendation and decision plan\n\n## Recommendation\n\nUse **C** as the main headline:\n\n> The invoicing tool built for European freelancers (VAT, SEPA, reverse charge handled).\n\nUse **B** (\"Invoices that chase themselves.\") as a feature line in the product page, emails and social posts, if automated payment reminders are live at launch. Do not lead with **A** unless you can back \"2x\" with measured data.\n\nWhy C:\n- **It tells the visitor who it is for.** A and B could describe almost any invoicing tool. C filters for the people on the waitlist and the LinkedIn audience, who are European freelancers.\n- **It is a claim you can check.** \"2x faster\" has no baseline (faster than what, measured how?). Misleading-claim rules in EU advertising mean a number like that needs evidence before it runs.\n- **B explains what happens after the invoice is sent.** That is a good feature, but it does not give a reason to pick Kitepay over another tool.\n- **Sales liking C is useful information.** It points at the buyer who is most likely to pay.\n\nCondition: C is only right if VAT, reverse charge and SEPA actually work at launch. If any of them is partial, narrow the wording to what is true.\n\n## How to decide\n\nAgree on this rule before anyone looks at results. The argument is about taste, so the rule has to settle it in advance.\n\n1. **Gate: is the claim true and provable at launch?** Any headline that fails is out. Product and whoever signs off on ads check each claim.\n2. **Comprehension:** in a 5-second test, what share of people correctly say what the product is and who it is for? This is the main criterion.\n3. **Qualified conversion:** cost per signup from someone who fits (a freelancer in the EU), not raw clicks. Add one qualifying question to the waitlist form (country, freelancer or not) so this can be measured.\n4. **Tiebreaker:** which headline works best as a social post and a page title.\n\nDecision rule:\n- Highest comprehension score wins, unless another headline clearly wins on qualified conversion.\n- A headline that wins on raw clicks but loses on comprehension is attracting curiosity, not buyers. It does not win.\n- If the results are inconclusive, default to C.\n- One named person makes the call, by **2026-10-21** (end of week 2), so the launch copy can be locked.\n\n## Test plan (4 weeks, EUR 3,000)\n\n| Step | When | What | Cost |\n|---|---|---|---|\n| Claim audit | Week 1 | Check every claim in each headline against the product | Internal |\n| 5-second comprehension test | Weeks 1-2 | About 40 freelancers in target countries (panel or waitlist), three versions of the page or card | ~EUR 300 incl. incentives |\n| Waitlist email split | Week 2 | Randomly split 1,200 into three groups of 400, one headline each; measure clicks and signups | EUR 0 |\n| LinkedIn ads | Weeks 2-4 | Same creative, three headlines, equal budget; measure CTR and cost per qualified signup | ~EUR 2,200 (~EUR 730 each) |\n| Buffer | Throughout | Landing page variant, contingencies | ~EUR 500 |\n\nBudget total: EUR 3,000.\n\nLimits on the numbers:\n- At a 10% click rate, telling 10% from 13% needs roughly 1,800 people per arm. The waitlist split (400 per arm) and the ads (a few hundred clicks per arm) only detect large gaps.\n- So the quantitative tests are for ruling out weak options and checking direction. The comprehension test is what separates C from B.\n- Ads continue after the decision as launch learning. They do not need to finish before the decision.\n"}]}, "opus-1": {"cost_usd": 0.16898, "seconds": 63, "output_tokens": 5409, "files": [{"name": "headline-decision.md", "text": "# Kitepay launch headline: how to decide, and what to pick\n\n## Short answer\n\n**My pick going in: B, \"Invoices that chase themselves,\" as the headline, with C used as the subheadline.** Run the 4-week test below to confirm it or overrule it. Agree on the decision rule before any data comes in.\n\n> **Invoices that chase themselves.**\n> Invoicing and payment links for European freelancers, with VAT, SEPA and reverse charge handled.\n\nThis assumes Kitepay really does send automatic payment reminders. If it doesn't do that yet, B is off the table.\n\n## Why, before testing\n\n| | A: \"Get paid 2x faster\" | B: \"Invoices that chase themselves\" | C: \"Built for European freelancers\u2026\" |\n|---|---|---|---|\n| Says what the product is | No. Could be a bank, a factoring service or a loan. | Yes: invoicing. | Yes, very clearly. |\n| Says why it's different | No. Every payment tool promises faster payment. | Partly. It names the pain (chasing clients) and the mechanism. | Yes. EU VAT, SEPA and reverse charge are the moat against global tools. |\n| Memorable or shareable | Medium | High | Low (it's 13 words of features) |\n| Can you back it up at launch? | **Not yet.** \"2x\" needs before/after data. In the EU, unsubstantiated numeric claims are a legal risk under unfair commercial practices rules, and they're also a credibility risk. | Yes, if the product sends reminders | Yes, if all three are fully supported |\n\n- **A** is the weakest. It's generic, and the claim can't be backed up yet. If beta data later shows something like \"paid in 9 days instead of 21,\" that number makes a good *proof point* lower on the page or in ads. It shouldn't be the headline before launch.\n- **B and C aren't competitors. They do different jobs.** B is the hook and C is the positioning. A headline/subheadline pair uses both, and that combination probably beats either one alone.\n- The real open question is which one goes in the big type. That's what the test answers.\n\n## How to decide: the test (EUR 3,000, 4 weeks)\n\n### 1. Agree on the rule first (week 1, a 30-minute meeting)\nThe CEO, marketing and sales sign off on this in writing before launch:\n\n- **Primary metric:** *cost per qualified signup from cold traffic.* A qualified signup is someone who signs up and answers \"Are you a freelancer based in the EU?\" with yes. Don't decide on clicks. Clever headlines win on clicks and can still lose on signups.\n- **Winner:** the variant with the lowest cost per qualified signup, provided its lead is clear (see \"What the test can detect\").\n- **If there's no clear lead:** go with whichever headline scores best on the clarity test in step 3. If that's tied too, choose **B headline + C subhead**.\n- Everyone accepts the result, including whoever backed the losing option.\n\n### 2. Cold-traffic ad test, the main signal (weeks 2\u20133, EUR 1,800)\n- Use **Meta (Facebook/Instagram) ads**, targeting freelancers and self-employed people in your launch countries. LinkedIn clicks typically cost several times more, so EUR 1,800 there would buy too few clicks to learn anything.\n- Run 3 variants at **EUR 600 each**. The ad image, the landing page and the call to action (\"Join early access\") are identical. Only the headline changes, and it changes in the ad and on the landing page together.\n- Test A exactly as written, to measure how much the idea of faster payment appeals. Testing it doesn't mean you can publish \"2x.\"\n- Track signups and the qualifying question for each variant using UTM tags.\n\n**What the test can detect:** at roughly EUR 1\u20131.50 per click, each variant gets about 400\u2013500 visitors. That's enough to spot a big difference, for example a 15% signup rate against 22%. It isn't enough to separate 15% from 17%. Results that close count as a tie, and the tiebreaker decides. A tie also tells you the headline choice matters less than everyone is arguing.\n\n### 3. Clarity test, the tiebreaker (week 1\u20132, EUR 450)\n- Run a 5-second test on an unmoderated testing tool (Lyssna, UsabilityHub or similar). Recruit about 50 people per variant who are freelancers or self-employed in the EU.\n- Show each person the hero section for 5 seconds, then ask: *\"What does this product do? Who is it for?\"*\n- Score the % of answers that are correct on both questions. A headline that people misread will hurt more at launch than any difference in click rate.\n\n### 4. Waitlist and interviews, for the \"why\" (weeks 1\u20132, EUR 375)\n- **Waitlist survey (free):** email the 1,200 people one question: *\"What's most painful about getting paid today? (a) waiting too long (b) chasing late clients (c) VAT/cross-border admin.\"* The three answers line up with A, B and C. The waitlist has already opted in, so treat this as context, not the deciding vote.\n- **Interviews:** 15 calls of 20 minutes each with waitlist members, with a EUR 25 voucher each. Show all three headlines and ask which one they'd forward to a freelancer friend, and why. The words people use here are good material for the final copy.\n- **LinkedIn (8k, free):** use it to recruit interviewees. A poll is fine for engagement, but don't count it as evidence. Your followers aren't the cold market, and posting order and timing distort the results.\n\n### 5. Decide and write the final version (week 4, EUR 375 buffer)\n- Apply the rule from step 1. Write the final hero section as **winner as headline, plus the positioning line**. Turn interview quotes into the page's proof points.\n- Keep the buffer for extending the ad test by a few days if the results are borderline at the end of week 3.\n\n## Budget\n\n| Item | EUR |\n|---|---|\n| Meta ad test (3 \u00d7 600) | 1,800 |\n| 5-second clarity test (~150 respondents) | 450 |\n| 15 interview incentives | 375 |\n| Buffer (extend the test or buy landing page tooling) | 375 |\n| **Total** | **3,000** |\n\n## Timeline\n\n| Week | What happens |\n|---|---|\n| 1 | Sign off the decision rule. Build the landing page variants and ads. Send the waitlist survey and book interviews. Launch the clarity test. |\n| 2\u20133 | Ads run (don't edit them mid-test). Do the interviews. |\n| 4 | Analyse, apply the rule, write the final hero copy and the launch assets. |\n"}]}, "opus-2": {"cost_usd": 0.26244, "seconds": 109, "output_tokens": 8788, "files": [{"name": "headline-decision.md", "text": "# Kitepay launch headline: how to decide, and what to pick\n\n## Recommendation\n\nLaunch with **B as the headline and C as the subhead**, unless the test below shows a clear winner:\n\n> **Invoices that chase themselves.**\n> Automatic payment reminders, SEPA payment links, and VAT and reverse charge handled. Built for European freelancers.\n\nDrop **A** unless you can back up \"2x\" with data before launch.\n\nThat's the default. The test can overturn it. Everyone agrees to the rule *before* any results come in, so the data settles the argument instead of whoever has the most seniority.\n\n## Why this default\n\nThe three options aren't really competing. Each does a different job:\n\n| | Type | Strength | Problem |\n|---|---|---|---|\n| **A** \"Get paid 2x faster\" | Outcome claim | Simple. It's the result people actually want | **2x faster than what?** Pre-launch, you have no data for it. In the EU, a specific performance claim you can't substantiate is a misleading-advertising risk (have someone check this before you use it). \"Get paid faster\" also sounds like every other invoicing tool |\n| **B** \"Invoices that chase themselves\" | Mechanism | Memorable and concrete. It hits a real, emotional pain: chasing late payers | It only works if automatic reminders ship at launch. \"Chase\" is an English idiom that may not land with non-native readers or survive translation |\n| **C** \"Built for European freelancers (VAT, SEPA, reverse charge)\" | Positioning / audience | The real differentiator against global tools. It tells the right people \"this is for you\" | Too long for a headline. It reads as a feature list |\n\nC is the strongest *reason to choose Kitepay*. B is the strongest *hook*. Putting B on top and C underneath gets you both. Sales is right that C matters. It just works better in the subhead, ads, and sales decks than in the headline itself.\n\n## Step 0: gate checks (days 1\u20133, free)\n\nDo these before spending anything. Each one can knock out an option:\n\n1. **Can you prove \"2x faster\"?** You'd need beta data or a sourced benchmark, e.g. \"beta users were paid in 14 days vs. a 28-day average\". If you can't, A is out, or it gets softened to \"Get paid faster\", which is weaker. If you can, A stays in the test with the proof on the page.\n2. **Do automatic reminders ship on launch day?** If they don't, B is out.\n3. **Which languages are you launching in?** If not English-only, test the headlines in the real languages. A wordplay winner in English can lose in German.\n\n## The test (4 weeks, EUR 3,000)\n\n### Budget\n\n| Item | EUR | What it tells you |\n|---|---|---|\n| Paid traffic test (Meta, or LinkedIn if you can afford the CPCs), 3 arms \u00d7 ~700 | 2,100 | Which message gets cold, targeted freelancers to sign up most cheaply. **This is the deciding metric** |\n| Message-testing panel with your target audience (e.g. Wynter, Lyssna; check current pricing) | ~600 | *Why* a message wins or loses: clarity, credibility, \"is this for me?\" Catches problems like B's idiom or A seeming unbelievable |\n| Incentives for 8\u201310 interviews from the waitlist (~EUR 30 vouchers) | 300 | The words freelancers use for their pain, which you can reuse in copy |\n\n### Setup\n\n- **Three landing pages, identical except for the hero headline and subhead.** Same design, same CTA (\"Join the waitlist\" / \"Get early access\").\n  - A: \"Get paid 2x faster.\" + proof line\n  - B: \"Invoices that chase themselves.\" + a neutral subhead\n  - C: \"The invoicing tool built for European freelancers.\" + \"VAT, SEPA and reverse charge handled.\"\n  - Give each arm a neutral subhead so you're testing the *idea* and not the B+C combination. B+C is the default. You don't need to prove it.\n- **Ads:** same image and same targeting (freelancers and self-employed people in your launch countries). The ad text uses the arm's headline. Use the platform's built-in A/B test feature with a fixed, equal budget per arm. **Don't** let the algorithm shift spend toward an early leader, because that ruins the comparison.\n- **Track** impressions, clicks, and waitlist signups for each arm, using UTMs and one signup form.\n\n### Timeline\n\n| Week | Activity |\n|---|---|\n| 1 | Gate checks. Build the pages. **All three stakeholders sign off on the decision rule.** Survey the waitlist and start interviews |\n| 2\u20133 | Paid test runs for 14 days straight (covers weekday/weekend swings). Panel test runs in parallel. **Don't stop early** |\n| 4 | Run `decide.py`, read the panel feedback, decide, write the final copy |\n\n### Decision rule (agree on this in week 1)\n\n1. **Primary metric: cost per waitlist signup** from cold paid traffic. This one number covers both \"does the ad get clicks\" and \"does the page convert\".\n2. **Clear winner** if the cheapest arm is **at least 20% cheaper per signup** than the runner-up **and** the difference is statistically meaningful (p < 0.10). `decide.py` checks both.\n3. **Veto:** if the panel shows most of the audience misreads or doesn't believe the winning headline, take the runner-up.\n4. **No clear winner \u2192 launch with the default (B + C).** This is the most likely outcome, and it's fine. It means the headline matters less than the team thinks, so choose on strategy (differentiation) and move on.\n\n### Be realistic about what EUR 2,100 can detect\n\nAt roughly EUR 1\u20131.50 per click you get ~500\u2013700 visitors per arm. That's only enough to spot **big** differences, something like 15% vs. 21% signup conversion. Small differences will look like noise. That's why the rule has a pre-agreed default. Don't keep spending to chase a tiny edge.\n\n## What *not* to use as the decision\n\n- **Your 1,200 waitlist members.** They already signed up because of whatever message you showed them, and they know you. Use them for interviews and the survey (what's your biggest invoicing headache? what made you sign up?), not for picking the headline.\n- **LinkedIn organic posts to your 8k followers.** The algorithm and your existing relationships drown out the message effect. Feel free to post all three angles. Treat the results as color, not a vote.\n- **Asking people which headline they prefer.** What people say they like doesn't predict what they click. Measure behavior.\n\n## Files\n\n- `headline-decision.md`: this document.\n- `decide.py`: put in each arm's spend, impressions, clicks, and signups, then run `python3 decide.py` to get the verdict under the agreed rule. It currently holds **made-up placeholder numbers**. In that example B is cheapest but not by enough, so the verdict is \"no clear winner \u2192 B + C\". This shows how often a close result ends up at the default.\n"}]}}, "verdicts": [{"pair": "1", "judge": "gpt", "order": "haiku shown first", "winner": "opus", "why": "B better connects messages to buyer needs and conditions feature claims. However, it still proposes advertising unsupported \u201c2x\u201d wording and miscalculates visitor estimates. A assumes unverified capabilities. Both lack credibility checks and sufficiently rigorous winner rules.", "haiku_scores": {"c1": 2, "c2": 1, "c3": 0, "c4": 2, "c5": 1, "c6": 1}, "opus_scores": {"c1": 2, "c2": 2, "c3": 1, "c4": 2, "c5": 1, "c6": 1}}, {"pair": "1", "judge": "gpt", "order": "opus shown first", "winner": "opus", "why": "A better compares buyer needs and qualifies feature claims. Both lack explicit audience assignment and credibility checks. A still proposes advertising unsupported \u201c2x\u201d wording; B recommends unverified capabilities, privileges sales opinion, and conflates LinkedIn audience with organic reach.", "haiku_scores": {"c1": 2, "c2": 1, "c3": 0, "c4": 1, "c5": 1, "c6": 1}, "opus_scores": {"c1": 2, "c2": 2, "c3": 1, "c4": 1, "c5": 1, "c6": 2}}, {"pair": "1", "judge": "deepseek", "order": "haiku shown first", "winner": "tie", "why": "Both deliverables fully satisfy all rubric criteria with clear recommendations, sound testing plans, and accurate use of input facts.", "haiku_scores": {"c1": 2, "c2": 2, "c3": 2, "c4": 2, "c5": 2, "c6": 2}, "opus_scores": {"c1": 2, "c2": 2, "c3": 2, "c4": 2, "c5": 2, "c6": 2}}, {"pair": "1", "judge": "deepseek", "order": "opus shown first", "winner": "opus", "why": "A's test plan is more executable, with sound budget arithmetic and a clear tiebreaker. B's budget splits are vague and its timeline is less concrete.", "haiku_scores": {"c1": 2, "c2": 2, "c3": 2, "c4": 1, "c5": 2, "c6": 1}, "opus_scores": {"c1": 2, "c2": 2, "c3": 2, "c4": 2, "c5": 2, "c6": 2}}, {"pair": "2", "judge": "gpt", "order": "haiku shown first", "winner": "opus", "why": "B better compares buyer needs and completes testing before deciding. A decides while ads are still running and prioritizes comprehension. B still fails to verify European feature claims, qualify signups, or supply its promised decision script.", "haiku_scores": {"c1": 2, "c2": 1, "c3": 2, "c4": 1, "c5": 1, "c6": 2}, "opus_scores": {"c1": 2, "c2": 2, "c3": 1, "c4": 2, "c5": 1, "c6": 2}}, {"pair": "2", "judge": "gpt", "order": "opus shown first", "winner": "opus", "why": "A provides stronger buyer-focused comparisons and a more executable, equivalent-condition test. However, its default copy needs verified European capabilities and qualified conversions. B handles claim verification better, but prioritizes comprehension and decides before its advertising evidence is complete.", "haiku_scores": {"c1": 2, "c2": 1, "c3": 2, "c4": 1, "c5": 1, "c6": 2}, "opus_scores": {"c1": 2, "c2": 2, "c3": 1, "c4": 2, "c5": 1, "c6": 2}}, {"pair": "2", "judge": "deepseek", "order": "haiku shown first", "winner": "tie", "why": "Both deliver clear, actionable plans with sound testing, evaluation, and risk mitigation. They are equally excellent.", "haiku_scores": {"c1": 2, "c2": 2, "c3": 2, "c4": 2, "c5": 2, "c6": 2}, "opus_scores": {"c1": 2, "c2": 2, "c3": 2, "c4": 2, "c5": 2, "c6": 2}}, {"pair": "2", "judge": "deepseek", "order": "opus shown first", "winner": "opus", "why": "A provides a more executable test plan, clear decision rule, and respects budget arithmetic; B's plan has timeline and budget inconsistencies.", "haiku_scores": {"c1": 2, "c2": 2, "c3": 2, "c4": 1, "c5": 1, "c6": 1}, "opus_scores": {"c1": 2, "c2": 2, "c3": 2, "c4": 2, "c5": 2, "c6": 2}}]}], "judge_prompt": "\"You are a demanding expert reviewer judging two deliverables for the same request, as the person who asked would. \"\n       \"Use the rubric. Check facts and arithmetic against the input. Do not reward length, formatting or the use of named frameworks for their own sake. \"\n       'Answer ONLY JSON: {\"a_scores\":{\"c1\":0-2,...},\"b_scores\":{\"c1\":0-2,...},\"winner\":\"A\"|\"B\"|\"tie\",\"why\":\"<=40 words\"}')"}