{"cases": [{"id": "pos-2", "family": "positioning", "prompt": "We need to rewrite how we describe our product. Everything we know is in brief.md. Give us the positioning.", "input": "brief.md", "skill": "dunford-obviously-awesome", "input_text": "# Shiftwise\nScheduling and time-tracking app. Built originally for any hourly workforce. 3 years old, 40 employees, USD 3.1M ARR.\nCustomer breakdown: 61% of revenue from independent veterinary clinics (2-6 vets), 15% restaurants, the rest a long tail of retail and gyms. Vet clinics have 2% monthly logo churn; restaurants 7%.\nWhy vets love it (from 12 interviews): handles on-call rotations and emergency shifts, tracks vet-tech certifications so an uncertified tech can't be scheduled for anaesthesia monitoring, exports hours to the payroll tools vets use. Two clinic managers said \"it's the only scheduler that understands we're a hospital, not a shop.\"\nCompetitors in deals: When I Work, Deputy, 7shifts (restaurants), and paper rotas. We lose to Deputy on price in restaurants.\nSales team pitches \"the easiest scheduling app for hourly teams\". Homepage hero shows a barista.\nCEO worries narrowing to vets makes the market too small. There are about 32,000 vet clinics in the US.\n", "rubric": [{"id": "c1", "criterion": "Choose a clear primary audience supported by the customer evidence.", "full_marks_when": "Positions Shiftwise primarily for independent veterinary clinics with 2\u20136 vets, explains why their revenue concentration and lower churn justify that focus, and distinguishes marketing focus from a requirement to abandon other customers.", "zero_when": "Keeps all hourly workforces as equally important audiences, prioritizes restaurants, or targets large veterinary enterprises without evidence."}, {"id": "c2", "criterion": "Translate the veterinary capabilities into a distinctive, credible customer promise.", "full_marks_when": "Connects on-call rotations, emergency shifts, certification-aware scheduling, and payroll exports to managing a clinic's hospital-like staffing needs. Explains why these matter beyond generic ease of scheduling without claiming guaranteed clinical safety or regulatory compliance.", "zero_when": "Offers only generic convenience or efficiency claims, lists features without customer meaning, or invents capabilities or guarantees."}, {"id": "c3", "criterion": "Stay factually and numerically faithful to the brief.", "full_marks_when": "Preserves the supplied facts and labels inference as inference. Any calculations are correct: veterinary revenue is $1.891M ARR, restaurant revenue is $465,000 ARR, and the remaining 24% is $744,000 ARR; monthly logo churn is 2% for vets versus 7% for restaurants. Does not confuse logo churn with revenue churn or treat 61% of revenue as 61% of customers.", "zero_when": "Materially misstates the evidence, makes an arithmetic error, fabricates research or customer results, or misrepresents revenue share or churn."}, {"id": "c4", "criterion": "Make the competitive distinction relevant to actual buying alternatives.", "full_marks_when": "Explains why a veterinary clinic would choose Shiftwise over general scheduling tools or paper rotas, using the clinic-specific workflows as the distinction. Avoids asserting that competitors lack particular features without evidence and avoids a price-led restaurant position given the losses to Deputy.", "zero_when": "Provides no meaningful competitive distinction, recommends competing on restaurant price, or bases differentiation on unsupported competitor claims."}, {"id": "c5", "criterion": "Deliver positioning and copy the team can use immediately.", "full_marks_when": "Provides a decisive positioning statement and usable customer-facing headline and supporting copy that identify the veterinary audience and concrete value. Resolves the mismatch between the current generic sales pitch, barista homepage, and veterinary customer strengths with specific replacement messaging or imagery direction.", "zero_when": "Supplies only analysis, questions, terminology, or a process for developing positioning, leaving the team to write the actual positioning themselves."}, {"id": "c6", "criterion": "Address the CEO's market-size concern without inventing a market estimate.", "full_marks_when": "Acknowledges the approximately 32,000 US veterinary clinics while noting that the eligible independent 2\u20136-vet subset and achievable revenue cannot be calculated from the brief. Gives a practical way to validate the focus, such as checking eligible clinic count, annual contract value, and attainable penetration, without weakening the positioning into an everyone message.", "zero_when": "Ignores the concern, assumes all 32,000 clinics fit the target, invents pricing or a dollar market size, or claims the market is definitely sufficient or insufficient without supporting evidence."}], "runs": {"haiku-skill-1": {"cost_usd": 0.01806, "seconds": 105, "output_tokens": 21410, "files": [{"name": "shiftwise-positioning-canvas.md", "text": "# Shiftwise positioning canvas\n\n**Prepared:** 2026-10-08\n**Status:** Provisional. Customer interviews and company revenue data support the vet focus. Competitor parity, vet win/loss, and buyer comprehension are untested.\n**Source:** brief.md (only source used)\n\n## Positioning in one line\n\nFor independent veterinary clinics with 2 to 6 vets, Shiftwise is scheduling and time-tracking software that checks staff certifications when shifts are assigned and handles on-call and emergency rotations. It is not the easiest scheduler for any hourly team.\n\n## 1. Scope and status\n\n- **Offering and current version:** Scheduling and time-tracking app as sold today. The brief describes no optional modules and no future capabilities, so every claim below is about the current product.\n- **Decision supported:** Replace the current sales pitch (\"easiest scheduling app for hourly teams\") and homepage positioning, and answer the CEO's concern that a vet focus makes the market too small.\n- **Current category and historical assumption:** The product was built for \"any hourly workforce,\" and sales pitches it as the easiest scheduling app for hourly teams. The homepage hero shows a barista. This frame reflects the original concept, not the segment where revenue and retention are strongest.\n- **Evidence boundary:** 12 customer interviews (segment mix not stated in the brief), revenue split and churn by segment (date not stated), a list of competitors seen in deals (win/loss and segment not stated), and one CEO concern. No demos, deal records, or usage data are in the brief.\n\n## 2. Evidence register\n\n| ID | Observation or claim | Evidence and date | Label | Limitation |\n| --- | --- | --- | --- | --- |\n| E1 | 61% of revenue (about USD 1.9M of 3.1M ARR) comes from independent vet clinics with 2 to 6 vets | Company revenue data, date not stated | Observed | Does not show how many clinics or their contract values |\n| E2 | Vet clinics lose about 2% of logos per month | Company churn data, date not stated | Observed | Monthly rate; about 21% annualized if constant |\n| E3 | Restaurants are 15% of revenue (about USD 0.47M) and lose about 7% of logos per month | Company churn data, date not stated | Observed | About 58% annualized if constant; reasons for churn not in brief |\n| E4 | Vets value handling on-call rotations and emergency shifts | 12 interviews | Observed | Count of vets who said this is not stated |\n| E5 | Vets value tracking vet-tech certifications so an uncertified tech can't be scheduled for anaesthesia monitoring | 12 interviews | Observed | Not known whether competitors offer the same gate |\n| E6 | Vets value exporting hours to the payroll tools vets use | 12 interviews | Observed | Payroll export is common; this may not be a differentiator |\n| E7 | Two clinic managers said: \"it's the only scheduler that understands we're a hospital, not a shop\" | Interview quotes | Observed | Two of 12 interviews; a quote is not a comparison with competitors |\n| E8 | Restaurant deals are lost to Deputy on price | Deal outcomes, date not stated | Observed | Number of deals and reasons beyond price not stated |\n| E9 | Competitors in deals: When I Work, Deputy, 7shifts (restaurants), and paper rotas | Deal records as summarized in brief | Observed | Segment of each deal not stated; no vet-specific list |\n| E10 | Sales pitch: \"the easiest scheduling app for hourly teams.\" Homepage hero shows a barista | Current marketing | Observed (messaging, not customer evidence) | Shows what is said, not what buyers value |\n| E11 | Around 32,000 vet clinics in the US | Brief, source not stated | Observed (unverified source) | Counts all clinic sizes; the 2 to 6 vet independent subset is not counted |\n| E12 | Vet clinic market is \"too small\" per CEO | CEO concern | Hypothesis | Opinion; no reach or revenue target is given to test it against |\n| H1 | Paper rotas and spreadsheets cannot enforce certifications at assignment time | Inference from E5 and E9 | Hypothesis | Not tested with buyers |\n| H2 | Deputy and When I Work cannot block assignments by certification or model on-call rotations | Not in brief | Unknown, treated as hypothesis | Must be checked with a live demo |\n| H3 | Vet practice-management software with built-in scheduling is a live alternative | Not in brief | Unknown | Not mentioned in any source; check deals |\n\nContrary evidence to keep in view: the \"only scheduler\" quote comes from two of 12 interviews. Competitors appear in deals with no segment label, so the vet alternatives set is not established.\n\n## 3. Buyer situation and competitive alternatives\n\n**Buying situation:** A clinic manager builds weekly rotas for 2 to 6 vets and techs. Consideration is triggered by after-hours and emergency coverage and by the need to make sure only qualified techs are assigned to restricted tasks such as anesthesia monitoring. Hours then feed payroll. The brief does not say what triggers a clinic to start looking, or who signs off.\n\n| Alternative | Why the buyer keeps or considers it | Where it falls short for this situation | Evidence |\n| --- | --- | --- | --- |\n| Paper rotas and spreadsheets | Free, familiar, already in place | Certification checks and on-call handoffs are likely manual (H1) | E9, H1 |\n| Deputy | Generic hourly scheduling; wins on price in restaurants | Vet fit unknown; whether it gates by certification or models on-call is unknown (H2) | E8, E9, H2 |\n| When I Work | Generic hourly scheduling; appears in deals | Vet fit unknown (H2) | E9, H2 |\n| Vet practice-management software scheduling | Already used in the clinic, possibly | Unknown whether it appears in deals (H3) | H3 |\n| 7shifts | Restaurant scheduling | Not a vet alternative in evidence; excluded from vet canvas | E9 |\n\n**Excluded theoretical rivals:** 7shifts, for the vet segment only; it appears in restaurant deals. Add it back if vet deals show it.\n\nRestaurants and the long tail (retail, gyms) have different alternatives and different value, so they need separate canvases. See section 5.\n\n## 4. Capability-to-value map\n\n| ID | Actual alternative | Demonstrable difference | What it enables | Buyer consequence / value theme | Evidence and confidence |\n| --- | --- | --- | --- | --- | --- |\n| V1 | Paper rotas and spreadsheets; generic schedulers (H2) | Certifications held on each tech's record; a tech without the required certification can't be scheduled for anesthesia monitoring | Manager staffs restricted tasks without checking credentials by hand | Fewer unqualified assignments on high-risk tasks; less compliance and safety exposure (size of exposure not measured) | E5. Valued in interviews; mechanism stated in brief; demo and competitor parity not verified |\n| V2 | Paper rotas; generic schedulers (H2) | Rotations and emergency shifts handled inside the scheduler | Manager builds on-call rotations and fills urgent gaps | Clinic keeps after-hours and emergency coverage without last-minute scrambling | E4. Valued in interviews; competitor parity not verified |\n| V3 | Manual hours entry or re-keying into payroll | Hours export to the payroll tools vets use | Hours reach payroll without re-keying | Time saved (not measured) | E6. Low confidence as a primary difference; likely table stakes; use as supporting proof |\n\n**Primary value themes:**\n1. **Staffing that respects clinical credentials.** Restricted tasks go only to qualified staff (V1).\n2. **Covering after-hours and emergency shifts.** Rotations and urgent gaps are handled in one place (V2).\n\n**Supporting proof:** Payroll export (V3). Use it to answer \"does it integrate with what we use?\" but not to lead.\n\n**Discarded claims:**\n- \"The easiest scheduling app for hourly teams\": ease is not evidenced as a difference, and it puts Shiftwise on the axis where Deputy wins on price.\n- \"Built for any hourly workforce\": breadth does not map to a value chain for any single buyer.\n- \"The only scheduler that understands we're a hospital\": a two-interview quote. Test it against competitors before using it as a claim. Use the wording as a messaging hypothesis.\n- Barista hero: no link to any value theme.\n\n## 5. Best-fit customers and boundaries\n\n- **Target segment:** Independent veterinary clinics with 2 to 6 vets, including the on-call and emergency coverage and certification-restricted tasks described in interviews.\n- **Observable fit signals:** Clinic runs after-hours or emergency rotations; has staff whose certifications limit which tasks they can cover (anesthesia monitoring is the example in the brief); exports hours to payroll tools common in vet clinics. Signals are drawn from interview descriptions; none has been tested as a screening rule.\n- **Why they care intensely:** A wrong assignment on a restricted task is a clinical and compliance problem, and an uncovered emergency shift is an operational one. V1 and V2 address both.\n- **Examples in the evidence:** E1 and E2 (61% of revenue, lowest churn); E4, E5, E7 (interview value).\n- **Poor fit boundary:** Clinics with no restricted tasks and no after-hours rotations would likely value the product less (hypothesis). Single-vet practices and corporate-owned groups are not covered by evidence in the brief.\n- **Reach and commercial viability:** Vets are about USD 1.9M of ARR with the lowest churn, so the segment is already commercially viable. The CEO's concern can be tested but not answered from the brief: 32,000 is all US vet clinics of every size, not the 2 to 6 vet independent subset, and the brief gives no customer count, contract value, or revenue target. Penetration needed to hit a goal cannot be calculated yet.\n- **Other segments considered:**\n  - *Restaurants (15%, USD 0.47M, 7% monthly churn, lost on price to Deputy):* alternatives (Deputy, When I Work, 7shifts) differ from vets, and no value difference is in evidence. Keep out of the vet canvas. Separate canvas needed before any restaurant positioning is written.\n  - *Retail and gyms (remaining 24%, about USD 0.74M):* no evidence in the brief on alternatives, value, or churn. Unassessed.\n\n## 6. Market frame comparison and decision\n\n| Candidate frame | Expected features | Expected competitors and buyer | Price/channel assumptions | Visibility of our value | Misleading expectations and education burden |\n| --- | --- | --- | --- | --- | --- |\n| **Current:** Easiest scheduling app for hourly teams | Ease of setup, broad industry coverage, mobile app | Deputy, When I Work, 7shifts; any hourly business owner | Low-price self-serve; price competition | V1 and V2 peripheral or absent; ease, which is unevidenced, is central | Buyers compare on price and breadth where Shiftwise is not shown to win. Low education burden, but the wrong comparison |\n| **Recommended:** Scheduling and time-tracking software for veterinary clinics (a subcategory of employee scheduling) | Credential-restricted shifts, on-call and emergency rotas, payroll export, multi-site or multi-vet support (the last not in brief; expectation must be tested) | Clinic managers in 2 to 6 vet practices; alternatives are paper, generic schedulers, possibly practice-management scheduling | Per-clinic pricing (not in brief); direct sales to clinic owners | V1 and V2 central because \"for veterinary clinics\" tells the buyer the rules are built in | Buyers may expect practice-management features (patient records, billing); the product does not claim them. Low education burden: scheduling is an existing category |\n| **Rejected for now:** Workforce scheduling for healthcare | Credential tracking across human clinical roles, compliance reporting | Hospital HR and staffing leaders; large healthcare vendors | Enterprise sales; long cycles | V1 visible, but V2 and the vet context are lost | Overreach beyond any evidence in brief; large education and sales burden |\n\n- **Selected frame:** Scheduling and time-tracking software for veterinary clinics.\n- **Framing style:** Win a subcategory of an existing category (employee scheduling).\n- **Why it wins this comparison:** Buyers already understand scheduling and time tracking. The qualifier tells a vet manager that certifications and on-call rotas come built in, which makes V1 and V2 visible at once. It removes the comparison on ease and price that the current pitch invites.\n- **Principal rejected frame:** The current \"easiest app for hourly teams.\" It keeps the product in the most crowded comparison and hides the vet value.\n- **Decision confidence and unresolved expectations:** Moderate on direction, low on specifics. The frame depends on whether vet buyers recognize V1 and V2 as differences from their alternatives (H2, H3). The \"veterinary clinics\" wording is untested against \"veterinary hospitals,\" which matches the customer quote.\n\nThe narrow frame is a business choice, not only a copy change. It requires: vet-specific onboarding (capturing each tech's certifications is likely more setup work; hypothesis), sales focus on vet clinics, and a decision on how restaurants are served. The brief does not record any of these decisions.\n\n## 7. Buyer-facing explanation\n\n> Shiftwise is scheduling and time-tracking software for veterinary clinics. It helps managers build on-call and emergency rotas, and it keeps each tech's certifications on record, so a tech without the required certification can't be scheduled for work like anesthesia monitoring. Hours export to the payroll tools clinics already use.\n\n**Claim trace:**\n- \"Scheduling and time-tracking software\" to brief product description.\n- \"For veterinary clinics\" to E1, E2, E4, E5.\n- \"On-call and emergency rotas\" to V2, E4.\n- \"Certifications on record; can't be scheduled for anesthesia monitoring\" to V1, E5.\n- \"Hours export to the payroll tools clinics already use\" to V3, E6 (supporting only).\n\n**Scope or important limitation:** The explanation covers scheduling and time tracking only. The brief establishes no other clinic-operations features, so do not describe them. It makes no claim that competitors lack certification gating, and it makes no claim about time or cost saved.\n\nThis explanation is a messaging translation and is not a substitute for the positioning analysis.\n\n## 8. Validation and revision plan\n\n| Question or hypothesis | Suitable buyer/deal evidence | What would support it | What would contradict it | Owner and next action |\n| --- | --- | --- | --- | --- |\n| Vet managers understand the product from the explanation (comprehension) | 5 to 8 clinic managers from 2 to 6 vet practices, including recent wins and losses | They describe scheduling plus certification and on-call without expecting patient records or billing | They expect practice-management or clinical records; they call it a generic scheduler | Unassigned. Interview guide needed |\n| Certification gating is a difference (H2) | Live demo against Deputy and When I Work with a restricted-task shift | Competitors cannot block an uncertified assignment or model on-call rotas | Either competitor blocks the assignment or handles on-call natively | Unassigned. Run before any V1 claim goes live |\n| Vets' real alternatives (H1, H3) | Vet deals, won and lost, with competitors named | Paper or spreadsheets dominate; the chosen rival lacks V1 or V2 | Practice-management scheduling or a generic tool wins the same deals on equivalent features | Unassigned. Pull deal records by segment |\n| V2 matters commercially | Vet clinics asked how they cover emergency shifts today and what it costs them | Manual coverage is a recurring pain point | Current coverage is adequate and not raised as a concern | Unassigned |\n| Reachable market supports the goal (CEO concern, E11, E12) | Count of US independent 2 to 6 vet clinics; current vet customer count and contract values; the revenue target | The reachable count supports the target at a realistic win rate | Reachable count is too small for the target | Unassigned. CEO to supply the target |\n| Restaurant and long-tail positioning (E3, E8) | Restaurant win/loss reasons beyond price; churn reasons | A distinct value exists for restaurants | Value is only price and no distinct value | Unassigned. Separate canvases |\n\n**Revision rule:**\n- If competitors gate by certification (H2 fails), drop V1 as a difference. Lead with V2 and the on-call workflow, and re-run the frame comparison.\n- If vet deals are lost to practice-management scheduling (H3), revisit the alternative set and the frame, possibly toward integration.\n- If the reachable vet count cannot reach the revenue target, keep the vet frame and treat the target as the open question; do not widen the claim to every clinical or hourly workplace without evidence.\n- If vets do not recognize \"veterinary clinics\" as the right label, test \"veterinary hospitals\" (the customer quote) before changing the frame.\n\n**Unresolved gaps, in priority order:**\n1. Whether competitors already gate by certification and model on-call rotas. This can overturn V1.\n2. Which alternatives vet buyers actually consider, including practice-management software.\n3. Vet win/loss rate and reasons. The brief has none.\n4. Reachable market: independent 2 to 6 vet clinics in the US, and the revenue target.\n5. Segment mix of the 12 interviews, and how many named V1 and V2.\n6. Whether payroll export differentiates at all.\n\nSeek comprehension and choice evidence, not approval of the wording.\n\n## 9. Implementation and review\n\n- **Sales and demo implications:** Lead vet demos with a credential-restricted anesthesia-monitoring shift and an emergency rota. Keep payroll export as a supporting point. Stop using \"easiest scheduling app\" in vet pitches. Do not use the \"only scheduler\" quote until it is tested.\n- **Marketing implications:** Replace the barista hero with a vet clinic context. Rewrite the homepage around the subcategory and the two value themes. Keep restaurant messaging out of the vet pages. Build a separate restaurant canvas before writing restaurant copy.\n- **Delivery implications:** Vet onboarding likely needs certification data for each tech (hypothesis). Support and onboarding should be prepared for that step.\n- **Pricing and product questions:** Restaurant pricing against Deputy is a team decision, not a positioning output. Whether to serve restaurants and the long tail is a business decision. Any vet-specific product additions are proposals, not approved changes.\n- **Review triggers:** A competitor ships certification gating or on-call rotas; vet deal results change the alternative set; the revenue target or reachable-market count changes; restaurant decision is made.\n- **Next review:** After the competitor demo and comprehension interviews. No date or owner is given in the brief; **agreed date and owner unresolved**.\n\n## Review gates\n\n| Gate | Result | Note |\n| --- | --- | --- |\n| Present offering | Pass | Claims are current capabilities; no roadmap in brief |\n| Evidence integrity | Pass | Labels and limitations on each item; hypotheses marked |\n| Real alternatives | Fail | Vet-specific alternatives not established (H2, H3) |\n| Bounded differentiation | Fail | V1 and V2 parity against named competitors not checked |\n| Value trace | Pass | Each theme traces to a difference, action, and consequence; parity remains open |\n| Strong customer fit | Pass | Observable signals named; boundary stated; signals untested |\n| Viable focus | Fail | Reachable count and revenue target missing |\n| Frame comparison | Pass | Three frames compared |\n| Category risk | Pass | Subcategory of an existing category; no education burden |\n| Honest gap diagnosis | Fail | No win/loss data for vets; gap not checked against deals |\n| Messaging follows analysis | Pass | Explanation traces to canvas |\n| Validation | Fail | No buyer comprehension or choice evidence yet |\n| Operational handoff | Pass | Implications recorded; decisions unassigned |\n| Reviewability | Pass | Dated; uncertainty and triggers named |\n\n**Result: Provisional.** Five gates fail (real alternatives, bounded differentiation, viable focus, honest gap diagnosis, validation). The unresolved decisions and validation plan above are explicit, so this can be used for planning, but not as settled positioning.\n"}]}, "haiku-skill-2": {"cost_usd": 0.00451, "seconds": 22, "output_tokens": 3687, "files": [{"name": "positioning.md", "text": "# Shiftwise positioning\n\n## Recommendation\n\nPosition Shiftwise as the scheduling and time-tracking app for independent veterinary hospitals. Lead every surface (homepage, sales pitch, deck) with vets. Keep the general hourly-workforce product underneath as the platform, and treat restaurants as a later expansion, not the headline.\n\n## Positioning statement\n\nFor independent veterinary clinics with 2 to 6 vets that run on-call rotations and emergency shifts, Shiftwise is the scheduling and time-tracking app built for how a hospital runs. Unlike paper rotas and generic hourly schedulers, Shiftwise handles on-call and emergency coverage, blocks uncertified techs from anaesthesia monitoring, and exports hours to the payroll tools vets already use.\n\n## Why this holds up (from the brief)\n\n- **Revenue and retention favour it.** Vet clinics are 61% of revenue and churn at 2% a month. Restaurants are 15% of revenue and churn at 7%. Annualised, that is roughly 21% logo churn for vets against roughly 58% for restaurants.\n- **Customers already describe it this way.** \"It's the only scheduler that understands we're a hospital, not a shop\" is a usable line from two clinic managers. Use their words, not ours.\n- **The differentiators are specific.** On-call and emergency rota handling, vet-tech certification gating, and payroll export are concrete features a competitor would have to build deliberately.\n- **The current message does not match the customers.** \"The easiest scheduling app for hourly teams\" and a barista on the hero describe the 15% segment and a generic market. Shiftwise loses to Deputy on price in restaurants, which is what a generic message invites.\n\n## Competitive frame\n\n- **Paper rotas** are the real alternative for many vet clinics. The brief does not say how vets choose between tools, so this needs checking.\n- **When I Work, Deputy, 7shifts** appear in deals, but the brief mostly ties them to restaurants. 7shifts is restaurant-specific. None is described as serving vet hospitals.\n- Against generic tools, the claim is depth of clinical staffing rules, not ease of use.\n\n## Messaging direction\n\n- **Headline idea:** Scheduling built for veterinary hospitals.\n- **Hero image:** replace the barista with a clinic-relevant scene (a shift board or rota with on-call and certification badges). Keep it generic, since no brand assets are in the brief.\n- **Proof points:** certification-gated anaesthesia coverage, on-call rotation handling, payroll export, the \"hospital, not a shop\" quote with attribution.\n- **Avoid:** \"easiest,\" \"any hourly team,\" and price-led claims. Those are the messages that put Shiftwise against Deputy on price.\n\n## The CEO's concern (narrowing to vets)\n\nThe concern is reasonable but the brief points the other way on the core risk:\n\n- The 32,000 US vet clinics figure is the total number of clinics, not the number of independent 2 to 6 vet clinics, which is the actual target. The brief does not give that number, so the true market size is unknown.\n- Narrowing is a wedge, not a ceiling. Vets are where the product is strongest and stickiest. A generic message is what forces price competition with Deputy in the segment that churns most.\n- Restaurants stay possible later, with their own positioning and proof. Expanding from a strong wedge is easier than broadening a weak generic position.\n\n## Gaps to close before committing\n\n1. **Segment size.** Count independent vet clinics with 2 to 6 vets in the US and estimate how many fit the profile.\n2. **Customer count and average revenue per account.** Neither is in the brief, so the penetration needed to reach a given ARR cannot be calculated.\n3. **Win and loss data for vets.** The brief has restaurant loss reasons (price against Deputy). It needs the same for vet deals, including who vets compare Shiftwise against.\n4. **Which feature drives the decision.** Is it on-call handling, certification gating, or payroll export? Interviews should answer this before the messaging is finalised.\n5. **Whether non-vet customers value certification rules.** That determines whether the clinical-compliance angle can travel to other sectors later.\n"}]}, "haiku-skill-3": {"cost_usd": 0.01783, "seconds": 101, "output_tokens": 21576, "files": [{"name": "positioning.md", "text": "# Shiftwise positioning canvas\n\n**Status: Provisional.** Built from brief.md only. No win/loss records, pricing, demos, or buyer testing were supplied. See the review gates at the end.\n\n**Working positioning:** Shiftwise is scheduling and time tracking for veterinary clinics. It covers on-call and emergency shifts, keeps credentialed staff off procedures they aren't certified for, and sends hours to the payroll tools clinics already use.\n\n## 1. Scope and status\n\n- **Offering and current version:** Shiftwise as sold today. The brief does not give a version number. The capabilities below are described in the brief as existing.\n- **Decision supported:** Rewrite of the product description: website hero, sales pitch, and the category used in deals.\n- **Prepared on:** 2026-10-08\n- **Status:** Provisional. Analysis rests on one brief and 12 vet interviews. Nothing has been tested with buyers.\n- **Current category and historical assumption:** \"The easiest scheduling app for hourly teams.\" The product was built for any hourly workforce, and the pitch and homepage still reflect that. The homepage hero shows a barista.\n- **Evidence boundary:** Company figures (3 years old, 40 employees, USD 3.1M ARR) and the revenue split have no stated date. The churn figures have no stated period. The 12 interviews have no stated dates or sample mix. No win/loss counts exist.\n- **Future capabilities:** None listed in the brief. None are used in this canvas.\n\n## 2. Evidence register\n\n| ID | Observation or claim | Source | Label | Limitation |\n| --- | --- | --- | --- | --- |\n| E1 | 61% of revenue from independent vet clinics (2-6 vets); 15% restaurants; the rest retail and gyms | Company figures, undated | Observed | Revenue share only. Says nothing about logo counts, margin, or growth. |\n| E2 | Monthly logo churn: vet clinics ~2%, restaurants ~7% | Company figures, period not stated | Observed | Shows the gap, not the cause. |\n| E3 | Vets value on-call rotation and emergency shift handling | 12 interviews | Observed | Interview-reported. Frequency and sample mix unknown. |\n| E4 | Tracks vet-tech certifications so an uncertified tech can't be scheduled for anesthesia monitoring | Product description and interviews | Observed (interview); feature to confirm by demo | Does not show how often this prevents a real error. |\n| E5 | Exports hours to the payroll tools vets use | Product description and interviews | Observed (interview) | Time saved not measured. Which payroll tools is not stated. |\n| E6 | Two clinic managers: \"It's the only scheduler that understands we're a hospital, not a shop.\" | 2 interviews | Observed | Two quotes. Not checked against competitor products. |\n| E7 | Competitors seen in deals: When I Work, Deputy, 7shifts (restaurants), paper rotas | Deal experience | Observed | No frequencies. Not split by segment except 7shifts. |\n| E8 | Losses to Deputy in restaurants, on price | Deal experience | Observed | Restaurants only. No count. |\n| E9 | Current pitch: \"the easiest scheduling app for hourly teams\" | Current marketing | Observed | Shows what is said, not how buyers respond. |\n| E10 | Homepage hero shows a barista | Current website | Observed | Visual signal only. Not tested. |\n| E11 | ~32,000 US vet clinics | CEO | Observed as a clinic count | A count, not a revenue market. No fit share or price. |\n| H1 | Generic schedulers (When I Work, Deputy, 7shifts) cannot enforce vet credential rules or on-call logic | Hypothesis | Hypothesis | Unverified. The core of the differentiation claim depends on it. |\n| H2 | Clinics' current alternative for on-call and credentialed scheduling is paper, or a practice-management system's scheduling module | Hypothesis | Hypothesis | The brief does not mention practice-management software. |\n| H3 | Vet value drives the 2% churn | Hypothesis | Hypothesis | Correlation implied. Churn reasons not supplied. |\n\n**Unknowns that could overturn the position:** vet win rate and vet loss reasons; price and revenue per clinic by segment; whether all vet customers use on-call and credential features; how practices with 7 or more vets fare; whether restaurant losses are price-only.\n\n## 3. Buyer situation and competitive alternatives\n\n**Buying situation:** A clinic manager at an independent vet clinic with 2-6 vets needs to staff on-call and emergency shifts, keep credentialed techs on procedures that require certification, and get hours into payroll. The brief quotes clinic managers but does not identify the buyer, the users, or the approval process.\n\n| Alternative | Why the buyer keeps or considers it | Where it falls short for this situation | Evidence |\n| --- | --- | --- | --- |\n| Paper rotas | Familiar, free, no setup | Likely no credential rules and no hours export (inherent to paper) | E7, E3 |\n| When I Work / Deputy | Established, generic, usually cheaper than specialist tools | Unverified for vet credential and on-call logic (H1). Deputy wins on price in restaurants. | E7, E8, H1 |\n| Practice-management scheduling module (if used) | Already in the clinic's workflow | Unknown whether it covers on-call or credentials | H2 |\n| Current workflow with manual payroll entry | Works for some clinics | Hours re-keyed by hand | Hypothesis; needs interview evidence |\n\n**Excluded theoretical rivals:** 7shifts is restaurant-specific and stays in the restaurant comparison only. It is not part of the vet consideration set on the evidence supplied.\n\n**Separate buying situations:** Vet clinics and restaurants have different alternatives and different value. They need separate comparisons. The restaurant comparison is a price fight against Deputy and 7shifts. The vet comparison is about on-call and credentials.\n\n## 4. Capability-to-value map\n\n| ID | Actual alternative | Demonstrable difference | What it enables | Buyer consequence / theme | Evidence and confidence |\n| --- | --- | --- | --- | --- | --- |\n| V1 | Paper rotas; generic schedulers (H1) | Builds on-call and emergency rotations in the schedule | Covers after-hours and emergency shifts without rebuilding the rota | Emergency coverage is kept running. **Theme A** | E3. Confidence medium. The consequence is not measured, and the generic-tool comparison is unverified. |\n| V2 | Paper rotas; generic schedulers (H1) | Tracks vet-tech certifications and blocks uncertified staff from anesthesia monitoring | Staffs procedures only with qualified people | Fewer unqualified assignments on high-risk procedures. **Theme B** | E4, E6. Confidence medium. No incident frequency. Generic-tool comparison unverified. |\n| V3 | Manual re-keying of hours | Exports hours to the payroll tools vets use | Hours reach payroll without re-keying | Less admin and fewer payroll errors. **Theme C** | E5. Confidence low to medium. Time saved not measured. Likely table stakes if generic tools also export. |\n\n**Primary value themes:**\n- **A. Covers the on-call and emergency shifts a clinic actually runs.** The lead theme.\n- **B. Keeps credentialed staff on the procedures that require them.** The strongest differentiator, if H1 holds.\n- **C. Hours go straight to payroll.** A supporting theme, not a lead.\n\n**Discarded claims:**\n- \"Easiest scheduling app for hourly teams.\" Generic, and competitors make similar claims. No ease measurement exists.\n- Barista hero image. Unrelated to the vet value.\n- \"Any hourly workforce\" as the lead. It dilutes Themes A and B.\n- Numerical outcomes (time or money saved). None are supplied, so none are used.\n- Roadmap items. None are supplied.\n\nThe clinic quote in E6 is useful as evidence of the theme. It is not proof of a competitor gap.\n\n## 5. Best fit customers and boundaries\n\n- **Target segment:** Independent veterinary clinics with 2-6 vets and hourly staff, including techs.\n- **Observable fit signals:**\n  - Runs on-call or after-hours emergency coverage across vets and techs.\n  - Has staff certifications that determine which shifts they can work, such as anesthesia monitoring.\n  - Pays staff through a payroll tool that accepts hours exports.\n- **Why they care intensely:** Each signal maps to a costly failure: uncovered emergencies (V1), unqualified staffing on high-risk procedures (V2), and manual payroll work (V3).\n- **Examples in the evidence:** E1, E2, E3, E6. The 61% revenue share and 2% churn show the segment already pays and stays.\n- **Poor fit boundary:**\n  - Clinics with no on-call or emergency coverage.\n  - Businesses without credentialed-task requirements.\n  - Single-vet practices and practices with 7 or more vets or multiple sites. The brief does not show how the product performs there, so treat this as an open boundary.\n  - Restaurants, retail, and gyms, where the credential and on-call needs are not evidenced.\n- **Reach and commercial viability:** Roughly 32,000 US vet clinics (E11) is a clinic count, not a revenue market. Answering the CEO's concern that a vet focus is too small requires three numbers the brief does not contain:\n  1. How many US clinics match the 2-6 vet profile with on-call and credentialed staff.\n  2. Current revenue per vet clinic.\n  3. Vet win rate and sales cycle.\n  The 61% revenue share and low churn suggest the segment works. Whether it supports the growth goal is untested.\n- **Other segment considered:** Restaurants (15% of revenue, ~7% churn, price-led competition from Deputy and 7shifts). Keep them served, but do not lead with them. The brief gives no evidence that credentials or on-call matter for restaurants. The long tail of retail and gyms has no evidence at all.\n\n**On the narrowing concern:** Leading with vets narrows the message, not the product. The product can keep serving restaurants. Whether a vet-led business reaches enough revenue is a question the sizing data above must answer.\n\n## 6. Market frame comparison and decision\n\n| Candidate frame | Expected features | Expected competitors and buyer | Price/channel assumptions | Visibility of our value | Misleading expectations and education burden |\n| --- | --- | --- | --- | --- | --- |\n| **A. Scheduling app for hourly teams** (current) | Shift templates, swaps, time clock | When I Work, Deputy, 7shifts, paper; any hourly manager | Low per-user SaaS; price-sensitive (E8) | V1-V3 peripheral or hidden | Buyers compare on price with Deputy. The restaurant-style hero sets the wrong expectation. Low education burden, but the value stays invisible. |\n| **B. Veterinary clinic scheduling** (a subcategory of workforce scheduling) | On-call rotations, credential rules, payroll export | Generic schedulers, paper, possibly practice-management modules (H2); clinic manager or owner | Per-clinic pricing (assumption) | V1-V3 central | \"Veterinary-only\" may read as too narrow (the CEO's concern). Vets may assume larger hospitals are served. Low education burden, since clinics already think in these terms. |\n| **C. Clinical staff scheduling** (hospital workforce) | Credential compliance, on-call, staffing rules | Healthcare workforce tools; hospital administrator | Enterprise-style assumptions | V2 central, V1 partly visible | Buyers expect depth the product does not have. Proof exists only for vets. High education burden. |\n\n- **Selected frame:** Scheduling for veterinary clinics, a subcategory of workforce scheduling.\n- **Framing style:** Win a subcategory. Clinics already recognize the context, so the buyer does not need a new category explained.\n- **Why it wins this comparison:** It makes Themes A and B visible and sets the right expectations: on-call, credentials, and payroll. Vets should expect exactly these features.\n- **Principal rejected frame:** Frame A. It hides the three themes and invites price comparison with Deputy.\n- **Decision confidence:** Moderate on direction, since the vet evidence is the strongest in the brief (revenue, churn, interviews). Low on two points: whether the subcategory is big enough, and whether \"hospital\" language stretches beyond 2-6 vet practices. Frame C is rejected for now because it has no evidence.\n- **Wording question:** Clinic managers call their clinics \"hospitals.\" Copy could say \"veterinary clinics\" or \"veterinary hospitals.\" Choose after testing whether \"hospital\" attracts practices the product does not serve well.\n\n## 7. Brief buyer-facing explanation\n\n> Shiftwise is scheduling and time tracking built for veterinary clinics. It handles on-call rotations and emergency shifts, keeps uncertified staff off procedures like anesthesia monitoring, and sends hours to the payroll tools your clinic already uses.\n\n**Claim trace:**\n- \"Scheduling and time tracking built for veterinary clinics\": E1, E3, E6.\n- \"On-call rotations and emergency shifts\": V1, E3.\n- \"Keeps uncertified staff off procedures like anesthesia monitoring\": V2, E4.\n- \"Sends hours to the payroll tools your clinic already uses\": V3, E5.\n\n**Scope or important limitation:** Shiftwise was built for any hourly workforce and still serves restaurants and other businesses. The veterinary focus leads the story; it is not exclusive. The sentence deliberately makes no claim that generic schedulers lack these features, because H1 is unverified.\n\nThis explanation is a messaging translation. It is not a substitute for the analysis above.\n\n## 8. Validation and revision plan\n\n| Question or hypothesis | Suitable buyer/deal evidence | What would support it | What would contradict it | Owner and next action |\n| --- | --- | --- | --- | --- |\n| **Comprehension:** Do clinic managers hear this as a vet-clinic tool with on-call and credential features? | 6-8 clinic managers outside the 12 interviews | They describe on-call and credentials without prompting | They assume a restaurant or general hourly tool | Unassigned. Run the test before the homepage changes. |\n| **Alternatives:** What do clinics use today, and do they have a practice-management scheduling module? (H2) | Same clinics, plus recently churned vet clinics | Paper or spreadsheets, with no on-call or credential handling | A practice-management module already enforces credentials and on-call | Unassigned |\n| **Value:** Has a credential or on-call failure, or manual payroll work, recently cost the clinic time or money? | Existing vet customers; sales team | Specific recent incidents | Problems are rare and handled fine by hand | Unassigned |\n| **Competitor gap (H1):** Can When I Work and Deputy enforce credential rules and on-call logic for a vet clinic? | Hands-on trial of each tool with a vet-clinic setup | They cannot, or only with workarounds | They handle it equally well | Unassigned |\n| **Win/loss:** Why do vet clinics buy, and why do they lose? | Vet deals from the last 12 months; churn reasons from vet clinics that left | Wins cite on-call, credentials, or payroll | Losses cite price or missing features | Unassigned. Pull the data first. |\n| **Restaurant losses:** Are the Deputy losses purely about price? | Restaurant losses | Price is the only reason | Restaurants also value the same themes | Unassigned |\n| **Sizing:** Does the reachable vet pool support the growth goal? | Clinic counts, revenue per vet clinic, win rate | Reachable pool times revenue per clinic supports the target | The pool is too small at current prices | Unassigned. Needs the CEO's target. |\n\n**Revision rule:**\n- If comprehension fails, revise the frame and copy first. Do not change features.\n- If a practice-management module covers on-call and credentials, Themes A and B narrow. Re-test them against that alternative before leading with them.\n- If the trial shows generic tools enforce credential rules, drop B as the differentiator and lead with A.\n- If vet losses are about price, revisit packaging and pricing. Do not change the positioning.\n- If sizing fails, keep the vet frame as the core and test an adjacent segment separately. Do not broaden to \"hourly teams\" by default.\n\n**Unresolved gaps, in priority order:**\n1. Do vet deals win on Themes A-C, or on something else?\n2. Is a practice-management scheduling module the real alternative?\n3. Does the 2-6 vet profile hold, or do larger practices buy too?\n4. Is the reachable pool big enough at current pricing?\n\n## 9. Implementation and review\n\n- **Sales and demos:** Open every vet demo with on-call and emergency coverage, then show the anesthesia credential block live. Payroll export is a supporting step. A demo confirms a capability, not an outcome.\n- **Marketing:** Replace \"easiest scheduling app for hourly teams\" with the vet-clinic positioning. Replace the barista hero with a clinic image, chosen after testing. Keep restaurants reachable, but not in the lead.\n- **Delivery and support:** Vet onboarding for on-call templates and credential setup. Identify which payroll tools clinics use and whether integrations are needed.\n- **Pricing and product questions (proposals, not approved):**\n  - Should restaurant features stay in primary navigation?\n  - Is vet-specific packaging needed to answer Deputy on price?\n  - Does the product fit practices with 7 or more vets?\n- **Review triggers:**\n  - A competitor adds credential or on-call handling.\n  - Vet churn rises above ~2% per month.\n  - Win/loss shows vet losses driven by price.\n  - The team decides to pursue restaurants again.\n  - A practice-management integration changes the alternatives.\n- **Next review:** After validation items 1-4 are complete. No owner or date is set; name both.\n\n## Review gates\n\nResult: **Provisional.** Five gates fail, so this canvas should not be treated as established positioning.\n\n| Gate | Result | Note |\n| --- | --- | --- |\n| Present offering | Pass, pending demo | Capabilities are described as current. Confirm by demo. |\n| Evidence integrity | Pass | Labels applied. Hypotheses separated from observations. |\n| Real alternatives | **Fail** | Practice-management alternative (H2) is unknown. |\n| Bounded differentiation | **Fail** | Differentiation against generic tools rests on H1, which is unverified. |\n| Value trace | **Fail** | Chain is explicit, but outcomes are unmeasured and the alternative comparison is untested. |\n| Strong customer fit | Pass, with caveat | Based on 12 interviews. Fit boundary at 7+ vets is open. |\n| Viable focus | **Fail** | Commercial viability not established. Sizing data missing. |\n| Frame comparison | Pass | Three frames compared. |\n| Category risk | Pass | Subcategory with low education burden. |\n| Honest gap diagnosis | **Fail** | No win/loss review yet. |\n| Messaging follows analysis | Pass | Every claim in the buyer explanation traces to a V or E ID. |\n| Validation | **Fail** | No buyer testing done. |\n| Operational handoff | Pass, partial | Implications recorded. Pricing decisions open. |\n| Reviewability | Pass | Dated, with uncertainty and triggers named. |\n"}]}, "opus-1": {"cost_usd": 0.1588, "seconds": 60, "output_tokens": 4961, "files": [{"name": "positioning.md", "text": "# Shiftwise Positioning\n\n*Draft 1, 2026-10-08. Based only on `brief.md`. Anything not in the brief is marked as an assumption or an open question.*\n\n---\n\n## The recommendation in one paragraph\n\nStop positioning Shiftwise as \"the easiest scheduling app for hourly teams.\" Position it as **the staff scheduling system built for veterinary clinics**: software that handles on-call rotations, emergency shifts, and certification rules, and sends hours to vet payroll. Your vet customers already see it this way. They bring in 61% of revenue, they churn at less than a third of the restaurant rate, and they describe the product in words no competitor can claim: *\"the only scheduler that understands we're a hospital, not a shop.\"* The generic pitch puts you head to head with Deputy and When I Work on price, a fight you're already losing. The vet pitch puts you in a category where your real competitor is a paper rota.\n\n---\n\n## 1. What the data says\n\n| | Vet clinics | Restaurants | Retail / gyms |\n|---|---|---|---|\n| Share of revenue | **61%** (~$1.9M) | 15% (~$0.47M) | ~24% (~$0.74M) |\n| Monthly logo churn | **2%** | 7% | not given |\n| Annual equivalent* | **~22% lost per year** | **~58% lost per year** | \u2014 |\n| Main competitor | Paper rotas, generic tools | 7shifts, Deputy (we lose on price) | When I Work, Deputy |\n| Why they buy | On-call, emergencies, cert-aware scheduling, vet payroll export | Not documented | Not documented |\n\n*Compounded: 1 \u2212 0.98\u00b9\u00b2 \u2248 22%; 1 \u2212 0.93\u00b9\u00b2 \u2248 58%.\n\nWhat this means: a restaurant customer stays about 14 months on average, a vet clinic about 50 (1 \u00f7 monthly churn). Before any difference in price, a vet clinic is worth about 3.5\u00d7 as much over its lifetime. The product is already built for vets. Only the marketing hasn't caught up.\n\n---\n\n## 2. Positioning canvas\n\n### Competitive alternatives (what vets would use if Shiftwise didn't exist)\n- **Paper rotas, whiteboards, spreadsheets.** This is the real default for a 2\u20136 vet practice.\n- **Generic shift schedulers** (When I Work, Deputy). They handle shifts but know nothing about on-call, emergencies, or who is qualified to do what.\n- **The practice manager's memory.** Someone just *knows* which tech can monitor anaesthesia. That works until it doesn't.\n\n### Unique attributes (what we have that the alternatives don't)\n1. **On-call and emergency shift handling.** Built-in rotations, not shifts squeezed into a retail template.\n2. **Certification-aware scheduling.** An uncertified vet tech *can't* be scheduled for anaesthesia monitoring. The system enforces it, so nobody has to remember.\n3. **Payroll export to the tools vet clinics already use.**\n\n### Value (why those attributes matter to a clinic)\n| Attribute | Value to the clinic |\n|---|---|\n| On-call and emergency handling | Night and weekend cover is always staffed and always fair, with no rebuilding the rota every time an emergency comes in |\n| Certification enforcement | **Patient safety and compliance.** The wrong person is never on the anaesthesia monitor. This is the strongest point because the stakes are clinical, not just admin. |\n| Payroll export | Hours go straight to payroll without re-keying. The practice manager gets time back at month end. |\n\n### Best-fit customer\n**Independent veterinary clinics with 2\u20136 vets** that run on-call or emergency cover and have a practice manager who builds the rota. This is the segment with 2% churn and the \"hospital, not a shop\" quote.\n\n### Market category\n**Staff scheduling for veterinary clinics.** Say it plainly. Don't invent a new category. A practice manager should know within five seconds that this is built for them.\n\n---\n\n## 3. Messaging\n\n### Positioning statement (internal)\n> For independent veterinary clinics that have to cover on-call nights, emergencies, and clinical roles that need certification, Shiftwise is the staff scheduling and time-tracking system built for how vet hospitals actually run. Generic schedulers like Deputy and When I Work treat a clinic like a shop. Shiftwise handles on-call rotations, stops uncertified staff from being booked for clinical tasks like anaesthesia monitoring, and exports hours straight to vet payroll.\n\n### Homepage hero\n**Headline options** (test these against each other):\n1. **Scheduling that knows you're a hospital, not a shop.** *(customer language, recommended)*\n2. Staff scheduling built for veterinary clinics.\n3. On-call, emergencies, and certifications. Finally, a rota that gets it.\n\n**Subhead:**\n> On-call rotations, emergency cover, and certification-aware scheduling for vet clinics, with hours exported straight to payroll.\n\n**Hero image:** replace the barista with a vet tech or practice manager in a clinic. The barista tells every vet visitor \"this isn't for you.\"\n\n### Three proof pillars (homepage sections and sales deck)\n1. **Cover every night and weekend.** On-call rotations and emergency shifts are built in.\n2. **The right person on every procedure.** Certification tracking blocks uncertified staff from roles like anaesthesia monitoring.\n3. **Payroll without re-keying.** Export hours to the payroll tools your clinic already uses. *(Name the specific tools here. The brief doesn't list them.)*\n\n### Sales one-liner (replaces \"the easiest scheduling app for hourly teams\")\n> \"Shiftwise is the scheduler built for vet clinics. It runs your on-call rota, makes sure only certified techs are booked for anaesthesia, and sends hours straight to payroll.\"\n\n### Handling objections\n- **\"We just use a spreadsheet / paper.\"** \u2192 \"Does your spreadsheet stop an uncertified tech being put on anaesthesia monitoring? Ours does.\"\n- **\"Deputy / When I Work is cheaper.\"** \u2192 \"They're built for shops and caf\u00e9s. Ask them how they handle an on-call rotation or a certification rule.\" *(Compete on fit, not price.)*\n\n### Words to stop using\n- \"Hourly teams\" / \"hourly workforce.\" Vets don't think of their clinic that way.\n- \"Easiest.\" It's a generic claim every competitor makes, and it says nothing about what makes you different.\n\n---\n\n## 4. The CEO's concern: \"Is the vet market too small?\"\n\nIt's a reasonable worry, and the numbers answer it.\n\n- **~32,000 vet clinics in the US.** You currently earn ~$1.9M from vets.\n- **Illustration only. Plug in your real figures:** if the average clinic pays ~$3,000/year (assumed, not in the brief), you have ~630 vet clinics, about **2% of the US market**. Reaching 20% share would be ~6,400 clinics, or **~$19M ARR from US vets alone**, about 6\u00d7 today's *total* company revenue.\n- Whatever the real price, the conclusion holds unless your vet share is already far higher than it looks: **you are nowhere near the ceiling.** The constraint today is reaching vets, not running out of them.\n\n**Narrowing the positioning doesn't mean narrowing the product or turning customers away:**\n- Existing restaurant, retail, and gym customers keep the product and keep getting support. You simply stop *spending acquisition money* on segments where you lose on price and churn at 7% a month.\n- The thing vets value (\"a hospital, not a shop\": clinical roles, on-call, certifications) points to natural **next segments** once vets are won. Examples are dental practices, physio and outpatient clinics, and veterinary practices outside the US. *These are hypotheses to validate, not findings from the brief.*\n- Owning a niche is how you earn the right to expand. Being \"the easiest scheduler for everyone\" means you're competing on price with Deputy everywhere.\n\n---\n\n## 5. What to do next\n\n1. **Rewrite the homepage** with the hero, subhead, and pillars above. Swap the barista image.\n2. **Retrain sales** on the new one-liner and objection handling. Point outbound and paid acquisition at vet clinics.\n3. **Put restaurants in maintenance mode.** Keep serving them, but stop marketing to them.\n4. **Fill the gaps** before final sign-off (open questions below).\n5. **Measure it:** vet share of new logos, homepage-to-demo conversion from vet traffic, and win rate against paper rotas and Deputy in vet deals.\n\n### Open questions (not answered by the brief)\n- How many vet clinic customers do we have, and what is the vet ARPA? (This makes the market-size math exact.)\n- Which payroll tools do we export to? Name them in the copy.\n- Which competitors actually show up in *vet* deals, and what's our win rate there?\n- Were all 12 interviews with vets? Only 2 managers gave the \"hospital, not a shop\" quote, so test the headline before betting the brand on it.\n- Do larger or emergency-only animal hospitals fit as well as 2\u20136 vet clinics?\n"}]}, "opus-2": {"cost_usd": 0.18306, "seconds": 70, "output_tokens": 5128, "files": [{"name": "positioning.md", "text": "# Shiftwise Positioning\n\n*Based on brief.md. Numbers marked **[assumption]** aren't in the brief and need checking before anyone relies on them.*\n\n---\n\n## The recommendation in one line\n\n**Stop selling Shiftwise as \"the easiest scheduling app for hourly teams.\" Sell it as the scheduling and time-tracking system for independent veterinary hospitals.**\n\nThe business has already chosen vets. 61% of revenue comes from them, and they churn at a third of the restaurant rate. The positioning hasn't caught up: the pitch and homepage still describe a generic tool for a barista, which is the one market where you lose on price.\n\n---\n\n## Why vets, from the evidence in the brief\n\n| Signal | Vet clinics | Restaurants |\n|---|---|---|\n| Share of revenue | 61% | 15% |\n| Monthly logo churn | 2% (~22% a year) | 7% (~58% a year) |\n| Competitive position | You win: the alternatives are generic tools and paper rotas | You lose to Deputy on price, and 7shifts is built for restaurants |\n| What customers say | \"The only scheduler that understands we're a hospital, not a shop.\" | Nothing in the brief |\n\nThe features vets praise (on-call rotations, emergency shifts, certification-gated scheduling, payroll exports) are not \"easy scheduling.\" They're clinical-workforce features, and the current message hides them.\n\n---\n\n## Positioning components\n\n### 1. Competitive alternatives (what vets would use if you didn't exist)\n- **Paper rotas, whiteboards and spreadsheets.** The most common alternative, and the one to beat first.\n- **Generic shift schedulers** (When I Work, Deputy). Fine for a shop. They don't model on-call, emergencies or who is qualified to do what.\n- **The practice manager's memory.** One person carries the rota and the compliance knowledge in their head.\n\n### 2. Unique attributes (what Shiftwise has that they don't)\n- **On-call and emergency shift handling** built in, not worked around.\n- **Certification-aware scheduling.** Shiftwise tracks vet-tech credentials and won't let an uncertified tech be scheduled for anaesthesia monitoring.\n- **Payroll export to the tools vet clinics already use.**\n- (Plus the basics: scheduling and time-tracking that staff actually use.)\n\n### 3. Value (what those attributes mean for a clinic)\n| Attribute | Value to the clinic |\n|---|---|\n| On-call and emergency shifts | The rota matches how a hospital actually runs, including nights, weekends and surprises, so you're not juggling a separate on-call spreadsheet. |\n| Certification-gated scheduling | No unqualified tech on an anaesthesia monitor. Patient safety and compliance are enforced by the schedule, not by someone remembering. |\n| Payroll export | Hours go to payroll without re-keying, so less admin and fewer errors. |\n| **Overall** | **A rota you can trust: covered, compliant, and paid correctly, without the practice manager holding it all together.** |\n\n### 4. Target customer (who cares most)\n**Independent veterinary clinics with 2\u20136 vets** that run on-call or emergency coverage and employ credentialed vet techs. The buyer is usually the **practice or hospital manager**, often with the owner-vet signing off.\n\nBest-fit signals: a paper or spreadsheet rota today, after-hours or on-call duty, multiple tech certification levels, and a manager who spends hours a week on scheduling.\n\n### 5. Market category\n**Scheduling built for veterinary hospitals.** In other words, staff scheduling for clinical teams, not shift scheduling for retail.\n\nThis category frames the right comparison. Next to \"easy scheduling app,\" you're a cheaper or pricier Deputy. Next to \"built for vet hospitals,\" Deputy is the generic tool that doesn't understand clinical work. The customers' own phrase, *\"a hospital, not a shop,\"* is the category in their words.\n\n### 6. Relevant trends (use lightly)\n- Pressure on vet-tech and vet staffing makes every scheduled hour and every qualified person count. **[assumption: widely reported industry issue; confirm with customers before you lean on it]**\n- More clinics are offering extended hours and urgent care, which means more on-call complexity. **[assumption]**\n\n---\n\n## Positioning statement (internal, not copy)\n\n> For **independent veterinary clinics** that juggle on-call rotations, emergencies and credentialed staff, **Shiftwise** is the **scheduling and time-tracking system built for vet hospitals**. Unlike generic shift apps like When I Work and Deputy, or paper rotas, Shiftwise understands clinical work. It schedules on-call and emergency cover, stops uncertified techs from being put on tasks like anaesthesia monitoring, and sends hours straight to vet payroll.\n\n---\n\n## Messaging\n\n**Homepage hero (options)**\n1. **Scheduling built for vet hospitals, not coffee shops.**\n   On-call rotations, emergency shifts and certification-aware rotas, with hours exported straight to payroll.\n2. **Your clinic is a hospital. Your rota should know it.**\n3. **The right people on every shift, including the 2 a.m. emergency.**\n\n**Hero image:** swap the barista for a vet tech or a clinic team, ideally a real customer.\n\n**Sales one-liner (replacing \"easiest scheduling app for hourly teams\"):**\n> \"Shiftwise is the scheduler built for independent vet clinics. It handles on-call and emergency shifts, makes sure only certified techs are scheduled for things like anaesthesia monitoring, and sends hours straight to your payroll.\"\n\n**Proof points to collect next:** named clinic testimonials (the two \"hospital, not a shop\" managers are first in line), hours saved per week on scheduling, retention (\"clinics stay\": 2% monthly churn), and the list of supported payroll integrations.\n\n**Messages to drop:** \"easiest,\" \"any hourly team,\" and \"for hourly workforces.\" They invite a price comparison with Deputy and erase your differentiators.\n\n---\n\n## Answering the CEO's \"too small\" concern\n\n**1. The niche is large relative to where you are today.**\nThere are ~32,000 US vet clinics. Vet revenue today is about **$1.9M** (61% of $3.1M). The brief doesn't give price or customer count, so here is an illustration: at **$3,000 per clinic per year [assumption]**, that's ~630 clinics, about **2% of the market**. Reaching 10% at the same price would be ~3,200 clinics and roughly **$9.6M ARR from vets alone**. Plug in the real ACV. Unless pricing is very high, the headroom is several times current revenue.\n\n**2. Caveat: not all 32,000 are independent.** Corporate groups own a meaningful share of US clinics **[assumption: check the current consolidator share]**, and they buy differently. Size the independent segment properly before you set targets.\n\n**3. Positioning narrowly doesn't mean refusing other customers.** Restaurants, retail and gyms can still sign up and be served. You're choosing who the message, roadmap and sales effort are *for*, not who is allowed to buy.\n\n**4. The niche is a beachhead, with logical next steps once it's won:**\n- Vet specialty and emergency/referral hospitals, where on-call needs are even more intense\n- Vet clinics in other English-speaking markets (Canada, UK, Australia) **[check payroll integrations]**\n- Other small clinical teams with the same \"hospital, not a shop\" problem: credentialed staff, on-call, compliance (e.g. dental, physio, small urgent-care) **[hypothesis: validate before you invest]**\n\n**5. Staying broad is the bigger risk.** In the broad market you're a small player losing to Deputy on price and to 7shifts on fit. In vets you win, and customers stay more than three times longer (roughly 50 months on average vs. 14).\n\n---\n\n## What to do with restaurants and the long tail\n\n- **Keep serving existing customers.** Don't churn them on purpose.\n- **Stop spending on acquiring them:** no restaurant-targeted ads, no discounting to match Deputy.\n- **Watch whether the long tail has its own \"clinical\" pockets** (e.g. physio studios inside gyms) that fit the positioning.\n\n---\n\n## What changes, and in what order\n\n1. **Sales pitch and deck:** new one-liner, lead with on-call and certification, open with vet customer stories. *(This week.)*\n2. **Homepage:** new hero, vet imagery, feature sections reordered (on-call \u2192 certifications \u2192 payroll \u2192 basics). *(Next 2\u20134 weeks.)*\n3. **Proof:** turn the 12 interviews into 3\u20135 case studies or quotes, with permission.\n4. **Channels:** vet practice-manager associations, vet conferences and vet payroll or PIMS partners, instead of generic SMB channels. **[assumption: map the specific ones]**\n5. **Roadmap filter:** prioritise anything that deepens the \"hospital, not a shop\" advantage.\n\n---\n\n## Open questions to answer before committing numbers\n\n- Actual ACV and customer count by segment (to replace the sizing illustration above).\n- How many of the 32,000 clinics are independent with 2\u20136 vets.\n- Which payroll and practice-management systems vets use, and which you already integrate with.\n- Does acquisition cost differ between vets and restaurants? If vets are cheaper to acquire as well as stickier, the case gets stronger.\n- Willingness to pay: with a specialised product, is there room to raise vet pricing?\n"}]}, "opus-3": {"cost_usd": 0.16326, "seconds": 55, "output_tokens": 5164, "files": [{"name": "positioning.md", "text": "# Shiftwise Positioning\n\n*Draft, 2026-10-08. Source: brief.md (company metrics, 12 customer interviews, deal notes).*\n\n---\n\n## The one-line version\n\n**Shiftwise is staff scheduling built for veterinary hospitals: on-call rotations, emergency cover, and credential-aware scheduling, so the right person is always on the rota.**\n\nWe currently say \"the easiest scheduling app for hourly teams.\" That puts us in the same category as When I Work and Deputy, where the buyer compares on price, and Deputy wins on price. Our best customers don't buy us because we're easy. They buy us because we understand they run a hospital.\n\n---\n\n## 1. Why change\n\nThe data in the brief points one way:\n\n| | Vet clinics | Restaurants | Long tail (retail, gyms) |\n|---|---|---|---|\n| Share of revenue | **61%** | 15% | ~24% |\n| Monthly logo churn | **2%** (~22%/yr) | 7% (~58%/yr) | not given |\n| Why they buy | On-call, emergency shifts, certification rules, vet payroll exports | Generic scheduling | Generic scheduling |\n| Competitive position | Few real alternatives (generic tools, paper) | Lose to Deputy on price; 7shifts is purpose-built for them | Commodity |\n\n- Most of our revenue already comes from vets, even though we market to everyone.\n- Vets stay. Restaurants churn about 3.5x faster, so a restaurant customer lasts about 14 months on average and a vet clinic about 50.\n- Vets explain why they chose us, and their reasons are features generic tools don't have. Restaurants have 7shifts built for them and Deputy undercutting us. We have no edge there.\n- Customers are already saying it for us: *\"It's the only scheduler that understands we're a hospital, not a shop.\"*\n\nOur current homepage, with a barista in the hero and a generic headline, is aimed at the segment that churns most and that we lose on price.\n\n---\n\n## 2. Positioning canvas\n\n### Competitive alternatives (what a vet clinic would use if we didn't exist)\n1. **Paper or spreadsheet rotas.** This is the most common real alternative. It's free and familiar, but it breaks on on-call swaps and nobody checks certifications.\n2. **Generic hourly schedulers (When I Work, Deputy).** These handle shifts and clock-in well. They don't model on-call or emergency cover, don't know which tech is certified for what, and see a clinic as a shop.\n3. **Practice-management software (PIMS) scheduling modules.** *Not mentioned in the brief. Check in sales calls whether this comes up.*\n\n### Unique attributes (what we have that they don't)\n- **On-call and emergency-shift rotations** as a built-in concept, not workarounds.\n- **Credential-aware scheduling.** Vet-tech certifications are tracked, and the scheduler blocks an uncertified tech from being assigned to tasks like anaesthesia monitoring.\n- **Payroll exports to the tools vet clinics already use.**\n\n### Value (what those attributes mean for the clinic)\n| Attribute | Value to the clinic |\n|---|---|\n| On-call and emergency rotations | Overnight and weekend cover is always staffed and fairly shared. Less burnout, fewer 2am phone trees. |\n| Certification enforcement | Lower clinical and liability risk. A scheduling mistake can't put an unqualified person on anaesthesia monitoring. |\n| Payroll exports | The practice manager doesn't re-key hours at month end. On-call and emergency pay is calculated correctly. |\n| All of the above | The practice manager stops spending hours on the rota every week and stops worrying about it. |\n\n### Target customer (best-fit)\n**Independent veterinary clinics with 2\u20136 vets** that run on-call or emergency coverage and employ credentialed vet techs. The buyer and day-to-day user is usually the **practice manager** or practice owner.\n\nSignals of a strong fit:\n- Offers after-hours or emergency care, or shares an on-call rota.\n- Has had a near miss or headache over who's qualified to do what.\n- Currently runs the schedule on paper or spreadsheets, or has tried a generic app and found it didn't fit.\n\n### Market category\n**Veterinary staff scheduling**, or more broadly **clinical scheduling for veterinary practices**.\n\nAvoid \"workforce management\" and \"hourly team scheduling.\" Both put us back in the Deputy price comparison.\n\n### Relevant trends (use lightly)\n- Vet-tech shortage and burnout make fair, predictable on-call rotas a retention issue, not only an admin one.\n- *Verify with current industry data before using in copy.*\n\n---\n\n## 3. Messaging\n\n### Positioning statement (internal)\n> For **independent veterinary clinics** that need to cover on-call and emergency shifts with properly credentialed staff, **Shiftwise** is a **veterinary staff scheduling app** that builds rotas around clinical rules: who's on call, who's certified for what, and how it all gets paid. Unlike generic scheduling apps like When I Work or Deputy, which treat a clinic like a shop, Shiftwise understands you're running a hospital.\n\n### Homepage hero (options)\n- **A.** *Scheduling for vet clinics, not coffee shops.*\n  Sub: On-call rotas, emergency cover, and certification checks, so the right person is always on shift.\n- **B.** *Your clinic is a hospital. Your scheduler should know that.*\n  Sub: Shiftwise handles on-call, emergency shifts, and vet-tech certifications, then sends hours straight to payroll.\n- **C.** *The rota that knows who's qualified.*\n  Sub: Staff scheduling built for veterinary practices.\n\nRecommendation: **B.** It echoes our customers' own words and sets up the contrast with generic tools.\n\n**Hero image:** replace the barista with a vet tech or practice manager in a clinic setting. A screenshot of an on-call rota with a \"not certified\" warning would also make the point.\n\n### Three message pillars\n1. **On-call, handled.** Build fair on-call and emergency rotas, swap shifts without a phone tree, and see who's covering tonight at a glance.\n2. **Only qualified hands.** Shiftwise tracks vet-tech certifications and won't let you schedule someone for anaesthesia monitoring unless they're certified.\n3. **Straight to payroll.** Export hours, including on-call and emergency time, to the payroll tools your clinic already uses.\n\n### Sales pitch (replaces \"the easiest scheduling app for hourly teams\")\n> \"Most scheduling apps were built for shops and restaurants. Shiftwise was built for clinics like yours. It handles on-call rotations and emergency shifts, makes sure only certified techs get scheduled for things like anaesthesia monitoring, and exports hours to your payroll. Clinic managers tell us it's the only scheduler that understands they're running a hospital.\"\n\n### Proof to collect\n- Named testimonials and quotes from the 12 interviewed clinics (get permission for the \"hospital, not a shop\" quote).\n- A metric such as hours per week saved on the rota, or certification conflicts caught.\n- Retention as proof: \"clinics stay with us\" (2% monthly churn, if we're comfortable sharing it).\n\n---\n\n## 4. Addressing the CEO's concern: \"Is the vet market too small?\"\n\nIt isn't small compared with where we are now. A quick estimate:\n\n- ~32,000 US vet clinics (from the brief).\n- Vet revenue today is ~$1.89M (61% of $3.1M).\n- **Assumption, not in the brief:** an average clinic pays ~$3,000/yr. On that assumption we have ~630 vet clinics, about **2% of US clinics**.\n- At the same price, every US clinic would be ~$96M ARR. Even if only half are independent and a good fit, that's ~$48M. Reaching **25% of that** would roughly **4x today's total ARR**.\n\n*Replace the $3,000 ACV with our real figure. Also note that the 32,000 figure covers all clinics. Some belong to corporate groups that may buy differently, so the independent count is lower.*\n\nThree more points:\n- **Niche positioning is about who we target, not who we accept.** Restaurants, gyms, and retail can still sign up. We just stop spending marketing and sales effort on segments where we have no edge and high churn.\n- **Retention compounds.** With 2% vs 7% monthly churn, each vet clinic is worth roughly 3.5x a restaurant customer at the same price. Win more vets and the base grows faster.\n- **There are logical ways to expand later**, each building on the same core (clinical rules, on-call, credentials):\n  - More vets: larger independents, small groups, emergency and specialty hospitals.\n  - Adjacent clinical settings with the same \"hospital, not a shop\" problem: dental practices, physiotherapy, small medical clinics, care homes. *These are hypotheses to test, not plans.*\n  - Outside the US: UK, Canada, and Australia have similar independent-clinic markets.\n\nNarrow now to win the segment, then expand from strength.\n\n---\n\n## 5. What changes, and what doesn't\n\n| Area | Change |\n|---|---|\n| Homepage | New vet-focused hero, copy, and imagery (Section 3). |\n| Sales pitch | Lead with on-call, certifications, and payroll. Drop \"easiest app for hourly teams.\" |\n| Target accounts | Prioritise independent vet clinics. Deprioritise outbound to restaurants. |\n| Product roadmap | Favour vet-specific depth (on-call, credentials, vet payroll, PIMS integrations) over generic features. |\n| Existing non-vet customers | Keep supporting them. No forced migration and no announcement needed. |\n| Pricing | Don't compete with Deputy on price. Price on clinical value (risk, time saved). *Review separately.* |\n\n---\n\n## 6. Open questions to validate\n\n1. **What is our actual ACV per vet clinic?** It's needed for the market-size estimate.\n2. **How many of the 32,000 clinics are independent with 2\u20136 vets?**\n3. **Does PIMS-bundled scheduling come up in vet deals?** If so, it's a key alternative to position against.\n4. **Win rate and sales cycle for vets vs restaurants.** This would strengthen the case.\n5. **Are there gyms or retail customers with low churn?** Check whether any part of the long tail behaves like the vet segment.\n6. **Test the headline.** Run options A, B, and C with 5\u201310 practice managers or as a landing-page A/B test.\n"}]}, "haiku-1": {"cost_usd": 0.0049, "seconds": 22, "output_tokens": 3808, "files": [{"name": "positioning.md", "text": "# Shiftwise positioning\n\n## Positioning statement\n\nFor independent veterinary clinics with 2 to 6 vets, Shiftwise is the scheduling and time-tracking app built for hospitals, not shops. It handles on-call rotations and emergency shifts, blocks an uncertified tech from being scheduled for anaesthesia monitoring, and exports hours to the payroll tools clinics already use. Generic hourly scheduling apps and paper rotas don't know what a clinical roster needs. Shiftwise does.\n\n## Why this position\n\n- **Revenue and retention.** Veterinary clinics are 61% of revenue and churn at 2% a month. Restaurants are 15% of revenue and churn at 7%.\n- **Buyers say it in their own words.** Across 12 interviews, the value was on-call handling, certification control, and payroll export. Two clinic managers said it's \"the only scheduler that understands we're a hospital, not a shop.\" Use that line as customer language, attributed to them.\n- **It avoids the crowded fight.** Deputy, When I Work, and 7shifts sell generic hourly scheduling. We lose to Deputy on price in restaurants. A vet-specific position doesn't compete on the same ground.\n\n## Target customer\n\n- Primary: independent veterinary clinics, 2 to 6 vets.\n- Buyer and user: clinic managers who build the rota and handle payroll.\n- Not targeted: restaurants, retail, gyms. Existing customers stay served; no new marketing spend goes to them.\n\n## Alternatives the buyer considers\n\n- Paper rotas and spreadsheets (the main one in vet clinics)\n- Generic hourly scheduling: When I Work, Deputy, 7shifts (restaurant-focused)\n\n## What we say, and what we stop saying\n\n| Stop | Use instead |\n| --- | --- |\n| \"The easiest scheduling app for hourly teams\" | \"Scheduling built for veterinary hospitals\" |\n| Barista hero image | A clinic scene: a vet, a tech, an on-call board |\n| Generic \"any hourly workforce\" framing | Named clinical needs: on-call, emergency cover, certified coverage |\n\n## Homepage hero: draft headlines\n\n1. Scheduling built for veterinary hospitals.\n2. On-call, emergency shifts, and certified cover, without the spreadsheet.\n3. The scheduler that knows you're a hospital, not a shop.\n\nPick one and test it against the current hero. Subhead: \"Rotas, certifications, and payroll exports for independent vet clinics.\"\n\n## Answering the CEO's concern\n\nNarrowing to vets is a smaller market than \"all hourly teams,\" but the brief's 32,000 figure is total US vet clinics, not our target. The number that matters is independent clinics with 2 to 6 vets, which the brief doesn't give. Get that count before deciding whether the market is too small. Until then, treat the claim as unverified.\n\nThe position also limits the risk in the other direction: vet clinics retain well, and a focused position gives us a clear reason to win. Expansion into adjacent clinical settings is a hypothesis for later, not part of this positioning.\n\n## Open decisions\n\n1. **Restaurants.** This doc assumes we stop marketing to them but keep serving them. If the company wants to exit or to keep a restaurant product, the positioning changes.\n2. **Market size.** Get the count of independent 2 to 6 vet clinics in the US before committing hard to this.\n3. **Hero and sales pitch.** The sales team currently pitches the generic line. They need the new statement and the proof points before the next round of calls.\n\n## Source\n\nAll facts come from brief.md. Nothing here is verified beyond that document.\n"}]}, "haiku-2": {"cost_usd": 0.00482, "seconds": 24, "output_tokens": 4200, "files": [{"name": "positioning.md", "text": "# Shiftwise positioning\n\n## The position\n\nShiftwise is scheduling and time tracking for veterinary hospitals. It is built around on-call rotations, emergency cover, and staff credentials, not generic hourly shifts.\n\n**For** independent veterinary clinics with 2-6 vets, **who** have to staff on-call rotations and emergency shifts with qualified people, **Shiftwise** is the scheduler that understands a clinic is a hospital. **Unlike** general hourly-shift tools, it blocks scheduling an uncertified tech onto anaesthesia monitoring and exports hours straight into the payroll tools clinics already use.\n\n## Headline and subhead\n\n- **Headline:** Scheduling built for veterinary hospitals.\n- **Subhead:** On-call rotations, emergency cover, and certified staff, handled. Hours go straight to payroll.\n\nAlternative headline: \"The scheduler for clinics that run like hospitals.\" Use it only if it survives a test with clinic managers; the strongest version of it is a quote from one of them, not our copy.\n\n## Message pillars\n\n1. **On-call and emergency cover are first-class.** Rotas are built around who is on call, not around a block of identical shifts.\n2. **Credentials are enforced, not tracked on a spreadsheet.** An uncertified tech cannot be placed on a shift that requires certification.\n3. **Hours reach payroll without re-keying.** Export goes to the tools vet clinics already run.\n\n## What to stop saying\n\n- \"The easiest scheduling app for hourly teams.\" It is the sales pitch today. It describes the whole long tail and puts us in direct price competition with Deputy, 7shifts, and When I Work. It does not describe the customers who stay.\n- The barista hero. It signals a restaurant product to the 61% of revenue that is vets. Replace it with a rota that shows an on-call row and a certification block.\n\n## Why this position holds\n\n- **Revenue concentration.** Vet clinics are about 61% of the USD 3.1M ARR, roughly USD 1.9M. Restaurants are 15%, roughly USD 0.47M.\n- **Retention.** Vet clinics churn at about 2% a month. Restaurants churn at about 7%. Roughly three and a half times the churn rate sits in the segment we currently market to least precisely.\n- **Customer language.** Twelve interviews surfaced the three pillars above, and they are the reasons vets give for choosing us. The line \"the only scheduler that understands we're a hospital, not a shop\" came from two clinic managers. Treat it as a message to test, not as evidence on its own.\n\n## Answering the CEO's concern\n\nNarrowing to vets does not make the market too small by default. The brief gives about 32,000 US vet clinics, and that is the addressable count for this positioning. It is not yet a dollar figure. Sizing it needs two numbers the brief does not contain: the number of current customers in each segment, and average revenue per account. With those, the question becomes concrete: how much of 32,000 clinics would pay for this, at what price.\n\nRestaurants and retail do not need to be dropped. They can keep the product as it is and stay out of the marketing. That keeps the vet message clean and avoids spending the brand on a fight we lose on price.\n\n## Open questions before this goes live\n\n1. **Competitor check.** Do When I Work, Deputy, or 7shifts offer on-call rotas or certification-gated scheduling? If they do, the differentiation rests on depth, not on the feature being absent. The brief does not say either way.\n2. **Segment economics.** Customer count and average revenue per account by segment, so the vet market can be sized in dollars.\n3. **Churn drivers.** Whether the 2% churn in vet clinics is tied to the on-call and certification features. If it is, those features should be the lead, and the proof should cite them.\n4. **Payroll integrations.** Which payroll tools vets actually use, so the export claim names them.\n5. **Decision owner.** Dropping \"hourly teams\" as the lead message changes what sales says on every call. Someone needs to own that change, and the CEO's concern needs an answer that uses the sizing numbers above.\n"}]}, "haiku-3": {"cost_usd": 0.00571, "seconds": 28, "output_tokens": 5661, "files": [{"name": "positioning.md", "text": "# Shiftwise positioning\n\n## Positioning statement\n\nFor independent veterinary clinics (2\u20136 vets) who have to staff on-call and emergency shifts safely, Shiftwise is the scheduling and time-tracking app that knows who is certified for what. It keeps uncertified staff off tasks like anaesthesia monitoring and sends hours straight to the payroll tools clinics already use. Unlike general hourly schedulers, it is built for a clinic that runs like a hospital, not a shop.\n\n## Why this and not \"the easiest scheduling app for hourly teams\"\n\n- The current line puts Shiftwise against When I Work, Deputy and 7shifts on their ground: general features and price. The brief says we already lose to Deputy on price in restaurants.\n- The vet value is specific and hard for a general tool to match quickly: on-call and emergency handling, certification-gated scheduling, and payroll export to the tools vets use.\n- Customers are not evenly split. Vets are 61% of revenue with 2% monthly logo churn. Restaurants are 15% with 7% churn. Positioning should follow the customers who stay.\n- The \"any hourly workforce\" pitch gives the broadest reach but no reason to choose us over cheaper tools.\n\n## Competitive frame\n\n- **Alternatives buyers use:** paper rotas and spreadsheets; general hourly schedulers (When I Work, Deputy, 7shifts).\n- **What we do that they don't:** on-call and emergency shift handling; credential checks that block unqualified staff from scheduled tasks; hours export to vet payroll tools.\n- **Value to the buyer:** no unqualified person on high-risk shifts; on-call rotas stop being built by hand; hours are payroll-ready.\n\n## Messaging\n\n**Headline options (pick one and test):**\n1. Scheduling for veterinary clinics, with certifications built in.\n2. The scheduler that understands you run a hospital.\n3. Rotas, on-call and certifications, for vet clinics.\n\n**Subhead:** Emergency shifts, on-call rotations and tech certifications in one schedule. Hours go straight to payroll.\n\n**Proof:** Use the clinic manager quote (\"it's the only scheduler that understands we're a hospital, not a shop\") once the customer has approved it for marketing.\n\n**Homepage hero:** Replace the barista. A clinic setting with a vet tech on shift matches the segment that is 61% of revenue. Remove restaurant and retail imagery from the top of the page.\n\n## Restaurants and the long tail\n\n- Do not lead with restaurants or the long tail in any positioning copy.\n- Keep serving them, but stop spending marketing on them until we understand the 7% churn and the Deputy price gap.\n- Leadership should decide whether restaurants are a supported segment or a wind-down. This document does not make that call.\n\n## The CEO's concern: is vet-only too small?\n\n- The brief cites about 32,000 vet clinics in the US. That is a count of clinics, not of customers. We do not know how many are independent, have 2\u20136 vets, and buy scheduling software.\n- Vet-first positioning is a wedge, not a ceiling. Credential-gated, on-call-aware scheduling is the part most likely to carry into other clinical settings, such as urgent care, dental or nursing. That is a hypothesis. The brief contains no evidence for it.\n- Recommendation: go vet-first in messaging now. Test adjacent clinical settings with a small number of interviews before adding them to the positioning. Do not rebuild the product for them yet.\n- Revenue context: 61% of $3.1M ARR is roughly $1.9M from vet clinics.\n\n## Open questions before this goes live\n\n- How many target clinics exist, and what share could realistically be won? Needed to answer the CEO's market-size question.\n- Average revenue per account by segment. The brief does not include it.\n- Win/loss data beyond the Deputy restaurant price loss. We need to know how often vet deals are lost, and to whom.\n- Whether the 12 interviews covered only vets or also restaurant customers, and whether the findings are representative.\n- Whether the \"hospital, not a shop\" quote can be used publicly.\n"}]}}, "verdicts": [{"vs": "opus", "pair": "1", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "haiku+skill", "why": "A grounds the veterinary focus in evidence and handles market uncertainty responsibly, but lacks ready-to-use headline copy and explicit customer-retention guidance. B offers usable messaging but invents competitor limitations, guarantees staffing outcomes, and asserts market sufficiency using assumed pricing."}, {"vs": "opus", "pair": "1", "judge": "gpt", "order": "opus shown first", "winner": "haiku+skill", "why": "B grounds the veterinary focus and handles market uncertainty responsibly, but lacks finished homepage copy. A supplies usable messaging while inventing competitor limitations, guaranteeing staffing outcomes, and asserting market sufficiency using assumed pricing and all clinics."}, {"vs": "opus", "pair": "1", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "tie", "why": "Both fully meet all criteria with clear vet focus, accurate facts, actionable messaging, and address the CEO's concern."}, {"vs": "opus", "pair": "1", "judge": "deepseek", "order": "opus shown first", "winner": "opus", "why": "A delivers immediate, usable positioning and copy; B is a provisional analysis requiring validation."}, {"vs": "opus", "pair": "3", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "haiku+skill", "why": "A grounds the veterinary focus in evidence and handles market uncertainty responsibly, though its messaging lacks a decisive headline. B supplies usable copy but invents competitor limitations, capabilities and guaranteed outcomes, then answers market concerns with unsupported pricing and market estimates."}, {"vs": "opus", "pair": "3", "judge": "gpt", "order": "opus shown first", "winner": "haiku+skill", "why": "B grounds the veterinary focus in evidence and handles market uncertainty responsibly, but lacks a decisive homepage headline. A invents capabilities and competitor limitations, promises guaranteed coverage, and manufactures pricing and market-size estimates."}, {"vs": "opus", "pair": "3", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "haiku+skill", "why": "A is factually perfect and addresses all criteria. B makes arithmetic errors and invents a market size."}, {"vs": "opus", "pair": "3", "judge": "deepseek", "order": "opus shown first", "winner": "opus", "why": "A delivers decisive, usable positioning; B is provisional analysis, not ready copy."}, {"vs": "haiku", "pair": "2", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "haiku+skill", "why": "A correctly distinguishes total clinics from the eligible target and proposes validation. B wrongly treats all 32,000 clinics as addressable and broadens certification enforcement beyond the evidence. A also makes unsupported claims about competitors\u2019 restaurant relevance."}, {"vs": "haiku", "pair": "2", "judge": "gpt", "order": "haiku shown first", "winner": "haiku+skill", "why": "B correctly distinguishes total clinics from the eligible target and preserves the specific certification capability. A incorrectly treats all 32,000 clinics as addressable and overstates certification and payroll functionality. Both introduce unsupported competitive assumptions; A provides stronger ready-to-use copy."}, {"vs": "haiku", "pair": "2", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "tie", "why": "Both deliverables fully satisfy all rubric criteria with clear, evidence-based positioning for veterinary clinics and actionable messaging."}, {"vs": "haiku", "pair": "2", "judge": "deepseek", "order": "haiku shown first", "winner": "haiku", "why": "Deliverable A is factually precise and provides more actionable, immediate copy. Deliverable B makes a calculation error (annualizing monthly churn incorrectly)."}, {"vs": "opus", "pair": "2", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "haiku+skill", "why": "A gives usable veterinary positioning and appropriately limits market sizing, though it misreads competitor context. B invents competitor limitations, guarantees safety and compliance, and uses assumed pricing and all clinics to assert unsupported market headroom."}, {"vs": "opus", "pair": "2", "judge": "gpt", "order": "opus shown first", "winner": "haiku+skill", "why": "B grounds veterinary positioning in evidence and acknowledges unknown market size. Its supporting copy needs development and annualized churn is slightly inaccurate. A asserts unsupported competitor limitations, guarantees safety/compliance, and invents pricing-based market projections."}, {"vs": "opus", "pair": "2", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "haiku+skill", "why": "B made arithmetic errors and invented market sizing; A stayed factually faithful."}, {"vs": "opus", "pair": "2", "judge": "deepseek", "order": "opus shown first", "winner": "opus", "why": "A provides more actionable copy and a clearer competitive distinction, fully meeting all criteria."}, {"vs": "haiku", "pair": "3", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "haiku+skill", "why": "A better bounds uncertainty and connects clinic workflows to value, though unsupported pricing assumptions and provisional messaging weaken it. B offers stronger usable copy but invents competitor feature gaps and overstates certification protection."}, {"vs": "haiku", "pair": "3", "judge": "gpt", "order": "haiku shown first", "winner": "haiku+skill", "why": "B better bounds competitive claims and staffing benefits, though unsupported pricing generalizations weaken factual fidelity and provisional gates reduce usability. A offers stronger ready-to-use copy but asserts unverified competitor deficiencies and overstates protection against unqualified assignments."}, {"vs": "haiku", "pair": "3", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "haiku", "why": "B delivers immediate, usable positioning and copy, while A provides a process and leaves the team to write it."}, {"vs": "haiku", "pair": "3", "judge": "deepseek", "order": "haiku shown first", "winner": "haiku", "why": "A delivers immediate, usable positioning; B is a provisional analysis requiring further work."}, {"vs": "haiku", "pair": "1", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "haiku+skill", "why": "A better bounds competitive claims and validates market viability, but leaves usable messaging provisional. B offers stronger immediate copy yet invents vets\u2019 main alternative and asserts unsupported competitor shortcomings. A also unnecessarily questions the interviews\u2019 veterinary context."}, {"vs": "haiku", "pair": "1", "judge": "gpt", "order": "haiku shown first", "winner": "haiku+skill", "why": "B better bounds competitive claims and market uncertainty, despite excessive provisionality and missing headline copy. A delivers usable messaging but invents the dominant veterinary alternative and asserts unsupported competitor shortcomings. B also leaves continued service to other segments unresolved."}, {"vs": "haiku", "pair": "1", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "tie", "why": "Both fully meet all rubric criteria with clear, evidence-based positioning for veterinary clinics."}, {"vs": "haiku", "pair": "1", "judge": "deepseek", "order": "haiku shown first", "winner": "haiku", "why": "A delivers immediate, usable positioning and copy. B is a process document, leaving the team to write the actual positioning."}]}, {"id": "pos-3", "family": "positioning", "prompt": "Help me figure out how to position this. Context in context.md.", "input": "context.md", "skill": "dunford-obviously-awesome", "input_text": "Product: Quillmark, a contract-redlining assistant. Upload a counterparty's contract, it marks clauses that deviate from your company's playbook and suggests fallback language.\nTeam: 2 founders (ex-lawyer, ex-engineer), pre-seed, 9 pilot customers, 3 paying.\nWho paid: in-house legal teams of 1-3 lawyers at Series B-D SaaS companies that sign 40-150 vendor and customer contracts a month. They said their alternative was \"the lawyer reads every page\" or sending overflow to outside counsel at USD 450/hour.\nWho did not convert: big law firms (wanted on-prem, 12-month security review) and solo founders (too few contracts).\nThings only we do: playbook is built from the customer's own last 50 signed contracts in about 20 minutes; every suggestion links to the precedent clause it came from.\nThe market is full of \"AI for legal\" tools (Harvey, Spellbook, Ironclad AI). Investors keep asking \"how are you different from Harvey?\"\n", "rubric": [{"id": "c1", "criterion": "Identify a focused target customer grounded in the paying pilots.", "full_marks_when": "Prioritizes in-house legal teams of 1\u20133 lawyers at Series B\u2013D SaaS companies handling 40\u2013150 vendor and customer contracts per month, and explains why large law firms and solo founders are poor initial targets.", "zero_when": "Targets legal professionals broadly, prioritizes non-converting segments, or fails to identify the demonstrated paying customer."}, {"id": "c2", "criterion": "Frame the problem around the customer's actual alternatives and workload.", "full_marks_when": "Explains how Quillmark helps small legal teams handle contract volume currently managed by reading every page or sending overflow to outside counsel at USD 450/hour, with a clear connection to review capacity and consistency.", "zero_when": "Relies on generic AI productivity claims without connecting the product to the buyers' existing workflow and alternatives."}, {"id": "c3", "criterion": "Turn the distinctive product capabilities into meaningful reasons to choose Quillmark.", "full_marks_when": "Connects building a playbook from the customer's last 50 signed contracts in about 20 minutes to easier setup and alignment with their negotiated positions, and connects precedent-linked suggestions to traceability and easier lawyer verification.", "zero_when": "Offers generic claims such as faster or smarter contract review, or merely lists features without explaining their value to the target buyer."}, {"id": "c4", "criterion": "Give a credible answer to the Harvey comparison without inventing competitor weaknesses.", "full_marks_when": "Provides a usable response explaining Quillmark's specific buyer, contract-redlining job, customer-derived playbook, and precedent-linked suggestions; avoids unsupported claims about what Harvey, Spellbook, or Ironclad AI cannot do.", "zero_when": "Leaves the investor's comparison unanswered, bases differentiation on being another legal AI tool, or invents competitor limitations to establish superiority."}, {"id": "c5", "criterion": "Deliver concrete positioning language that the founders can use immediately.", "full_marks_when": "Includes a clear recommended positioning statement and concise customer-facing wording that communicate who Quillmark serves, what it does, and why it matters, with a brief explanation of the positioning choice.", "zero_when": "Only presents questions, abstract advice, frameworks, or a menu of possibilities without recommending a position and supplying usable wording."}, {"id": "c6", "criterion": "Maintain factual and numerical accuracy and calibrate claims to the evidence.", "full_marks_when": "Preserves the input facts when used: 9 pilot customers, 3 paying, teams of 1\u20133 lawyers, 40\u2013150 contracts per month, USD 450/hour outside counsel, and a playbook built from 50 signed contracts in about 20 minutes. If calculating pilot conversion, states 3/9 = 33.3%. Treats these early signals as provisional and does not invent savings, accuracy, security capabilities, guaranteed legal outcomes, or proven market-wide uniqueness.", "zero_when": "Makes material factual or arithmetic errors, fabricates performance or customer evidence, or presents the small pilot sample as proof of broad market demand."}], "runs": {"haiku-skill-1": {"cost_usd": 0.00428, "seconds": 21, "output_tokens": 3321, "files": [{"name": "positioning.md", "text": "# Quillmark positioning\n\n## Recommendation\n\nDon't position Quillmark as \"AI for legal.\" That category is crowded, and it makes you answer \"how are you different from Harvey?\" on Harvey's terms. Position it as the contract-redlining assistant for in-house teams of 1-3 lawyers who review 40-150 counterparty contracts a month and have outgrown \"the lawyer reads every page.\"\n\nThe two things only you do are the wedge:\n\n1. **Playbook from your own paper.** A customer's last 50 signed contracts become their playbook in about 20 minutes. Competitors who need weeks of playbook setup or a generic template lose on time-to-value.\n2. **Every suggestion cites its precedent.** Lawyers can check where each fallback came from. That matters to people who have to defend a redline to a GC or a counterparty.\n\nLead with the outcome and the buyer, then the proof. Don't lead with model quality.\n\n## Candidate one-liners\n\n- \"Quillmark marks the clauses in a vendor's contract that break your playbook, and suggests fallback language from your own signed deals.\"\n- \"Your last 50 signed contracts become your playbook in 20 minutes. Every redline suggestion links to the precedent it came from.\"\n- \"Contract review for in-house teams that sign 40-150 vendor and customer contracts a month and can't afford another associate.\"\n\nPick one as the homepage line and test the other two in outreach.\n\n## Draft answer to \"how are you different from Harvey?\"\n\n\"Harvey is built for big-firm and enterprise legal, with long security reviews and deployments. We're built for the in-house team of one to three lawyers that was sending overflow to outside counsel at $450 an hour. We're live in weeks, not a 12-month review. We learn the playbook from the customer's own signed contracts in about 20 minutes, and every suggestion links to the precedent clause behind it, so the lawyer can verify it in seconds.\"\n\nCheck the Harvey claims against their current product and pricing before using this with investors. The memo relies on general knowledge, not a fresh competitive review.\n\n## Who to sell to, and who to stop chasing\n\n- **Target:** in-house legal at Series B-D SaaS companies, 1-3 lawyers, 40-150 contracts a month.\n- **Stop chasing for now:**\n  - Big law firms. They wanted on-prem and a 12-month security review, which a pre-seed company can't meet yet.\n  - Solo founders. Too few contracts to justify the tool.\n\nNaming who you're not for makes the positioning sharper and saves sales time.\n\n## Where the competitors really compete\n\n- **Spellbook** works inside Word, where lawyers already draft and redline. If buyers say \"I don't want to leave Word,\" that's the objection to prepare for. Confirm Spellbook's current in-house offering before relying on this.\n- **Ironclad** is a contract lifecycle platform with AI features. It competes for the \"one system for all contracts\" budget. Your pitch is narrower and faster to adopt, not a replacement for a CLM.\n- **Harvey** competes for big-law and enterprise budgets. Stay out of that fight.\n\n## Proof to collect from the 3 paying customers\n\nThe positioning only holds if the paying customers can back it with numbers:\n\n- Contracts reviewed per month, before and after.\n- Hours saved per contract, or outside-counsel spend avoided. The $450/hour anchor is the ROI comparison buyers already use.\n- Minutes from signup to a working playbook.\n- How often lawyers accept a suggestion, and whether they cite the precedent link when they do.\n\nUse one quote and one number from each paying customer in the pitch. A specific number (\"cut outside-counsel spend by X\") beats any tagline.\n\n## Risks\n\n- **Category confusion.** If the homepage says \"AI legal assistant,\" the buyer will compare you to Harvey on feature lists. The playbook-from-your-paper claim has to be visible above the fold.\n- **The 20-minute claim.** It's your strongest differentiator, so it needs to hold on messy real contract sets, not just clean samples. Measure it on the 9 pilot accounts.\n- **Overlap with the big players.** Their in-house products may improve. Your moat is the onboarding flow and the per-suggestion precedent trail, not the model.\n\n## Open questions\n\n- Is the precedent trail the main reason the 3 paying customers bought, or is it the speed of setup? The answer changes which one leads.\n- What did the 6 non-paying pilots want that they didn't get?\n- Is the $450/hour outside-counsel comparison what buyers actually use to justify the purchase, or was it just the alternative they mentioned?\n"}]}, "haiku-skill-2": {"cost_usd": 0.00526, "seconds": 27, "output_tokens": 4896, "files": [{"name": "positioning.md", "text": "# Quillmark positioning\n\n## Recommendation\n\nOwn a narrow category: **playbook-based contract redlining for in-house teams**. Don't compete as \"AI for legal.\" Lead with the precedent-linking, since it's the one thing a generic tool can't copy quickly and it answers the Harvey question directly.\n\nPositioning statement:\n\n> For in-house legal teams at growing SaaS companies that review 40\u2013150 vendor and customer contracts a month, Quillmark is a redlining assistant that flags clauses off your playbook and suggests fallback language. The playbook is learned from your own last 50 signed contracts, and every suggestion links to the precedent clause it came from. Unlike general legal AI, it knows your company's positions, not the market's.\n\nHeadline candidates:\n\n1. **Redlines that cite your own precedent.** (Recommended. Leads with the differentiator and the trust feature.)\n2. Your playbook, learned from the contracts you already signed.\n3. First-pass review of every inbound contract, without the overflow bill.\n\n## Dunford's five components\n\n**Competitive alternatives.** What buyers do today, per the context:\n- The lawyer reads every page (cost: in-house time, slow turnaround).\n- Outside counsel at ~USD 450/hour (cost: money, latency).\n- Generic AI tools (Harvey, Spellbook, Ironclad AI) that don't know the company's positions. This is the one to watch, because buyers will compare on it.\n\n**Unique attributes.**\n- Playbook built from the customer's own signed history in ~20 minutes.\n- Each suggestion traces to a precedent clause.\n\n**Value.** Fast, consistent first-pass review on high-volume paper, without sending overflow out, and suggestions the lawyer can defend to the business because they point at contracts the company already signed.\n\n**Target customers.** In-house teams of 1\u20133 lawyers at Series B\u2013D SaaS companies. This matches the 3 paying customers and is the segment that converted.\n\n**Market category.** Playbook-based contract review. Choosing this name means you're not a smaller Harvey. You're a different tool for a different job.\n\n## Answering \"how are you different from Harvey?\"\n\nSuggested answer, roughly:\n\n> Harvey is a general legal workspace, mostly sold to large firms. We do one job for in-house teams: review the contracts that come in against your positions. Our playbook isn't a template we ship. It's built from what you've already signed, and every suggestion links to the clause it came from.\n\nKeep it about the customer's job and their data. Avoid claims that Harvey or Spellbook can't do something unless you've verified it against their current product.\n\n## Who not to chase\n\n- **Big law firms.** They asked for on-prem and a 12-month security review. A pre-seed team can't win that now, and it pulls the product toward a segment that doesn't match the paying base.\n- **Solo founders.** Too few contracts to justify the playbook setup.\n\n## Language to avoid\n\n- \"AI lawyer\" or \"replaces counsel.\" The in-house lawyer is the user and the buyer's champion. Frame the product as making them faster and more consistent.\n- Speed or accuracy claims against named competitors without data.\n- \"Built for legal\" as a headline. Every tool says it.\n\n## Open questions before this goes in the deck\n\n1. **Which attribute actually closes deals?** Ask the 3 paying customers what they'd do if Quillmark disappeared tomorrow, and what made them say yes. If they name speed, lead with speed. If they name the precedent links, lead with those.\n2. **Who is the buyer?** The lawyer uses it, but a GC or legal ops lead may sign. The context doesn't say who holds the budget. Confirm before writing buyer-facing copy.\n3. **What did the 9 pilots compare Quillmark against?** If most lost to \"the lawyer reads every page,\" the story is about trust and time saved. If they lost to Harvey or Ironclad, the story needs a sharper contrast.\n4. **Is the \"20 minutes\" claim measured?** Use it only if it's been timed on real customer data.\n5. **Switching cost.** Does the playbook stay with the customer if they leave? If so, that's a retention point; if not, don't pitch it as a moat.\n\n## Verification note\n\nCompetitor descriptions here are general and not checked against current product pages or pricing. Verify them before using them externally.\n"}]}, "haiku-skill-3": {"cost_usd": 0.00487, "seconds": 23, "output_tokens": 4259, "files": [{"name": "positioning.md", "text": "# Quillmark positioning\n\n## Recommendation\n\nPosition Quillmark as the redlining tool for in-house legal teams that sign a high volume of mid-market contracts, built on the company's own signed history. Don't position it as \"AI for legal.\" That category is where Harvey, Spellbook, and Ironclad AI already compete, and it invites a breadth comparison we will lose.\n\n## Target\n\n- **Buyer:** in-house legal teams of 1-3 lawyers at Series B-D SaaS companies, signing 40-150 vendor and customer contracts a month.\n- **Not for now:** big law (on-prem and 12-month security reviews don't fit a pre-seed team) and solo founders (volume too low). Neither converted, and chasing them pulls the product toward enterprise requirements.\n\n## Positioning statement\n\nFor in-house legal teams at growth-stage SaaS companies who review dozens of counterparty contracts a month, Quillmark is a redlining assistant that flags clauses deviating from a playbook built from the company's own last 50 signed contracts, and suggests fallback language. Every suggestion links to the precedent clause it came from, so the lawyer can defend it.\n\n## Answer to \"how are you different from Harvey?\"\n\n> Harvey is a broad legal AI platform. We do one job for the in-house team drowning in mid-market paper: redline each incoming contract against your own signed history. The playbook takes about 20 minutes to build from your last 50 contracts, and every suggestion points to the precedent clause it came from, so the lawyer can check the reasoning in seconds.\n\nBefore using this in a pitch, verify what Harvey currently sells and to whom. The context doesn't cover their product or customers, so the answer above describes our focus rather than their limits.\n\n## Message priority\n\n1. **Your own paper.** Playbook built from the customer's last 50 signed contracts in about 20 minutes. This answers the investor question and the buyer's \"why should I trust a playbook I didn't write.\"\n2. **Traceability.** Every suggestion links to a precedent clause. Lawyers have to justify redlines to business stakeholders, and this gives them something to point to.\n3. **Volume economics.** The real alternatives are \"the lawyer reads every page\" or outside counsel at USD 450/hour. Lead with the cost of those alternatives, using the customers' own numbers.\n\n## Avoid\n\n- \"AI lawyer,\" \"replace your legal team,\" or generic \"AI for legal\" framing. Buyers are lawyers; position the product as their leverage.\n- Enterprise and big-law features (on-prem, long security reviews) as headline claims. They signal the wrong customer.\n\n## Risks to the position\n\n- **Copyable features.** Precedent linking and fast playbook setup are the kind of thing a larger platform can add. The defensible part is likely the workflow and the per-customer playbooks that accumulate over time. This is a hypothesis until there's evidence for it.\n- **Thin evidence.** 3 paying out of 9 pilots is a small base. Confirm the 20-minute setup claim holds across all 9 pilots, not just the ones that paid.\n- **Unverified value driver.** Ask the 3 paying customers which feature they'd miss most. If precedent links don't come up unprompted, they may not be the main reason people pay, and the lead message should change.\n\n## Questions for the founders\n\n- Who is the buyer: the GC or senior lawyer, or whoever sends the contracts out for review?\n- Does per-seat pricing fit a \"40-150 contracts a month\" pitch, or would per-contract pricing match how customers think about the cost?\n- What did the 3 paying customers compare Quillmark against before they bought? Their answers are the most reliable positioning input available.\n"}]}, "opus-1": {"cost_usd": 0.14386, "seconds": 48, "output_tokens": 4344, "files": [{"name": "positioning.md", "text": "# Quillmark positioning\n\nBuilt from `context.md`. Anything marked **[verify]** is an assumption or a claim about competitors that should be checked before it goes in front of customers or investors.\n\n---\n\n## 1. The core shift: stop competing with \"AI for legal\"\n\nWhen investors ask \"how are you different from Harvey?\", they are placing you in the category *AI for legal*. In that category you look like a smaller, later Harvey, and you lose.\n\nYour paying customers didn't compare you to Harvey. They compared you to:\n\n1. **The in-house lawyer reading every page themselves**, or\n2. **Sending overflow to outside counsel at USD 450/hour.**\n\nThose are your real competitive alternatives, so build the positioning around beating them, not around beating other AI tools.\n\n---\n\n## 2. Positioning, piece by piece\n\n### Who it's for (best-fit customer)\nIn-house legal teams of **1\u20133 lawyers** at **Series B\u2013D SaaS companies** that sign **40\u2013150 vendor and customer contracts a month**.\n\nWhy this group: they have enough volume to hurt, but not enough headcount to keep up, and they already pay outside counsel for overflow. That budget is visible and easy to measure.\n\n**Not for (say this plainly, internally and to investors):**\n- **Big law firms.** They need on-prem and a 12-month security review, so the sales cycle doesn't fit a pre-seed company.\n- **Solo founders.** Too few contracts to build a playbook from, and too little pain.\n\nKnowing who you turn away is evidence of focus, and investors read it as a strength.\n\n### What they do today (competitive alternatives)\n| Alternative | What it costs them |\n|---|---|\n| Lawyer reads every page | The lawyer's time, deals wait in a queue, and sales complains that legal is the bottleneck |\n| Overflow to outside counsel | USD 450/hr, plus days of turnaround |\n| Generic AI legal tool **[verify per tool]** | Has to be set up with a playbook someone writes by hand, and its suggestions don't show where they came from |\n\n### What only Quillmark does (unique attributes)\n1. **The playbook builds itself from your last 50 signed contracts, in about 20 minutes.** No one has to write a playbook first. Your existing paper becomes the playbook.\n2. **Every suggestion links to the clause it came from.** \"We accepted this cap with Acme in March\" instead of \"the AI recommends\u2026\"\n\n### Why that matters to the buyer (value)\n| Attribute | Value to the in-house lawyer |\n|---|---|\n| Playbook from your own contracts | Useful on day one, with no consulting project or setup work. The tool reflects what *your company* actually agrees to, not generic \"market standard\" terms. |\n| Precedent link on every suggestion | **Trust and defensibility.** The lawyer can check a suggestion in seconds. When pushing back on a counterparty or explaining a call to the CFO or sales, they can point to precedent: \"we've signed this before.\" |\n| Both together | Fewer contracts go to outside counsel, deals move faster, and terms stay consistent across 40\u2013150 contracts a month |\n\n### Market category\nDon't call it \"AI for legal.\" Pick a frame where the precedent idea is the obvious leader. Options, ranked:\n\n1. **\"Precedent-based contract review\"** *(recommended)*. Names the thing only you do. It invites the question \"what does precedent-based mean?\", and the answer is your pitch.\n2. **\"Contract review for lean in-house teams\"**. Very clear about who it's for, but says less about how it's different.\n3. \"Playbook automation.\" Accurate, but it sounds like something an existing CLM vendor could add as a feature.\n\n---\n\n## 3. Messaging\n\n**One-liner**\n> Quillmark redlines incoming contracts using your own signed precedent, so a 2-person legal team can keep up with 150 contracts a month.\n\n**Shorter, for the homepage**\n> Redline every contract the way you've already agreed to sign it.\n\n**Supporting proof points**\n- Your playbook is ready in about 20 minutes, built from your last 50 signed contracts.\n- Every suggested edit links to the clause you signed before.\n- Send less overflow to outside counsel at USD 450/hr.\n\n**Words to use / avoid**\n- Use: *your precedent, your playbook, what you've signed before, defensible, traceable, overflow.*\n- Avoid: *AI-powered legal assistant, revolutionize, GPT for lawyers*. These put you right back in the Harvey comparison.\n\n---\n\n## 4. The investor answer: \"How are you different from Harvey?\"\n\n> \"Harvey is a general-purpose legal AI built mostly for large law firms and enterprise legal departments **[verify current positioning]**. We don't compete with them for those customers. In fact, we walked away from big law because of on-prem and year-long security reviews.\n>\n> We sell to the 1\u20133 person legal team at a Series B\u2013D SaaS company drowning in 40\u2013150 contracts a month. Their alternative isn't Harvey; it's reading every page themselves or paying outside counsel $450 an hour.\n>\n> We do one thing those tools don't: we build the playbook from the customer's own last 50 signed contracts in about 20 minutes, and every suggestion cites the precedent clause it came from. The lawyer can trust it on day one and defend it to the counterparty. 3 of our 9 pilots are paying, all from exactly that segment.\"\n\nFollow-up you should expect: **\"What stops Harvey or Ironclad from adding 'learn from your contracts'?\"** Prepare an honest answer. The strongest defensible pieces are probably:\n- **Focus.** The mid-market in-house segment is a different sales motion from enterprise and big law.\n- **Speed to value.** 20 minutes vs. an implementation project.\n- **Precedent linking as a product philosophy, not a feature.** It shapes the whole UX around traceability.\n- Over time, **switching costs**: the customer's precedent library and review history live in Quillmark.\n\nDon't claim a technical moat you don't have.\n\n---\n\n## 5. Gaps to close before leaning hard on this\n\n1. **Quantify the outside-counsel savings.** Ask your 3 paying customers: how many contracts a month went to outside counsel before vs. now, and how many hours each? Example of the math to fill in **[assumption, not data]**: 10 overflow contracts/month \u00d7 2 hrs \u00d7 USD 450 = USD 9,000/month. A real number from a real customer is your best pricing and fundraising asset.\n2. **Measure turnaround time.** Time from contract received to redline sent, before vs. after. \"Legal is no longer the bottleneck for sales\" is a message the CFO and CRO will hear.\n3. **Understand the 6 pilots that didn't pay.** Were they all big law or solo founders? If any were in-segment, find out why they didn't buy. It may reveal a weakness in the positioning.\n4. **Edge cases for the 20-minute claim.** What happens with fewer than 50 prior contracts, or messy and scanned ones? Qualify the claim if needed (\"from your recent signed contracts\").\n5. **Check competitor claims.** Confirm how Harvey, Spellbook and Ironclad AI currently handle playbooks and citations before saying \"they don't\" in public. In public copy, say what *you* do, not what others don't.\n\n---\n\n## 6. Summary\n\n| | |\n|---|---|\n| **For** | 1\u20133 lawyer in-house teams at Series B\u2013D SaaS, 40\u2013150 contracts/month |\n| **Instead of** | Reading every page, or overflow to outside counsel at USD 450/hr |\n| **Quillmark is** | Precedent-based contract review |\n| **That** | Redlines incoming contracts against what you've already signed |\n| **Unlike** | Generic legal AI that needs a hand-built playbook and gives unsourced suggestions |\n| **Because** | The playbook builds from your last 50 contracts in ~20 min, and every edit links to its precedent |\n"}]}, "opus-2": {"cost_usd": 0.13639, "seconds": 45, "output_tokens": 4060, "files": [{"name": "positioning.md", "text": "# Quillmark Positioning\n\n*Draft, 2026-10-08. Built from context.md: 9 pilots, 3 paying. The structure follows April Dunford's method. Start from who actually paid and what they would do without you, then work forward.*\n\n---\n\n## 1. The core insight\n\nYour paying customers did not compare you to Harvey. They compared you to **\"the lawyer reads every page\"** and to **outside counsel at $450/hour**. That is your real competition, and it is a much easier fight than \"AI for legal.\"\n\nSo stop positioning inside \"AI for legal.\" That category has well-funded players, and you can only lose a feature-by-feature comparison there. Position as **the fastest way for a small in-house legal team to apply its own precedent to every incoming contract.**\n\n---\n\n## 2. The positioning components\n\n| Component | Quillmark |\n|---|---|\n| **Competitive alternative** (what customers do without you) | (1) An in-house lawyer reads every page by hand. (2) Overflow goes to outside counsel at about $450/hr. (3) Possibly a static Word or Notion playbook nobody keeps up to date (*assumption, verify*) |\n| **Unique attributes** | (a) The playbook is built automatically from the customer's **own last 50 signed contracts in about 20 minutes**. (b) **Every suggested redline links to the precedent clause it came from.** |\n| **Value those attributes create** | (a) No playbook project. You are useful on day one, and the team does not spend weeks writing a playbook. (b) The lawyer can verify a suggestion in seconds instead of re-researching it, so they trust it enough to actually use it. Together: review more contracts in-house, at the positions the company has already accepted, and send less to outside counsel. |\n| **Best-fit customer** | In-house legal teams of **1\u20133 lawyers** at **Series B\u2013D SaaS companies** handling **40\u2013150 vendor and customer contracts a month**. Enough volume to hurt, too few lawyers to keep up, and enough signed history (50+ contracts) to build a playbook from. |\n| **Market category** | **Playbook-based contract review for lean in-house legal teams.** Not \"legal AI.\" |\n| **Relevant trend** (optional) | SaaS legal teams are being asked to support more deals without new headcount (*verify with customer quotes before using*) |\n\n### Why the losses help\n- **Big law firms** wanted on-prem and a 12-month security review. That is a different buyer with a different sales motion, and you can't serve them at pre-seed. Exclude them on purpose.\n- **Solo founders** sign too few contracts, and they likely don't have 50 signed contracts to build a playbook from, so the core feature doesn't work for them.\n- Both losses point to the same rule: **you win where contract volume is high and lawyer count is low.** Make that the qualifying filter.\n\n---\n\n## 3. Messaging\n\n**One-liner**\n> Quillmark redlines incoming contracts against your company's own precedent, so a small legal team can keep up without sending overflow to outside counsel.\n\n**Short pitch (about 30 seconds)**\n> If you're one of two or three lawyers at a growing SaaS company, you're reviewing 40 to 150 contracts a month. Either you read every page yourself, or overflow goes to outside counsel at $450 an hour. Quillmark reads your last 50 signed contracts and builds your playbook in about 20 minutes. Then, when a counterparty's paper comes in, it flags every clause that departs from what you've accepted before and suggests fallback language. Each suggestion links to the exact clause in your past contracts that it came from, so you can check it in seconds.\n\n**Homepage headline options**\n1. *Your playbook, built from your own contracts in 20 minutes.*\n2. *Redline like you've already negotiated this before, because you have.*\n3. *Every suggestion, backed by a contract you already signed.*\n\n(Option 1 leads with time-to-value. Option 3 leads with trust. Test both. Option 2 is the most memorable but the least specific.)\n\n**Proof points to collect now** (from the 3 paying customers)\n- Contracts reviewed per lawyer per month, before and after\n- Outside counsel spend on contract overflow, before and after\n- Time to review a typical vendor agreement, before and after\n- One quote about the precedent links (\"I can trust it because I can check it\")\n\n---\n\n## 4. Answering \"How are you different from Harvey?\"\n\nDon't argue features. Change which comparison is being made.\n\n> \"Harvey is a general-purpose legal AI platform. Its core customers are large law firms and big enterprise legal departments. We do one job for a different buyer: a 1\u20133 lawyer in-house team at a SaaS company that's drowning in vendor and customer paper. Our customers weren't choosing between us and Harvey. They were choosing between us and reading every page themselves, or paying outside counsel $450 an hour.\n>\n> Two things make that work. First, we build the playbook from *their own* last 50 signed contracts in about 20 minutes, so there's no implementation project. Second, every redline links to the precedent it came from, so a lawyer can trust it without redoing the research. A general-purpose tool gives you an answer. We give you *your company's* answer, with the receipt.\"\n\n**Then show the evidence:** 3 of 9 pilots paying, all in the target segment, and both non-converting groups explained by the segment filter, not by product gaps.\n\n**Short versions for other competitors** (*verify each against current product pages before using externally*):\n- **Spellbook:** an AI drafting and review assistant for lawyers in general. Quillmark is tuned to one team's own signed precedent.\n- **Ironclad AI:** AI inside a full CLM platform, which means a CLM purchase and rollout. Quillmark is useful on day one without replacing your contract system.\n\n**Expect this follow-up:** \"What stops Harvey from adding this?\" Answer: they could build the feature, but it isn't worth their focus. Their buyer and price point point upmarket. Your moat is focus on the segment, plus customer data that builds up over time (playbooks, accept/reject signals on suggestions). Be honest that the feature alone is not a moat.\n\n---\n\n## 5. Things to avoid saying\n\n- \"AI for legal\" / \"AI legal assistant.\" This puts you in Harvey's category.\n- \"For legal teams\" with no qualifier. It invites big law, which you've already learned is a bad fit.\n- \"Replaces your lawyer.\" Your buyer *is* the lawyer. The pitch is that they keep up, not that they get replaced.\n- Accuracy claims you can't back up. Lean on \"traceable to precedent\" instead.\n\n---\n\n## 6. Open questions and next tests\n\n1. **Do the 3 payers share a trigger?** For example, a recent funding round, a new sales team, or an outside counsel bill that shocked someone. If so, that trigger is your outbound targeting signal.\n2. **Who signs the check?** The GC, or the CFO because of outside counsel spend? This changes whether you lead with time saved or dollars saved.\n3. **Pilots that didn't convert inside the target segment:** 9 \u2212 3 = 6 non-payers, and context.md only explains two groups. Were any of the 6 good-fit companies? If so, find out why they didn't buy before you tighten the messaging.\n4. **Is 50 signed contracts a hard minimum?** If yes, say so as a qualifier (\"built for teams with 50+ signed contracts\"). It filters out bad fits.\n5. **Pricing anchor:** compare against outside counsel spend, not other software. (Illustrative only: if overflow is 10 contracts a month at 2 hours each, that's about $9,000/month of outside counsel time. Replace this with real customer numbers.)\n6. **Headline test:** run options 1 and 3 from section 3 on a landing page or in cold email and see which gets more replies from in-house lawyers.\n"}]}, "opus-3": {"cost_usd": 0.14154, "seconds": 49, "output_tokens": 4269, "files": [{"name": "positioning.md", "text": "# Quillmark Positioning\n\nDraft, 2026-10-08. Based on `context.md`: 9 pilots, 3 paying.\n\n---\n\n## 1. The positioning in one line\n\n**Quillmark is contract review for small in-house legal teams. It redlines incoming contracts against what your company has already signed, and it cites the precedent behind every suggestion.**\n\nShort version for a homepage or deck: *\"Redline every contract the way you already negotiate. Every suggestion cites your own precedent.\"*\n\n---\n\n## 2. Positioning, piece by piece\n\n### Competitive alternatives (what customers do today)\nCustomers who paid didn't compare Quillmark to Harvey. They compared it to:\n1. **The lawyer reads every page.** This is the default, and it is the real competitor.\n2. **Sending overflow to outside counsel at about USD 450/hour.**\n\nPositioning should be built against these two, not against other AI tools. The AI tools only matter for answering investors and handling objections (see section 4).\n\n### Unique attributes (only Quillmark does these)\n1. **The playbook comes from your last 50 signed contracts, in about 20 minutes.** Nobody has to write a playbook first, and nobody has to set one up for weeks.\n2. **Every suggestion links to the precedent clause it came from.** It doesn't just say \"AI thinks this is risky.\" It says \"you accepted this language in the Acme MSA, March 2025.\"\n\n### Value (what those attributes get the customer)\n| Attribute | Value to a 1\u20133 lawyer team |\n|---|---|\n| Playbook built from your own contracts | Useful on day one. You don't need a playbook project that a 2-person team will never get around to. It flags deviations from *your* positions, not generic \"market standard.\" |\n| Precedent-linked suggestions | The lawyer can check a suggestion in seconds instead of re-researching it. It's defensible to the business (\"we've agreed to this before\"). It's consistent across contracts and across whoever is reviewing. |\n| Both together | The team keeps up with 40\u2013150 contracts a month without hiring and with less outside counsel spend. |\n\n**Gap to close:** We don't have measured outcomes yet. Collect these from the 3 paying customers before claiming any numbers:\n- Minutes per contract, before and after\n- Outside counsel hours avoided per month\n- Share of suggestions accepted as-is\n\nIllustrative only, not a claim: if a team sends 15 contracts a month to outside counsel at 2 hours each, that is about USD 13,500/month. That's the budget Quillmark competes for.\n\n### Best-fit customer (who cares most)\n- **In-house legal team of 1\u20133 lawyers**\n- **At a Series B\u2013D SaaS company**\n- **Signing 40\u2013150 vendor and customer contracts a month**\n- **Already has ~50+ signed contracts** to build the playbook from. This is a hard requirement for the product to work.\n- Strong signal: they already pay outside counsel for overflow, or the GC is the bottleneck on sales deals.\n\n**Not a fit, and should be disqualified early:**\n- **Big law firms.** They need on-prem and a 12-month security review. Their \"precedent\" is spread across many clients, so the core idea fits poorly.\n- **Solo founders and very early companies.** They sign too few contracts and don't have enough signed history to build a playbook.\n\n### Market category\nAvoid **\"AI for legal.\"** That puts Quillmark next to Harvey, a well-funded general platform, and Quillmark loses that comparison.\n\nRecommended frame: **\"contract review for in-house teams\"** (or \"in-house redlining\"). The buyer already understands this category and budgets for it. Inside it, the difference is **precedent-based**:\n\n> Other tools review contracts against what *the law* or *the market* says. Quillmark reviews them against what *you* have agreed to.\n\n\"Precedent-based contract review\" is a candidate subcategory name. Test it in sales calls before committing to it.\n\n### Relevant trend (optional, for investors)\nIn-house teams at growth-stage companies are asked to handle more contract volume without adding headcount, and outside counsel rates keep rising. Leave this out of customer messaging; customers already feel it.\n\n---\n\n## 3. Messaging\n\n**Headline options**\n- \"Redline every contract the way you already negotiate.\"\n- \"Your last 50 contracts are your playbook.\"\n- \"Contract review that shows its work.\"\n\n**Supporting points** (order: fast setup, then trust, then outcome)\n1. **Live in 20 minutes.** Upload your last 50 signed contracts and Quillmark builds your playbook. No playbook-writing project.\n2. **Every suggestion cites your precedent.** See exactly which signed contract each fallback came from. Check it in seconds.\n3. **Keep up without hiring.** Clear routine deviations fast and send less overflow to outside counsel.\n\n**Proof to collect (priority):** one customer quote naming the outside counsel spend they cut, and one time-per-contract number.\n\n---\n\n## 4. \"How are you different from Harvey?\"\n\n**Don't argue about features. Answer by segment and approach.**\n\n> \"Harvey is a broad AI platform built mostly for large law firms and big legal departments. We tried that segment. Big firms wanted on-prem and a 12-month security review, so we deliberately don't sell to them.\n>\n> We serve the 1\u20133 lawyer legal team at a Series B\u2013D SaaS company signing 40 to 150 contracts a month. Today they either read every page or pay outside counsel $450 an hour. We do one job for them: redline incoming contracts against *their own* signed history. The playbook is built from their last 50 contracts in 20 minutes, and every suggestion cites the precedent it came from. A small team trusts that and can use it on day one.\n>\n> 3 of our 9 pilots are paying, and all 3 fit that profile exactly.\"\n\nWhy this works: it shows you chose the segment on purpose and backed it with evidence. It doesn't claim to beat Harvey at Harvey's own game. And it makes the 3 paying customers look like proof of focus rather than a small number.\n\n**Related objections, one line each**\n- *\"Spellbook / Ironclad AI do playbooks too.\"* \u2192 \"Those playbooks have to be written or configured. Ours is built from your signed contracts, and every suggestion is traceable to one of them.\" (**Before using this line, check each competitor's current playbook features.** They change quickly, and an investor or prospect may know them well.)\n- *\"Ironclad does this inside the CLM.\"* \u2192 \"That works if you've already moved onto Ironclad. Most teams our size haven't. Quillmark works on the contract in front of you.\"\n- *\"Can't a general LLM do this?\"* \u2192 \"It can suggest language. It can't tell you what *you* agreed to last quarter, or prove it.\"\n\n---\n\n## 5. Risks and open questions\n\n1. **The moat question.** \"Built from your own contracts\" can be copied as a feature. The lasting advantage is setup speed plus trust from citations, and depth in one segment. Be prepared for \"what stops Harvey from adding this?\" A good answer is focus on the segment, plus retention and usage data once you have it.\n2. **The evidence is thin.** 3 paying customers is a signal, not proof. Every claim above should be checked against those 3. Ask them in their own words why they bought. Whatever words they use become the headline.\n3. **Cold-start edge cases.** What happens when a customer has fewer than 50 signed contracts, or messy ones (scanned PDFs, inconsistent templates)? This sets the lower bound of who is actually a fit.\n4. **Who buys.** The user is the lawyer. The budget may sit with the GC or CFO, framed as outside counsel savings. Find out who signed for the 3 paying customers.\n5. **Whether the 6 who didn't convert fit the profile.** If any of them were small in-house SaaS teams, find out why they didn't buy. That could change the profile.\n\n---\n\n## 6. Next steps\n- [ ] Interview the 3 paying customers on why they bought, which words they use, and the before/after time and spend.\n- [ ] Sort the 6 non-converters: how many were in the target profile?\n- [ ] Check current Spellbook and Ironclad AI playbook features.\n- [ ] Test 2\u20133 headline options on outbound and in demos.\n- [ ] Update the deck: replace any \"AI for legal\" framing with the in-house, precedent-based framing, and add the Harvey answer to the FAQ slide.\n"}]}, "haiku-1": {"cost_usd": 0.00539, "seconds": 29, "output_tokens": 5103, "files": [{"name": "positioning.md", "text": "# Quillmark positioning (draft)\n\n## Recommendation\n\nStop positioning as \"AI for legal.\" That frame puts Quillmark next to Harvey, Spellbook, and Ironclad AI, and it loses on breadth. Position on the workflow instead: first-pass review of incoming counterparty paper against the company's own playbook, for in-house teams that have no legal ops function to configure a platform.\n\n## Positioning statement\n\nFor in-house legal teams of 1-3 lawyers at Series B-D SaaS companies that review 40-150 vendor and customer contracts a month, Quillmark is a redlining assistant that checks incoming contracts against your playbook and proposes fallback language. Unlike general legal AI, it builds the playbook from your last 50 signed contracts in about 20 minutes, and every suggestion links to the precedent clause it came from.\n\n## Three things to lead with\n\n1. **The job.** Flag where the counterparty's paper deviates from playbook and say what to send back. This is the task the buyer already pays outside counsel $450/hour to do.\n2. **Every suggestion is checkable.** Each flag links to the precedent clause it came from. Legal buyers distrust AI output they cannot verify, and this turns a black box into something a lawyer can confirm in seconds. This is the strongest differentiator in the material you have, so lead with it.\n3. **Setup takes minutes, not a project.** The playbook comes from their own signed history, so nobody has to write one from scratch.\n\n## Answer to \"how are you different from Harvey?\"\n\nShort version: \"Harvey is a general legal AI platform, built largely for big firms and enterprise legal departments. We do one workflow, incoming contract review, for in-house teams that don't have the people to configure a platform. Every suggestion links to the precedent it came from.\"\n\nUse the short version in a pitch. Keep the longer points for follow-up questions:\n- We are not trying to be research, drafting, and a platform. Narrower scope means faster time to value for a small team.\n- Our playbook comes from their history, not a template or a consultant's project.\n- Our suggestions are traceable to the customer's own signed language.\n\n## What to avoid saying\n\n- \"AI lawyer\" or anything implying the tool replaces counsel. Lawyers will reject it, and it invites liability questions.\n- \"More accurate than Harvey\" or similar comparative accuracy claims. You have no benchmark, and an investor will ask for one.\n- \"The Harvey for in-house teams.\" It concedes the frame and makes you look like a copy.\n- Anything about big law. You have already learned they don't convert, so do not present them as a future segment without evidence.\n\n## Risks to address before investors do\n\n- **Feature copying.** Harvey, Spellbook, or Ironclad could add \"learn the playbook from past contracts.\" Be explicit about what is defensible: the review workflow, precedent linking, and the accumulated per-customer playbook data. Do not claim the playbook feature alone is a moat.\n- **Market size.** The narrow focus is a strength for sales but invites a TAM question. Build the number: count of Series B-D SaaS companies in your reachable market, times contracts per month, times the outside-counsel cost you displace. Put that number in the deck.\n- **Evidence.** 3 paying of 9 pilots will get asked about conversion and retention. Have an honest answer ready for why the other 6 haven't converted.\n- **Pricing and buyer.** The interviews say the lawyer felt the pain. Find out whether the lawyer or the GC/CFO signs. Positioning for the wrong buyer will stall the deal after a good pilot.\n\n## Competitor facts to verify before use\n\nI wrote the competitor descriptions from memory and did not check current product pages or pricing. Confirm before they go in a deck:\n- Harvey's current customer base and whether it sells to mid-market in-house teams.\n- Spellbook's current product (Word-based drafting and review) and whether it does playbook-based review.\n- Ironclad's AI features and whether its playbook tooling requires setup by a legal ops team.\n\n## Open questions for the founders\n\n- Which feature did the 3 paying customers name as the reason they paid? That is your real positioning proof. If it is the precedent links, lead with it. If it is speed, lead with speed.\n- Do customers review contracts inside Word, a CLM, or Quillmark's own interface? If Word is where lawyers work, a Word-native competitor is a direct threat, and the answer changes the positioning.\n- What do pilots that did not convert say was missing?\n"}]}, "haiku-2": {"cost_usd": 0.00455, "seconds": 22, "output_tokens": 3779, "files": [{"name": "positioning.md", "text": "# Quillmark positioning\n\n## Recommendation\n\nDon't position Quillmark as \"AI for legal.\" That category is crowded and the investor question (\"how are you different from Harvey?\") is a sign the current framing invites the comparison.\n\nPosition it as the contract redlining tool for small in-house legal teams that review high volumes of vendor and customer paper, and that get their playbook from their own signed contracts. The buyer is a 1-3 lawyer team at a Series B-D SaaS company. The promise is that the lawyer keeps control and can check every suggestion against a precedent.\n\n## Positioning statement\n\nFor in-house legal teams of 1-3 lawyers at growing SaaS companies that sign 40-150 vendor and customer contracts a month, Quillmark is a redlining assistant that builds a playbook from the company's last 50 signed contracts in about 20 minutes, flags clauses that deviate from it, and suggests fallback language. Every suggestion links to the precedent clause it came from, so the lawyer can see why it was made and defend it. Unlike general legal AI platforms built for large firms, Quillmark is calibrated to the company's own history.\n\n## Why this works\n\n- **The segment is specific.** Your paying customers share a profile: small team, high volume, no time to write a playbook, and an alternative that is either reading every page or paying outside counsel at USD 450/hour. Say those things directly. They are more persuasive than feature lists.\n- **The wedge is setup speed.** Writing a playbook by hand is the usual reason in-house teams never start. A 20-minute setup from contracts they already have removes that barrier.\n- **Traceability fits how lawyers work.** A lawyer who must justify a position to a business owner or to outside counsel needs to point at the precedent. Linking each suggestion to its source is the answer to \"why are we asking for this?\"\n- **Explicit non-targets focus the message.** Big firms (on-prem, 12-month security review) and solo founders (too few contracts) did not convert. Don't chase them in marketing or product decisions.\n\n## Answer to \"how are you different from Harvey?\"\n\nShort version: \"Harvey is a broad platform for large firms and enterprise legal departments. We're a focused tool for the small in-house team that signs a lot of paper and has no time to build a playbook. We learn from your own signed contracts in about 20 minutes, and every suggestion cites the precedent it came from.\"\n\nCheck this against Harvey's, Spellbook's, and Ironclad's current websites before using it. Competitor positioning changes often, and I haven't verified their current targeting here.\n\n## Risks to address\n\n1. **Precedent is not the same as best practice.** A playbook built from past signed contracts encodes whatever terms the company accepted before, including bad ones. Position the setup as a starting point the lawyer reviews and edits, not an automatic standard. Otherwise the first bad suggestion undermines trust.\n2. **\"Every suggestion links to precedent\" may be copyable.** Competitors can add citations. The harder-to-copy combination is the focused segment, the fast setup from the customer's own corpus, and the workflow built around it. Lead with the combination, not the citation alone.\n3. **Conversion is 3 of 9 pilots.** Find out why the other 6 did not convert before committing to this message. If the reason is price, volume, or integration, the positioning needs to address it, not just the category.\n4. **Harvey moving down-market.** If a larger platform starts serving 1-3 lawyer teams with a cheaper tier, the segment-focus argument weakens. Watch for it.\n\n## Next steps\n\n- Interview the 3 paying customers and record their exact words for the problem and the alternative. Use their language in the website and pitch.\n- Ask the 6 non-converting pilots why they stopped. Sort answers into segment mismatch, product gap, price, and timing.\n- Test the positioning statement with 5 prospects in the target profile. Ask them to repeat it back. If they can't, simplify it.\n- Draft the investor answer and check it against the competitor sites.\n"}]}, "haiku-3": {"cost_usd": 0.0047, "seconds": 21, "output_tokens": 3990, "files": [{"name": "positioning.md", "text": "# Quillmark positioning\n\n## Answer to \"how are you different from Harvey?\"\n\nDon't compete on breadth of legal AI. Compete on the workflow of a 1-3 lawyer in-house team reviewing 40-150 contracts a month, and on the fact that the playbook is built from their own signed contracts.\n\nSuggested line:\n\n> Harvey and similar tools answer \"what does the market say?\" Quillmark answers \"what have we already agreed to?\" Your playbook comes from your own last 50 signed contracts, and every suggestion links back to the clause it came from.\n\nVerify any claim about competitors' products before it goes in a deck. This memo does not check their current features.\n\n## Positioning statement\n\nFor in-house legal teams of 1-3 lawyers at Series B-D SaaS companies who sign 40-150 vendor and customer contracts a month, Quillmark is a contract redlining assistant that flags clauses deviating from your playbook and suggests fallback language. Unlike general legal AI platforms, Quillmark builds the playbook from your own signed contracts in about 20 minutes and traces every suggestion to the precedent it came from.\n\n## Compare against what they replaced\n\nBuyers named two alternatives: \"the lawyer reads every page\" and outside counsel at USD 450/hour. Lead with those, not with model accuracy.\n\n- Cost per contract reviewed, versus outside counsel at USD 450/hour.\n- Hours of lawyer time saved per month at their volume.\n\nCollect these numbers from the paying customers. Don't estimate them.\n\n## Who to target\n\n- **Lead with:** in-house teams of 1-3 lawyers at Series B-D SaaS companies with high contract volume. The three paying customers came from here.\n- **Deprioritize:** big law firms, which want on-prem and a 12-month security review that a pre-seed company can't absorb yet. Solo founders don't have enough contracts. Neither group should shape the homepage or the roadmap.\n\n## Lead demos with the precedent link\n\nA lawyer has to justify a deviation to a business stakeholder. A suggestion that cites the signed clause it came from is easier to trust and defend than a generic suggestion. This is the strongest answer to \"why not Harvey,\" because it's about their data and their history, not model quality.\n\n## Weak spots investors will probe\n\n- **\"Harvey could add this.\"** The defense is the playbook-from-precedent workflow and time to value. Show a real customer going from upload to a usable playbook, not a scripted demo.\n- **Three paying out of nine pilots.** Have the reasons for non-conversion ready, and the reasons the three converted.\n- **Confidentiality.** The playbook is built from the customer's most sensitive documents. Expect security questions even from the target segment. The big-law non-conversion shows security review is real friction.\n- **\"20 minutes\" and \"last 50 contracts.\"** Measure both across pilots before they become headline claims.\n\n## Open questions\n\n- What would the three paying customers say they'd lose without Quillmark? Use their words in the positioning.\n- Is pricing per contract, per seat, or per company? The positioning and the ROI framing depend on it.\n- For most target customers, are the last 50 signed contracts digital and searchable? If not, the 20-minute claim doesn't hold.\n"}]}}, "verdicts": [{"vs": "haiku", "pair": "2", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "haiku", "why": "B better connects setup and traceability to buyer value. A explicitly invents competitors\u2019 inability to know company positions. Both use unverified competitor targeting; B also invents customer setup barriers. Verification caveats do not cure unsupported claims."}, {"vs": "haiku", "pair": "2", "judge": "gpt", "order": "haiku shown first", "winner": "haiku", "why": "A better connects setup and precedent links to buyer value and treats pilot evidence cautiously. Both use unsupported competitor descriptions; B additionally invents buyers\u2019 AI alternatives, competitors\u2019 ignorance of company positions, and citation defensibility against copying."}, {"vs": "haiku", "pair": "2", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "tie", "why": "Both deliverables fully meet all criteria, providing accurate, focused, and actionable positioning grounded in the input."}, {"vs": "haiku", "pair": "2", "judge": "deepseek", "order": "haiku shown first", "winner": "tie", "why": "Both deliver excellent, accurate positioning grounded in the input, with clear, usable language and a credible Harvey answer."}, {"vs": "opus", "pair": "2", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "haiku+skill", "why": "Both identify the paying segment and translate features into useful positioning. Both invent competitor limitations. A makes fewer unsupported claims; B adds unproven speed, exhaustive detection, implementation, competitor strategy, and customer-data moat claims. Its illustrative $9,000 calculation is correct."}, {"vs": "opus", "pair": "2", "judge": "gpt", "order": "opus shown first", "winner": "opus", "why": "A better connects setup and precedent links to buyer value and actual alternatives. Both undermine their Harvey answers with unsupported competitor claims and overstate unproven benefits. B barely explains how the 20-minute playbook reduces setup effort."}, {"vs": "opus", "pair": "2", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "tie", "why": "Both fully meet all rubric criteria with accurate, focused, and actionable positioning."}, {"vs": "opus", "pair": "2", "judge": "deepseek", "order": "opus shown first", "winner": "opus", "why": "A provides more concrete, immediately usable positioning language and a more thorough, actionable answer to the Harvey comparison."}, {"vs": "opus", "pair": "1", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "opus", "why": "B better connects workload, alternatives, and consistent review. Both invent competitor limitations despite verification caveats. A invents deployment speed; B claims unproven capacity for two lawyers handling 150 contracts monthly. B\u2019s illustrative savings arithmetic is correct and labeled hypothetical."}, {"vs": "opus", "pair": "1", "judge": "gpt", "order": "opus shown first", "winner": "opus", "why": "A better connects workload, alternatives, and consistency and recommends usable positioning. Both invent competitor limitations despite verification caveats. A overstates capacity and savings; B additionally fabricates deployment timing and leaves homepage wording undecided."}, {"vs": "opus", "pair": "1", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "tie", "why": "Both deliverables fully meet all rubric criteria with accurate, focused, and actionable positioning."}, {"vs": "opus", "pair": "1", "judge": "deepseek", "order": "opus shown first", "winner": "opus", "why": "A provides more concrete, immediately usable positioning language and a more complete, structured answer."}, {"vs": "haiku", "pair": "1", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "haiku", "why": "B provides clearer usable positioning with fewer fabricated claims. A invents competitor setup delays and Quillmark deployment speed. B still omits segment exclusions, underdevelops workload alternatives, and proposes a market-size calculation missing billable hours per contract."}, {"vs": "haiku", "pair": "1", "judge": "gpt", "order": "haiku shown first", "winner": "haiku", "why": "A supplies a clearer recommended statement and fewer unsupported claims. B better explains targeting and workload, but invents deployment timing and competitor setup/security limitations. Verification caveats do not make those claims credible."}, {"vs": "haiku", "pair": "1", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "tie", "why": "Both deliverables fully meet all rubric criteria with accurate, focused, and actionable positioning."}, {"vs": "haiku", "pair": "1", "judge": "deepseek", "order": "haiku shown first", "winner": "tie", "why": "Both deliverables fully meet all rubric criteria with accurate, focused, and actionable positioning."}, {"vs": "haiku", "pair": "3", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "haiku+skill", "why": "A gives a credible, focused Harvey response. B invents competitors\u2019 market-based orientation; its verification caveat does not repair the proposed wording. Both underdevelop review capacity and consistency. A also makes an unsupported verification-speed claim."}, {"vs": "haiku", "pair": "3", "judge": "gpt", "order": "haiku shown first", "winner": "haiku+skill", "why": "B answers Harvey through Quillmark\u2019s specific workflow; A invents a market-versus-precedent distinction that its verification caveat does not cure. Both underdevelop capacity and consistency benefits. B\u2019s claim that lawyers verify reasoning \u201cin seconds\u201d is unsupported."}, {"vs": "haiku", "pair": "3", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "tie", "why": "Both accurately address all criteria with clear, usable positioning grounded in the input facts."}, {"vs": "haiku", "pair": "3", "judge": "deepseek", "order": "haiku shown first", "winner": "tie", "why": "Both fully meet all criteria with accurate, focused, and actionable positioning."}, {"vs": "opus", "pair": "3", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "haiku+skill", "why": "A offers focused, usable positioning with better evidence calibration, though capacity benefits are underdeveloped and verification speed is unmeasured. B connects workload more clearly but invents competitor limitations, a hard 50-contract requirement, and unproven staffing and spending outcomes."}, {"vs": "opus", "pair": "3", "judge": "gpt", "order": "opus shown first", "winner": "haiku+skill", "why": "B provides focused, usable positioning and a cautious Harvey comparison. A invents competitor limitations, a hard 50-contract requirement, and unproven outcomes. B could better connect workload to capacity and consistency; its seconds-to-verify claim is also unsupported."}, {"vs": "opus", "pair": "3", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "tie", "why": "Both fully meet all criteria with accurate, focused, and actionable positioning."}, {"vs": "opus", "pair": "3", "judge": "deepseek", "order": "opus shown first", "winner": "opus", "why": "A provides more concrete, immediately usable positioning language and a more thorough, actionable plan."}]}, {"id": "int-2", "family": "interviews", "prompt": "I have calls with 6 warehouse managers on Thursday. Background in plan.md. Write my interview guide.", "input": "plan.md", "skill": "fitzpatrick-mom-test", "input_text": "Hypothesis: mid-size warehouses (20-80 pickers) lose a lot of time to mis-picks and would buy a wearable that vibrates when a picker grabs from the wrong bin.\nI have a hardware prototype (wrist band + bin tags). Pricing idea: USD 40 per picker per month.\nThe 6 managers came through a logistics meetup; none has seen the prototype. 45-minute calls, on site, so I could see the floor.\nMy co-founder wants me to demo the band in the first 5 minutes because \"they'll get it instantly\".\nGoal for Thursday: decide whether to keep building hardware or pivot to software only.\n", "rubric": [{"id": "c1", "criterion": "Provides a ready-to-use guide for six 45-minute on-site manager interviews.", "full_marks_when": "Includes a timed agenda totaling 45 minutes, spoken questions, useful follow-up prompts, and space to observe the warehouse floor; accurately treats the managers as meetup contacts who have not seen the prototype.", "zero_when": "Provides only general interviewing advice, an impractical agenda, or a guide based on materially incorrect interview circumstances."}, {"id": "c2", "criterion": "Tests whether mis-picks are a significant, recurring problem without assuming the hypothesis is true.", "full_marks_when": "Starts with open questions about warehouse operations and costly picking problems, then asks for recent concrete mis-pick incidents, frequency, detection, consequences, and supporting records. Separates measured facts from estimates and allows other problems to emerge.", "zero_when": "Leads managers to agree that mis-picks waste substantial time or relies primarily on opinions and hypothetical future behavior."}, {"id": "c3", "criterion": "Uses the on-site setting to test how errors happen and whether the proposed intervention fits actual picking work.", "full_marks_when": "Includes specific observation prompts for the picking workflow, bin identification, when errors occur and are detected, and existing checks; probes whether wrist bands and bin tags could reliably detect and prevent the observed errors, including installation, worker acceptance, and operational constraints.", "zero_when": "Ignores floor observation or assumes that grabbing from the wrong bin explains all mis-picks and that the prototype would prevent them."}, {"id": "c4", "criterion": "Investigates purchasing reality and costs accurately rather than treating enthusiasm as willingness to pay.", "full_marks_when": "Asks about current solutions, their costs and shortcomings, budget ownership, approval steps, and evidence of spending or a concrete pilot commitment. If presenting the USD 40 per picker per month idea, does so after learning current costs and correctly states that 20\u201380 paid picker seats would cost USD 800\u20133,200 per month, without equating picker headcount with confirmed paid seats or inventing savings.", "zero_when": "Uses a simple 'Would you buy this?' as purchase validation, invents financial benefits, or misstates the supplied price or its arithmetic."}, {"id": "c5", "criterion": "Protects discovery from an early prototype pitch while making any demo informative.", "full_marks_when": "Explicitly addresses the co-founder's proposed first-five-minute demo and delays it until independent problem and workflow evidence has been collected. Any later demo uses neutral questions about fit, failures, limitations, and next steps, with reactions recorded separately from pre-demo evidence.", "zero_when": "Opens with the prototype pitch or treats immediate excitement, compliments, or comprehension as evidence of demand."}, {"id": "c6", "criterion": "Produces evidence that can support Thursday's hardware-versus-software decision.", "full_marks_when": "Includes consistent notes across all six calls and a practical synthesis rule comparing problem severity, current alternatives, purchasing evidence, hardware feasibility, and whether software could address the observed causes. Allows continued hardware work, software exploration, or insufficient evidence, and acknowledges that six meetup contacts do not establish market-wide demand.", "zero_when": "Offers no way to turn interview findings into the stated decision, forces a pivot from weak evidence, or treats favorable reactions from six contacts as conclusive validation."}], "runs": {"haiku-skill-1": {"cost_usd": 0.01044, "seconds": 55, "output_tokens": 10064, "files": [{"name": "interview-guide.md", "text": "# Warehouse manager interview guide\n\n**Calls:** 6 warehouse managers, Thursday, 45 minutes each, on site\n**Mode:** Learning only. No pitch, no pricing, no commitments on Thursday.\n**Decision this informs:** keep building the wristband hardware, or pivot to software only.\n**Sample note:** all 6 came from one logistics meetup. Treat them as a convenience sample. Count specific episodes, not enthusiasm.\n\n## Three things we need to learn\n\n1. **What mis-picks cost, in the manager's own facts and numbers.**\n   - Assumption: mis-picks waste real time and money at sites with 20-80 pickers.\n   - Would undermine it: mis-picks are rare, caught cheaply later, or small next to problems the manager is working on.\n   - If so: neither hardware nor software is clearly needed here.\n\n2. **What they already do about it, and what happened when they tried.**\n   - Assumption: existing controls (scanning, WMS checks, training) leave a gap a band could fill.\n   - Would undermine it: current equipment or a software fix already works well enough, or they tried a wearable or handheld before and dropped it.\n   - If so: a software-only path becomes the default.\n\n3. **What it takes to put new hardware on a floor.**\n   - Assumption: a wristband and bin tags could be rolled out without a long approval process.\n   - Would undermine it: IT, safety, HR, or worker representatives would block body-worn devices or any tracking of individual pick accuracy; or charging, sizing, gloves, and damage make devices impractical.\n   - If so: the hardware path needs a different buyer and approval plan, or stops.\n\n## Before the call\n\n- Agree who leads and who takes notes, especially if the co-founder attends.\n- Do not show the band or mention the price until the demo slot at the end.\n- Ask for 5 minutes on the floor. Ask them to pick the area.\n\n## Call structure\n\n| Time | Section |\n| --- | --- |\n| 0-5 | Opening and site context |\n| 5-20 | Last mis-pick episode |\n| 20-32 | Current controls and past attempts |\n| 32-40 | Past rollouts, approval, and floor reaction |\n| 40-45 | Floor observation, optional demo, close |\n\n## 0-5: Opening\n\n\"Thanks for making time. We're trying to understand how picking accuracy works at sites like yours. We're not selling anything today. I'd like to hear about your operation and one recent example, and I'll take notes.\"\n\n- Roughly how many pickers per shift, and how many orders per day?\n- How do you pick: batch, zone, wave, or something else? Which WMS or system do you use?\n- Who signs off on equipment spending at your site?\n\n## 5-20: Last mis-pick episode\n\nOpening: \"Think about the last time someone grabbed from the wrong bin, or a wrong item reached a customer. Walk me through what happened.\"\n\nProbes, chosen according to what they say. Don't read the list in order.\n- How did you find out? Who told you, and when?\n- What happened next, step by step?\n- How long did the fix take, and who was involved?\n- What did it cost: returns, reshipping, credits, extra labour, complaints?\n- Was that typical? When did you last see one like it?\n- How many happen in a typical week? How do you know? Do you log them? Could you show me the log?\n\nEvidence sought: a reported past episode with a real cost, plus the log if one exists (observed artifact).\n\n## 20-32: Current controls and past attempts\n\n- How does a picker confirm they're at the right bin today? A scanner at each location, voice, pick-to-light, or nothing?\n- Where does your system catch errors, and where does it miss them?\n- Tell me about the last change you made to picking accuracy. What was it, why, who decided, and what did it cost?\n- Have you tried something that didn't work? What happened, and why did you stop?\n- Have you looked at a wearable or hands-free device before? What happened?\n- Which current steps still leave a gap? (Record this as interpretation, in the manager's own words.)\n\nIf they describe a cheap or software fix, ask: \"What did that fix cost, and what does it still miss?\"\n\n## 32-40: Past rollouts, approval, and floor reaction\n\nKeep these about things that have already happened. Avoid asking how pickers would feel.\n\n- Tell me about the last piece of hardware you rolled out to pickers. Who asked for it, who approved it, and how long did it take?\n- Did it stick? If people stopped using it, what happened?\n- If a device recorded speed or accuracy per picker, has that come up before? What happened?\n- What happened to devices on shift: charging, damage, loss, cleaning, gloves?\n- Who would need to sign off on something worn on the body?\n\n## 40-45: Floor observation, optional demo, close\n\n**Do not demo before this point.** A demo first makes people polite about the product, and we lose the chance to hear the problem in their words. If the co-founder wants them to see the band, it goes here, after the questions.\n\n- Ask: \"Could you show me how a pick works, from the trolley to the bin?\" Watch for: confirmation steps, timing, where pickers look, where the error could happen. Record these as observed behavior.\n- If there's time and they're interested, show the band and bin tag. Then ask: \"Walk me through where this would have helped on the shift you described.\" Tag everything they say after the demo as reaction to the product.\n- Do not ask whether they like it, would buy it, or what it should cost.\n- Close: \"What did we miss? Who else on the operations or IT side should we talk to?\"\n\n## Do not ask\n\n- \"Would you use this?\" or \"Would you buy this?\"\n- \"Does $40 per picker per month sound right?\"\n- \"Don't you think mis-picks are a big problem?\"\n- \"How much would this save you?\" Ask for the actual cost of the last episode instead.\n- \"Would your pickers accept a wearable?\" Ask about the last rollout instead.\n\n## Notes template, one per call\n\nTag each entry: [Observed] [Reported past] [Opinion] [Commitment] [Interpretation] [Unknown]\n\n- **Site:** pickers per shift, pick method, system\n- **Last episode:** trigger, steps, cost, outcome [Reported past]\n- **Current controls:** what they catch and what they miss\n- **Past attempts:** what was tried, and why it ended\n- **Approval and floor reaction:** from actual past events only\n- **Floor observation:** [Observed]\n- **Demo reactions:** [Opinion], influenced by the product\n- **Contradictions:** anything that cuts against our three assumptions\n- **Follow-ups:** requested / agreed / completed\n\n## Thursday readout: what would push which way\n\nAgree on the thresholds before Thursday. For example, how many of the 6 need to describe a recent, costly, uncaught mis-pick before we call the problem real. The signals below are directional only.\n\n- **Toward keeping hardware:** several managers describe recent, costly mis-picks that current controls miss, and a rollout path that doesn't run into IT, safety, or worker blockers.\n- **Toward software only:** managers say existing scanning, WMS checks, or training mostly work, or a cheap software change stopped the problem.\n- **Toward stopping both:** mis-picks are rare or cheap, or nobody tracks them.\n"}]}, "haiku-skill-2": {"cost_usd": 0.01272, "seconds": 73, "output_tokens": 13787, "files": [{"name": "interview-guide.md", "text": "# Warehouse Manager Interview Guide\n\n**Purpose:** Learn how wrong-bin picks happen, what they cost, and what already catches them. Decide whether to keep building hardware or pivot to software only.\n\n**Format:** 45 minutes, on site, six managers from one logistics meetup. None has seen the prototype. Use this same guide for all six so the notes are comparable.\n\n**This is a learning call, not a sales call.** Don't mention price (USD 40/picker/month) in any part of the call.\n\n## Before the call\n\n- Confirm the site fits the target: roughly 20\u201380 pickers. Note it if it doesn't.\n- Note each manager's role: buyer, approver, or advocate. Ask in Section 6 if unclear.\n- Keep the prototype out of sight until Section 7.\n- Label every note as you write it: **Observed**, **Reported past**, **Opinion**, **Commitment**, **Interpretation**, or **Unknown**.\n\n## Learning objectives\n\n1. **How often do wrong-bin picks happen, what do they cost, and does anyone measure them?** Could undermine the plan if the cost is small or nobody tracks it because it isn't a priority.\n2. **What already catches them?** Could undermine the plan if scanning or the WMS already stops most wrong-bin picks, so a wearable adds little and a software-only fix may be enough.\n3. **What would have to be true on the floor and in purchasing for a new device to be adopted?** Could undermine the plan if no one with budget authority is reachable, or if the site rules rule out devices on pickers.\n\n## Opening (2 min)\n\nSay something close to this:\n\n> \"Thanks for making time. I'm talking to warehouse managers about wrong-bin picks, meaning a picker takes an item from the wrong location or the wrong SKU, and how you handle them. I'm building something in this area, so I'll ask mostly about your current process. This isn't a pitch, and 'that's not a problem for us' is a useful answer. Is it OK if I take notes?\"\n\n## Section 1: Your operation (3 min)\n\n- Walk me through a typical shift. How many people pick, and across how many zones?\n- What are you responsible for day to day? Who do you report to?\n- How are picks checked today?\n\n*Note headcount against the 20\u201380 range.*\n\n## Section 2: The last wrong pick (13 min)\n\n**This is the priority section. If time runs short, cut Sections 1 and 3 before this one.**\n\n- Think about the last time a wrong item or wrong bin caused a problem. Walk me through what happened, from the picker getting the order to the moment you knew about it.\n\nFollow-ups, used as needed. Don't read them all out:\n- How did you find out: a customer, a returns check, a cycle count, a supervisor?\n- Where in the building was it? If it's allowed and safe, can we walk over to that area? *Observe bin labels, tag placement, and how the picker confirms the location.*\n- What happened next? Who was involved? How long did it take to resolve?\n- What did it cost in time, returns, reshipping, or credits? How did you arrive at that number?\n- What did you change afterward, if anything?\n- When was the time before that?\n\n*Keep follow-ups short: \"And then?\", \"What did you do next?\" Don't explain the problem or your idea in your own words.*\n\n## Section 3: Frequency and measurement (4 min)\n\n- How many wrong picks happened last month? How do you know?\n- Do you track them? Where? Could you show me the report or a recent example?\n- Which of your numbers do you trust most? Least?\n\n*If they don't track it, ask: \"What would you estimate, and what is it based on?\" Record the answer as Reported or Unknown.*\n\n## Section 4: What you've already tried (6 min)\n\n- What have you tried to reduce wrong picks in the last year or two? Training, bin labels, zone changes, incentives, scanners, pick-to-light, system rules.\n- For the one that worked best, what made it work? For one that didn't, what happened?\n- What does your scanning or WMS check at the bin? Does it stop a wrong-bin pick before the item leaves the shelf, or only after?\n- What did you spend on accuracy tools last year? Who approved it?\n\n*This section tests the software-only alternative. Note whether existing checks already catch most wrong-bin picks. Don't suggest solutions.*\n\n## Section 5: The floor and pickers (4 min)\n\nPickers aren't on the call, so anything the manager says about them is Reported or Opinion.\n\n- Have you rolled out anything new that pickers had to wear, carry, or use? What happened?\n- What rules apply to devices or wearables on your floor: safety, HR, union, hygiene, sizing, charging?\n- Who would have to be consulted before a pick-related device went on the floor?\n\n*Don't ask whether pickers would like a vibrating band. That produces an opinion, not evidence.*\n\n## Section 6: Buying (4 min)\n\n- The last piece of warehouse equipment or software you bought: who started it, who approved it, how long did it take?\n- Which budget would this come from? Who controls it?\n- Who else has to agree before a pilot could run?\n\n*If they lack authority, ask who has it. That sets up the introduction ask in Section 8.*\n\n## Section 7: The prototype (6 min)\n\n**Only after Section 6.** Showing it first makes every later answer a reaction to the product, not to their problem. Their instant understanding is not evidence of demand. Vibration is hard to judge from a description, so once you have their process, let them feel it.\n\n1. Show the band and one bin tag. Describe it in one sentence: \"It buzzes the picker's wrist when they reach into a bin other than the one the order calls for.\"\n2. Hand it over. Ask them to try it, or watch a picker try it if you're on the floor. Note what they do: put it on, check the fit, ask about charging or battery, look at the tag.\n3. After their reaction, ask:\n   - Where would the tags go on your shelving, and what would have to change?\n   - What would stop this from working on your floor?\n   - Who would wear it on a shift, and what happens at shift change?\n   - What would you need to see before one zone tried it?\n4. Don't ask: \"Would you buy this?\", \"Would you pay [price]?\", or \"Do you like it?\"\n\n*Mark everything said after the demo as post-pitch.*\n\n## Section 8: Next step (3 min)\n\nChoose one ask before the call, matched to what you can actually deliver this week. Don't improvise a pilot you can't run.\n\n- Send the last three months of wrong-pick or return data for one zone. *(Time and data.)*\n- Introduce me to the person who approves tools or budget. *(Introduction.)*\n- A two-week test in one zone with three to five pickers. *(Time. Only if the hardware is ready.)*\n\nAsk directly: \"Would you send me [X] by [date]?\"\n\n- **Yes:** Record as Requested and Agreed. Mark Completed only when you actually receive it or it happens.\n- **No:** \"What would make that worth doing?\" Record as Declined, with their reason.\n\nClose by thanking them and stating only the follow-up you have committed to.\n\n## Right after each call (5 min)\n\nFill in before the next call:\n\n- Headcount, and whether the site is in the 20\u201380 range\n- Wrong-pick frequency and cost, with the exact figure and its source\n- Whether existing scans or the WMS catch wrong-bin picks: yes, no, or partly\n- Who approves and which budget applies\n- Past rollouts of devices, and any rules that apply\n- Next step: requested, agreed, completed, or declined, with the date\n- Reactions to the prototype, marked post-pitch\n- Unknowns\n\n## Reading the six calls on Thursday\n\nThe thresholds below are my starting judgment, not validated cutoffs. Adjust them if the calls point somewhere unexpected.\n\n- **Hardware case is supported** if about four of six managers report recurring wrong-bin picks with a cost they can estimate, existing checks miss wrong-bin picks in their operation, and at least two agree to a concrete next step.\n- **Software-only case is supported** if most say scanning or the WMS already stops wrong bins, the cost is small or unmeasured, or the real pain is reporting.\n- **Unclear** if the calls produce enthusiasm without numbers, or agreed next steps don't happen.\n\nSix managers from one meetup is a small, non-random group. Treat it as evidence about these six sites, not about market size.\n"}]}, "haiku-skill-3": {"cost_usd": 0.0097, "seconds": 48, "output_tokens": 8823, "files": [{"name": "interview-guide.md", "text": "# Warehouse manager interview guide\n\n45 minutes, on site, 6 managers. Mid-size warehouses (20\u201380 pickers).\n\n## Purpose\n\nDecide whether to keep building the wrist band or pivot to software only.\n\nTo make that call, learn:\n\n1. Whether mis-picks are a costly, recent, recurring problem, and whether anyone measures them.\n2. What they already do about it, and what happened to those attempts.\n3. How equipment decisions get made, and whether a wearable would survive the floor.\n\nThis is a learning call, not a sales call. Ask about past events, not whether they like the idea. Any pitch comes late and is marked as such in your notes.\n\n## Before the call\n\n- Confirm headcount of pickers on a typical shift. Outside 20\u201380, note it; it limits the conclusion.\n- Ask for a walk of the pick area (10 minutes, can run during or after the talk). Watch how a wrong pick gets caught: scanner confirmation, put-to-light, paper, or nothing.\n- Bring a notebook and a copy of the timing plan below. Ask permission before recording anything.\n- If the manager has a mis-pick or returns report, ask to see it. Something they show you is observed evidence; something they describe is reported.\n\n## Opening (2 min)\n\n\"Thanks for making time. I'm trying to understand how picking errors actually get caught here and what they cost you. I'm not here to sell anything today. I'd like to hear about specific recent events, so I'll ask for stories more than opinions. Is it okay if I take notes?\"\n\n## Timing\n\n| Min | Section | Goal |\n|---|---|---|\n| 0\u20132 | Opening | Set expectations |\n| 2\u201312 | A. Context and floor | Confirm fit; see how errors surface |\n| 12\u201325 | B. Last mis-pick episode | Reconstruct a real event and its cost |\n| 25\u201332 | C. Tried and tools | What they did about it and what happened |\n| 32\u201338 | D. Decisions and floor acceptance | Who decides; how workers respond to devices |\n| 38\u201342 | E. Stimulus (demo) | Reactions to specific moments only |\n| 42\u201345 | F. Next step and close | Ask for one concrete step, or none |\n\nIf time runs short, skip E and shorten A. Do not skip B.\n\n## A. Context and floor (10 min)\n\n- Walk me through a normal shift for one picker. How many picks an hour? How many pickers on a shift?\n- What do pickers use: handheld scanners, voice, paper, pick-to-light, put walls?\n- How do you find out a wrong item or wrong bin was picked? Who notices first, and when?\n  - Probe: Is that usually found at pick, at pack, or by the customer?\n\n## B. Last mis-pick episode (13 min)\n\nCore section. Spend most of your time here.\n\n- Tell me about the last time a picker took the wrong item or put something in the wrong bin. Start from when it happened.\n  - Probes: How did you find out? What happened next? Who did you call? How long until it was fixed? What did it cost (time, reshipping, complaints, labour)?\n- Was it recorded anywhere? Can I see the record?\n- What did you change afterward, if anything?\n- If nothing comes to mind: \"When was the last week that errors were bad? What happened that week?\"\n- If they describe a general pattern (\"it happens all the time\"): \"Think of the most recent one. What was it, and when?\"\n\nRecord: date, product or area, how it was caught, time to resolve, cost as reported or as measured.\n\n## C. What's been tried, and tools (7 min)\n\n- In the last two years, what have you tried to reduce picking errors?\n  - Probes: What happened to it? Who decided to stop or keep it? Is it still in use?\n- Have you looked at scanner confirmation, pick-to-light, voice picking, or a warehouse system upgrade? What happened with each?\n- Do you track mis-pick rate today? Where, and how often is it reviewed?\n  - Probe: Is that number measured or estimated?\n\n## D. Decisions and floor acceptance (6 min)\n\n- Think of the last change to how pickers work, such as new scanners or a new system. Walk me through how it got approved: who asked, who signed, how long it took, roughly what it cost.\n- If a new device were worn by pickers, who would need to say yes?\n- Has there been budget for error reduction this year? Where does it sit?\n  - Do not ask about price here.\n- When you last rolled out a device to pickers, how did they react? What did they push back on?\n- Are there safety, hygiene, or labour-relations concerns about devices or alerts on the body during a shift?\n- How many alerts do pickers already get from their current equipment?\n\n## E. Stimulus: the band (4 min, optional)\n\nOnly after A\u2013D. If time is short, skip this section.\n\nSay: \"I'd like to show you one thing and get your reaction to specific moments, not an overall opinion.\"\n\nShow the band and its bin tags. Then ask:\n\n- Walk me through a normal shift where a picker grabs from the wrong bin. Who sees what, and when?\n- What would you have to change in your process to use this?\n- What would have to be true for pickers to wear it for a full shift?\n- What do you pay today for the closest thing you use?\n\nDo not ask: \"Would you buy it?\", \"What would you pay?\", \"Do you like it?\"\n\nIf they ask about price: \"We're still working that out. What do you pay today for the closest thing?\"\n\nRecord reactions as opinion unless they describe a specific action.\n\n## F. Next step and close (3 min)\n\n- Who else should I talk to? Especially whoever approves equipment, a floor supervisor, and safety or HR.\n- Is there a specific next step that would be useful to you? Examples:\n  - Share one week of mis-pick data.\n  - Introduce the operations or IT lead.\n  - Observe a shift or run a short floor trial with two or three pickers.\n- If they say \"send me information\": \"What would you want to read first? What would make 30 minutes of your team's time worth it?\"\n\nRecord each ask as requested, agreed, completed, or declined. A promised introduction is requested until it happens.\n\n## Interviewer rules\n\n- Do not pitch in the first 30 minutes.\n- Do not defend the idea. If they criticise it, ask what they have seen go wrong before.\n- Do not treat \"great idea\" or \"I'd use that\" as evidence. Ask about the last time they did the related thing.\n- Do not ask feature questions. If someone requests a feature, ask what result they want and how they get it today.\n- Keep your own stories short. Return to their episode.\n\n## Evidence capture (one row per call)\n\n| Field | Notes |\n|---|---|\n| Site, headcount, role | Confirms fit and who this person is |\n| Last mis-pick episode | Date, how caught, time to resolve, cost (reported or measured) |\n| Measured? | Yes / no / partly |\n| Previous fixes | What, when, outcome, still in use? |\n| Budget owner and approval path | Named person, last approval, how long it took |\n| Floor objections | Labour, safety, hygiene, alert fatigue |\n| Reaction to stimulus | Quote, and whether it's opinion or a described action |\n| Next step | Requested / agreed / completed / declined, with what |\n\nLabel each claim as observed, reported past behaviour, opinion, commitment, interpretation, or unknown.\n\n## Deciding on Thursday evening\n\nSet these thresholds before the first call. The numbers below are suggestions for you to adjust.\n\nKeep building hardware if most of these hold:\n\n- At least 4 of 6 describe a specific mis-pick episode from the last three months, with a cost they can estimate.\n- At least 3 of 6 name a budget owner and have bought floor equipment in the last year.\n- Fewer than 2 of 6 raise a labour, safety, or hygiene blocker on wearables.\n- At least 2 of 6 agree to a concrete next step.\n\nPivot to software only if:\n\n- Most managers say mis-picks are rare, cheap, or not measured.\n- Existing scanner confirmation, pick-to-light, or system features already handle the problem to their satisfaction.\n- Floor or labour objections come up repeatedly.\n\nWith six calls, treat any pattern as a signal, not proof. Note the sites that don't fit the 20\u201380 picker range separately.\n\n## Note on the demo\n\nYour co-founder wants the band demoed in the first five minutes because \"they'll get it instantly.\" This guide puts it at minute 38 instead. Early demos produce praise that describes the device, not the problem. Quick understanding is also not evidence of need. Discuss this with your co-founder before Thursday; if you agree to demo early on some calls, record which calls and treat those reactions as contaminated by the pitch.\n"}]}, "opus-1": {"cost_usd": 0.16504, "seconds": 62, "output_tokens": 5238, "files": [{"name": "interview-guide.md", "text": "# Interview Guide: Warehouse Managers (Thursday, 6 calls)\n\n**Purpose of Thursday:** decide whether to **keep building hardware** or **pivot to software only**.\n**Format:** 45 min, on site, 1:1 with a manager of a 20\u201380-picker operation.\n**Hypothesis being tested:** mis-picks cost mid-size warehouses a lot of time, and they'd pay ~USD 40/picker/month for a wrist band that vibrates on a wrong-bin grab.\n\n---\n\n## Read this first: why the demo goes at the end, not the start\n\nYour co-founder wants to demo the band in the first 5 minutes. I'd push back on that. Here's why:\n\n- **If you demo first, every later answer is about your band and not about their warehouse.** People are polite to founders holding a prototype. \"That's cool\" isn't evidence, and you'd learn nothing about whether mis-picks are their real problem.\n- **Thursday is a hardware-vs-software decision.** To make it, you have to hear how they pick today, what they already carry or scan with, and what they think of new devices, *before* they've seen your answer. Otherwise you can't tell \"they want a wearable\" apart from \"they were being nice about the wearable.\"\n- **Your co-founder still gets their test.** The demo happens at minute ~33, and you note the first thing they say and do. If they \"get it instantly,\" you'll see it then. And you'll also know whether getting it matters for a problem they actually have.\n\nIf you want a compromise: keep the band in your bag and out of sight until the demo section.\n\n---\n\n## Before each call (5 min)\n\n- [ ] Look up the company: what they ship, how many sites, any WMS vendor mentioned on LinkedIn/job posts.\n- [ ] Charge the band, test the bin tags, bring a spare.\n- [ ] Print the debrief sheet (bottom of this doc), one per call.\n- [ ] Ask for permission to take notes (and record, if they're OK with it).\n- [ ] Bring a second person to take notes if you can. You'll be walking and talking.\n\n## Rules of thumb during the call\n\n- Ask about **the last time it happened**, not \"would you\" or \"do you usually.\" Past behavior is evidence; predictions aren't.\n- When you hear a number (\"we get a lot of mis-picks\"), follow up once: *\"Roughly how many last week? How do you know?\"*\n- Don't pitch, correct them, or explain your idea before the demo section.\n- When they complain, ask *\"What have you tried to fix that?\"* If they've tried nothing, it probably isn't a priority.\n- Let silence sit. The useful stuff often comes after a pause.\n\n---\n\n## Call structure (45 min)\n\n| Time | Section | Goal |\n|---|---|---|\n| 0\u20133 | Intro | Set the frame: you're learning, not selling |\n| 3\u201313 | Their operation | How picking works today, tools, people |\n| 13\u201323 | Floor walk | See it; ask questions in context |\n| 23\u201333 | Errors & what they cost | Is mis-picking a real, costly, owned problem? |\n| 33\u201340 | Demo + reaction | Reaction to the band; hardware vs. software signals |\n| 40\u201345 | Buying & close | Who buys, how, next step |\n\nIf time runs short, cut the floor walk down. **Never cut the errors section.**\n\n---\n\n### 1. Intro (0\u20133 min)\n\n> \"Thanks for having me. I'm working on tools for picking accuracy in warehouses your size. Today I mainly want to understand how your operation works and what's actually painful. I'm not here to sell anything. Near the end I'll show you a rough prototype and get your honest take. Harsh is more useful than nice. OK if I take notes?\"\n\n### 2. Their operation (3\u201313 min)\n\n1. Walk me through what a picker does from the start of a shift to the first order out the door.\n2. How many pickers per shift? How many shifts? How many SKUs, and how many pick locations?\n3. What do pickers use to know what to pick? *(Paper, RF scanner, voice, pick-to-light, tablet, something else?)*\n4. Do they scan the bin or item to confirm the pick? Is that scan enforced or skippable?\n5. What WMS or system are you on? Who looks after it: you, IT, a vendor?\n6. What do pickers already wear or carry? *(Gloves, scanners, ring scanners, headsets, PPE)*\n7. What's turnover like among pickers? How long does it take to train a new one?\n\n> **Listen for:** scan verification already in place (that's competition, and it may mean the hardware isn't needed); paper or no-scan processes (strongest fit for the band); gloves and high turnover (matter for wearable design and charging).\n\n### 3. Floor walk (13\u201323 min)\n\nAsk to see the pick area. Watch for a few minutes before you ask anything.\n\n**Watch for:**\n- [ ] How fast picks happen. Where do pickers hesitate or double-check?\n- [ ] Bin layout: similar-looking SKUs next to each other? Clear labels?\n- [ ] What's on their hands and wrists (gloves, scanners, watches, nothing)?\n- [ ] Where devices get charged and stored. Is there a charging station already?\n- [ ] Wi-Fi/connectivity: dead zones, metal racking, cold rooms.\n- [ ] Where errors get caught: pack station, QC, customer returns?\n\n**Ask in context:**\n- \"Where do most wrong picks happen, physically? Can you show me?\"\n- \"What happens right here when someone grabs the wrong item?\"\n- \"If you added one more device for pickers, where would it live overnight?\"\n\n### 4. Errors and what they cost (23\u201333 min) \u2014 the most important section\n\n1. Tell me about the last mis-pick that really hurt. What happened, and what did it cost?\n2. Roughly how many mis-picks do you get in a week? **How do you know?** Is it tracked, or a guess?\n3. Where do they get caught? At packing, QC, or by the customer?\n4. What does one mis-pick cost you? *(Re-pick labor, shipping, returns, chargebacks, lost customer)*\n5. What have you tried to reduce them? How did that go? What did it cost?\n6. Who gets in trouble when the error rate goes up? Is accuracy one of your KPIs?\n7. Where does mis-picking rank among your top 3 headaches right now? What are the other two?\n\n> **Listen for:** a specific recent story, a tracked number, money already spent on fixes, accuracy as someone's KPI. **Red flags:** \"It happens sometimes,\" no tracking, never tried to fix it, or it isn't in their top 3. If they rank something else higher (labor shortage, pick speed, slotting), dig into that. It may be your software pivot.\n\n### 5. Demo and reaction (33\u201340 min)\n\nTake out the band now. Show it, then **let them handle it** (put it on, grab from a wrong bin).\n\nAfterward, write down the **first thing they say or do**, word for word if you can.\n\n1. \"What's your gut reaction?\"\n2. \"Where would this help most here? Where would it get in the way?\"\n3. \"What would your pickers say about wearing it all shift?\"\n4. \"What breaks this in your building?\" *(Gloves, charging, losing it, hygiene/sharing, cold, union or privacy concerns, tagging thousands of bins)*\n5. \"Who would put the bin tags on and keep them updated when you re-slot?\"\n6. **Hardware vs. software probe:** \"Suppose you got the same wrong-bin alert on the scanner or device pickers already carry, with no new hardware. Better, worse, or no different than the band?\" \u2192 *Why?*\n7. \"What would it need to connect to? Your WMS?\"\n\n> **Listen for:** whether they care about the *alert* (problem \u2192 software may be enough) or the *hands-free, no-scan* part (that's what only hardware gives you). Also: tagging and charging pain may sink the hardware even if they like the idea.\n\n### 6. Buying and close (40\u201345 min)\n\n1. \"Last time you bought a new tool for the floor, how did it work? Who signed off? How long did it take?\"\n2. \"What do you spend per picker per month on tools/devices today?\" *(Don't name your price first.)*\n3. Only after that: \"If this cut mis-picks by half, what's that worth to you per picker per month?\" Then, if useful: \"We've been thinking around $40. How does that land?\"\n4. **The real test, so ask it:** \"Would you run a 4-week pilot on one zone or shift?\" Or, if they lean toward software: \"Would you share a month of mis-pick data so we can see if we can predict errors?\"\n5. \"Who else should I talk to? Another manager, your ops director, IT?\"\n\n> **Strong signal:** they agree to a pilot with a date, name a budget owner, introduce you to someone, or offer data. **Weak signal:** \"Send me some info,\" \"Interesting, keep me posted.\"\n\nThank them. Send a follow-up within 24 hours that names the next step.\n\n---\n\n## Debrief sheet (fill in within 15 min of each call)\n\n**Company / manager / # pickers:**\n**Current picking method:** paper / RF scan / voice / pick-to-light / other: ____\n**Scan verification already enforced?** Y / N\n**Mis-pick rate (and is it tracked?):**\n**Cost per mis-pick (their words):**\n**Mis-pick in their top 3 problems?** Y / N. What were the top 3?\n**Already tried to fix it? With what?**\n**First reaction to the band (verbatim):**\n**Hardware blockers raised:** gloves / charging / tagging / hygiene / privacy / connectivity / other\n**Band vs. alert on existing device:** prefers band / prefers existing device / no preference. Why:\n**Current spend per picker/month:**\n**Reaction to $40:**\n**Commitment offered:** pilot (date?) / data / intro / none\n**Surprise of the call:**\n\n---\n\n## Thursday evening: making the decision\n\nPut the 6 debrief sheets side by side and count.\n\n**Problem is real (go/no-go for the whole idea):**\n- Mis-picks in the top 3 at **\u22654 of 6** sites, with a tracked number or a specific costly story.\n- If fewer: neither hardware nor software. Look at what they *did* rank top 3. That may be the real opportunity.\n\n**Keep building hardware if most of these hold:**\n- Several sites pick **without enforced scan verification** (paper, voice, or skippable scans).\n- They valued **hands-free/no extra step** over just getting an alert.\n- Hardware blockers (tagging, charging, gloves) came up but felt solvable, not deal-breakers.\n- **\u22652 pilot commitments with a date**, at or near $40/picker/month.\n\n**Pivot to software only if most of these hold:**\n- Most sites already carry RF scanners or devices and said \"alert on my existing device is fine/better.\"\n- Tagging bins, charging, or wearing a band all shift came up as real objections.\n- They offered **data or WMS access** more readily than pilot space for the band.\n- Willingness to pay came out closer to software-seat prices than device-rental prices.\n\n**If it's mixed:** look at *which* sites liked the hardware (paper-based? smaller? higher turnover?). The answer may be \"hardware for a narrower segment,\" not a yes or no.\n\nKeep in mind that 6 interviews from one meetup is a small, friendly sample. Treat the result as a strong direction, not proof, and use the pilot commitments as the deciding vote.\n"}]}, "opus-2": {"cost_usd": 0.14957, "seconds": 53, "output_tokens": 4595, "files": [{"name": "interview-guide.md", "text": "# Warehouse Manager Interview Guide \u2014 Thursday\n\n**Format:** 45 min, on site, 6 managers (mid-size, 20\u201380 pickers)\n**Decision this feeds:** keep building hardware (band + bin tags) vs. pivot to software only.\n\n---\n\n## Before you go: the demo question\n\nYour co-founder wants to demo the band in the first 5 minutes. **I'd push back and demo at minute ~33 instead.** Here's why:\n\n- **Once they've seen the band, every answer is about the band.** You'll get reactions to your solution (\"cool, would the battery last a shift?\") instead of facts about their problem. You can't un-show it.\n- **\"They'll get it instantly\" is the risk, not the upside.** Managers who met you at a meetup will be polite. A cool gadget in hand produces enthusiasm that feels like validation but doesn't predict buying.\n- **The hardware-vs-software decision depends on *why* mis-picks happen.** If wrong-bin grabs are the main cause, a wrist band fits. If the causes are bad slotting, wrong quantities, mislabeled bins, or WMS data errors, a band doesn't help and software might. You only learn this if you ask before showing them anything.\n- **Demoing late still gives your co-founder what they want.** You get the reaction. And because you already know their pain, you can tell whether the reaction is real.\n\nA compromise if your co-founder insists: bring the band, keep it in the bag, and say in the intro \"I'll show you something at the end.\"\n\n---\n\n## Running order (45 min)\n\n| Time | Section | Purpose |\n|---|---|---|\n| 0\u20133 | Intro | Set expectations, ask permission to take notes |\n| 3\u201310 | Their operation | Baseline: size, process, tools |\n| 10\u201322 | Last mis-pick story | Root causes; how much it really hurts |\n| 22\u201330 | Floor walk | See it for yourself: bins, scanners, gloves, physical constraints |\n| 30\u201333 | Current fixes and spend | What they've tried, what they pay for today |\n| 33\u201340 | Demo + reaction | Hardware-specific objections |\n| 40\u201344 | Value and buying process | Price anchor, who decides |\n| 44\u201345 | Close | Ask for a next step |\n\nIf time runs short, cut from the floor walk or the demo. **Don't cut the mis-pick story.**\n\n---\n\n## 0\u20133 \u00b7 Intro\n\n> \"Thanks for having me. I'm looking into picking accuracy in warehouses your size. I'm not here to sell you anything today. I'm trying to understand how things actually work on your floor. I have something to show you near the end, but I'd like to hear about your operation first. Okay if I take notes?\"\n\n## 3\u201310 \u00b7 Their operation\n\n1. Walk me through what happens between an order coming in and it leaving the dock.\n2. How many pickers per shift? How many shifts? How much turnover or temp staff, especially at peak?\n3. How do pickers know what to pick: paper, RF scanner, voice, pick-to-light, a mobile app?\n4. **Do pickers scan the bin or item barcode when they pick?** *(Critical. If they already scan and confirm, wrong-bin errors may already be caught, and the band's core value shrinks.)*\n5. Which WMS do you run? Who owns it, you or IT/corporate?\n6. Roughly how many SKUs and bin locations? *(This tells you how many tags they'd need to install and maintain.)*\n\n## 10\u201322 \u00b7 The last mis-pick (the most important section)\n\nAsk about specific past events, not opinions or hypotheticals.\n\n7. **Tell me about the last mis-pick you heard about. What happened?**\n   - How was it found: at pack, at QC, or by the customer?\n   - What caused it? *(Listen for: wrong bin / wrong quantity / wrong item in the right bin because of a slotting or putaway error / mislabeled bin / lookalike SKUs / new picker / system data error.)*\n   - What did it cost to fix? Who spent time on it?\n8. Was that one typical? What other kinds of mis-picks do you see?\n9. **Of all your mis-picks, roughly what share is a picker reaching into the wrong bin, versus everything else?** *(This is the number your decision depends on. Push gently for an estimate.)*\n10. Do you track your mis-pick rate? What is it? Can you show me the report? *(If they don't measure it, that tells you a lot about how much it hurts.)*\n11. What does one mis-pick cost you, all in: re-ship, labor, chargebacks, lost customer?\n12. Where does mis-picking rank among your problems this quarter? What's above it? *(If labor shortage, throughput, or space come first, note it.)*\n\n## 22\u201330 \u00b7 Floor walk\n\nAsk to see a pick in progress. Watch for:\n\n- [ ] Do pickers wear gloves? What kind? *(Affects a wrist device.)*\n- [ ] Is a scanner already in their hands or on their wrist? *(Is there room for one more device?)*\n- [ ] Bin types: shelving, flow rack, pallet, cold storage, high-reach\n- [ ] Bin density and label condition: could you physically put a tag on each bin?\n- [ ] Charging stations and lockers: where would 50 bands live overnight?\n- [ ] Mezzanines, freezers, dust, wash-downs (affects hardware durability)\n- [ ] Union, safety, or hygiene rules about wearables\n\nAsk a picker if possible: *\"What makes you grab the wrong thing?\"*\n\n## 30\u201333 \u00b7 Current fixes and spend\n\n13. What have you tried to reduce mis-picks? What happened?\n14. What are you paying for today: scanners, voice, pick-to-light, WMS add-ons, extra QC staff?\n15. Have you looked at a hardware rollout before (scanners, voice headsets)? How did it go? Who installed it, and how long did it take?\n\n## 33\u201340 \u00b7 Demo + reaction\n\nHand them the band. Say as little as possible, then stop talking.\n\n16. \"What's your first reaction?\" *(Then wait.)*\n17. What would stop this from working here?\n18. Who on your floor would push back on wearing it?\n19. **If the same check ran on the scanner or phone your pickers already carry, with no band and no bin tags, would that be better, worse, or the same?** *(This is the direct hardware-vs-software test. Ask it neutrally.)*\n20. What would installing tags on [their bin count] bins take? Who would do it?\n\n## 40\u201344 \u00b7 Value and buying process\n\n21. If mis-picks dropped by half, what would that be worth to you per month?\n22. *(Only after their answer:)* \"One price we're considering is about $40 per picker per month. For you that's around $[pickers \u00d7 40]/month. How does that land?\"\n23. Who signs off on a purchase like this? What budget would it come from? Is there a threshold where it needs corporate/IT approval?\n24. What would you need to see to run a pilot? Paid or free, how long, how many pickers?\n\n## 44\u201345 \u00b7 Close\n\n25. \"Who else should I talk to, inside your company or at another site?\"\n26. **Ask for a concrete next step:** a pilot conversation, access to their mis-pick data, or time with their ops/IT lead. *(If they agree to something that costs them time or data, that's a real signal. \"Sounds interesting, keep me posted\" is not.)*\n\n---\n\n## Interview rules\n\n- **Ask about the past, not the future.** \"When did that last happen?\" beats \"Would you use\u2026?\"\n- **Don't pitch before minute 33.** If they ask what you're building, say: \"I'll show you at the end. I don't want to bias you.\"\n- **Let silences run.** Count to five before you fill a pause.\n- **Compliments are noise.** \"That's really cool\" counts for nothing. Commitments (time, data, money, intros) count.\n- **Dig into numbers.** \"A lot\" \u2192 \"Roughly how many last week?\"\n\n---\n\n## Scorecard: fill in within 15 min of each call\n\n| | M1 | M2 | M3 | M4 | M5 | M6 |\n|---|---|---|---|---|---|---|\n| Pickers (peak) | | | | | | |\n| Pickers already scan bins? (Y/N) | | | | | | |\n| Tracks mis-pick rate? (Y/N + rate) | | | | | | |\n| % of mis-picks that are wrong-bin grabs | | | | | | |\n| Cost per mis-pick ($) | | | | | | |\n| Mis-picks in their top-3 problems? | | | | | | |\n| Physical blockers (gloves, freezer, union, tag install) | | | | | | |\n| Prefers band / software on existing device / neither | | | | | | |\n| Reaction to $40/picker/mo | | | | | | |\n| Concrete next step offered? | | | | | | |\n\n### Decision rules (set these **before** Thursday and agree on them with your co-founder)\n\n**Keep building hardware if most managers (4+ of 6):**\n- say wrong-bin grabs are a major share of mis-picks, **and**\n- don't already scan-confirm each pick, **and**\n- have no physical blockers to a band + bin tags, **and**\n- prefer the band over software on their existing device, **and**\n- offer a concrete next step (pilot, data, intro).\n\n**Pivot toward software if:**\n- most already scan-confirm (so the remaining errors are quantity, slotting, or data problems), **or**\n- tagging bins or adding a wearable is a recurring blocker, **or**\n- they'd take the same check on their existing scanner/phone just as happily.\n\n**Neither: rethink the problem if:**\n- mis-picks aren't in their top 3 problems, or they can't put a cost on them. In that case the issue is the hypothesis, not hardware vs. software.\n\nSix interviews is a small sample. Treat a 3\u20133 split as \"not yet decided\" and plan follow-ups. Don't break the tie toward whichever option you were already leaning to.\n"}]}, "opus-3": {"cost_usd": 0.17382, "seconds": 66, "output_tokens": 5584, "files": [{"name": "interview-guide.md", "text": "# Interview Guide: Warehouse Managers (Thursday, 6 calls)\n\n**What Thursday is for:** decide whether to keep building hardware or pivot to software only.\n**What has to be true for hardware to win:** (1) mis-picks cost these sites real money, (2) the tools they already have don't catch them, (3) the fix has to happen *at the moment of the grab* (a wrist buzz can do that, software can't), and (4) the site will tolerate a wearable plus tagging every bin at roughly $40/picker/month.\n\nEach section below tests one of these. If (1) or (2) fails, the hardware question doesn't matter. If (3) or (4) fails, that points toward software.\n\n---\n\n## On the co-founder's \"demo in the first 5 minutes\" idea\n\n**Recommendation: don't. Do the demo in the last 10 minutes.**\n\n- If they see the band first, the rest of the call is about the band. You'll hear reactions to the device (\"cool,\" \"my guys would lose it\") and not whether mis-picks are a problem they'd pay to fix. You only get to hear about the problem unprompted once.\n- People are polite to founders holding prototypes. \"They'll get it instantly\" is exactly the danger: understanding the product isn't the same as needing it, and a demo-first call makes those two hard to tell apart.\n- A late demo is a better demo. By then you know their pick method, error rate, and what they've tried, so you can say \"remember the case you mentioned...\" and watch whether it lands.\n\n**Compromise if your co-founder pushes back:** run 1\u20132 calls demo-first and the rest demo-last, then compare how much problem detail you got. Just don't make it all six.\n\n---\n\n## Before each call (5 min)\n\n- Look up the company: what it sells (each-pick e-commerce vs. case/pallet, cold storage, 3PL vs. own inventory).\n- Bring: the prototype (bag it until the demo), one or two bin tags, a notepad with the scorecard (end of this doc).\n- Ask in advance if you can walk the floor during a pick wave. Watching picking beats any answer about picking.\n- Get a co-worker to take notes or ask permission to record. Write down exact words, not summaries.\n\n## Rules for the conversation\n\n- Ask about **past behavior**, not hypotheticals. \"When did that last happen?\" beats \"Would you...?\"\n- Don't pitch, describe the solution, or say \"wearable\" before the demo section.\n- When they give a generic answer (\"errors are a big deal\"), ask for a specific instance: \"Tell me about the last one.\"\n- Get numbers wherever possible: rates, dollars, headcount, hours.\n- Silence is fine. Let them fill it.\n- Compliments (\"that's neat\") and future promises (\"we'd definitely look at that\") are worth nothing. Commitments (time, intros, a pilot, money) are evidence.\n\n---\n\n## Call Structure (45 min)\n\n| Time | Section | Goal |\n|---|---|---|\n| 0\u20133 | Opening | Set the frame |\n| 3\u201313 | Their operation | Context and baseline numbers |\n| 13\u201323 | Mis-picks specifically | Is the problem real and costly? |\n| 23\u201330 | What they've tried / use now | Is it already solved? |\n| 30\u201335 | Buying and rollout | Who decides, what blocks hardware |\n| 35\u201343 | Demo + reaction | Hardware vs. software signal |\n| 43\u201345 | Close | Commitment and next step |\n\nIf you get a floor walk, use it during sections 2\u20134 and ask the questions while you watch.\n\n---\n\n### 1. Opening (0\u20133 min)\n\n> \"Thanks for making time. I'm working on picking accuracy in mid-size warehouses, and I'm still at the stage of learning how operations really run. I'm not here to sell anything today. Most of the time I'll just ask how things work for you. Near the end I'll show you something early we've built and I'd like your honest reaction. Blunt is more useful to me than polite.\"\n\n### 2. Their operation (3\u201313 min)\n\n1. Walk me through how an order goes from coming in to leaving the dock.\n2. How many pickers do you have on a typical shift? Peak season? How many are temps or seasonal?\n3. How do pickers know what to pick and where? *(paper list / RF scanner / mobile app / voice / pick-to-light / goods-to-person)*\n4. Is each pick confirmed? How: scanning the bin, the item, both, or nothing?\n5. What WMS do you run? Who owns it, you or corporate/IT?\n6. How many pick locations/bins do you have? How often do bins get re-slotted?\n7. What do pickers wear and carry? *(gloves, scanner on wrist/finger, phone, vests)*\n8. What metrics are you measured on? Which one gets you the most heat from your boss?\n\n> **Listen for:** If they scan every item at pick, most mis-picks are already caught, and the band's value shrinks to *speed* (catching before the scan) rather than *accuracy*. That's a big difference. Bin count tells you how expensive tag install is.\n\n### 3. Mis-picks (13\u201323 min)\n\n9. *Don't lead.* Ask: \"What are the biggest problems on the pick floor right now?\" **Note whether mis-picks come up on their own, and in what rank.**\n10. *(If not raised)* \"How about picking errors: wrong item, wrong quantity, wrong bin?\"\n11. What's your pick accuracy or error rate? How do you know? *(measured vs. guessed)*\n12. Tell me about the last mis-pick that caused real trouble. What happened, who found it, what did it cost?\n13. Where are errors caught: at pick, at pack/QC, or by the customer?\n14. What does one error cost you? *(re-pick labor, reship, chargebacks, lost customer)* Have you ever totaled it up for a month?\n15. Which errors are most common: wrong bin, wrong quantity, wrong SKU in the right bin (mis-slotting), damaged?\n16. Is it spread evenly or concentrated: new hires, temps, specific zones, look-alike SKUs, peak season?\n17. How much time do pickers lose *before* an error is caught, e.g. walking back, searching, waiting on a supervisor?\n\n> **Listen for:** *Wrong bin* is the only error type the band addresses. Wrong quantity and mis-slotting (wrong item placed in the right bin) it won't catch. If most errors are those types, that's a strong signal against this hardware design specifically.\n> **Quick math to do afterward:** errors/month \u00d7 cost/error vs. pickers \u00d7 $40. With 50 pickers that's $2,000/month. Is their error cost clearly above that?\n\n### 4. What they've tried / use now (23\u201330 min)\n\n18. What have you done to reduce picking errors? How did it go?\n19. Have you looked at or tried pick-to-light, voice picking, ring scanners, vision systems? Why did/didn't you go with it?\n20. Have you ever tried a wearable on the floor? *(ring scanners, smartwatches, vests)* How did pickers take to it? What happened with charging, breakage, loss?\n21. If you could wave a wand and fix one thing about picking, what would it be?\n22. Is anyone currently spending money (or budget) on this problem? How much?\n\n> **Listen for:** If they've never tried anything, the problem may not hurt enough. If they abandoned a wearable because of charging/loss/pickers refusing it, that's a direct hardware risk. If they say \"my real problem is X\" and X is software (slotting, labor planning, WMS data), write it down. That's your pivot candidate.\n\n### 5. Buying and rollout (30\u201335 min)\n\n23. Last time you bought new equipment or software for the floor, how did that work? Who signed off, how long did it take?\n24. Do you have a budget you control, or does this go to corporate / IT / finance?\n25. What would make IT or corporate say no to something new on the floor? *(wireless, security, integration)*\n26. What does it take to roll out something physical, like new scanners? Who handles charging, storage, replacements?\n27. Are pickers unionized, or is there any sensitivity about devices that track them?\n28. How do you usually pay for floor tech: buy outright, lease, per user per month?\n\n> **Listen for:** Hardware-specific blockers: tagging thousands of bins, daily charging, lost devices, union/surveillance concerns, long capex approvals. Each one counts toward software-only.\n\n### 6. Demo + reaction (35\u201343 min)\n\nOnly now take out the band and a bin tag.\n\n> \"Here's what we've built. Pickers wear this band, the bins get these tags, and if a picker reaches into the wrong bin, the band vibrates before they pull the item.\"\n\nHand it to them and let them put it on. **Stay quiet for a bit** and watch what they do and ask first.\n\n29. Where on your floor would this help, and where wouldn't it?\n30. Thinking about [the incident from Q12], would this have caught it?\n31. What would stop you from rolling this out?\n32. What would your pickers say? Your IT people?\n33. *(Tag install)* You've got ~[N] bins. How would putting a tag on each one go?\n34. **The hardware vs. software question:** \"Suppose the same check ran with no new device: on the scanners/phones you already use, or as reports from your WMS data showing where and why errors happen. Would that be more or less useful than the band? Why?\"\n35. *(Price, last)* \"We're thinking about $40 per picker per month. How does that land?\" Then **stop talking**. If they balk, ask what they'd compare it to.\n\n> **Listen for:** Do they talk about *their* floor (\"in zone C this would...\") or the gadget (\"cool, how's the battery\")? The first means fit. The second means novelty.\n\n### 7. Close (43\u201345 min)\n\n36. Who else should I talk to: another manager, your IT lead, a picker supervisor?\n37. **Commitment ask (pick one that fits):**\n    - \"Could I come back and watch a full pick wave / see your error log?\"\n    - \"If we ran a 2-week pilot in one zone, would you be up for it?\"\n    - \"Can you share last month's mis-pick numbers so I can check whether this pays off for you?\"\n38. \"Anything I should have asked but didn't?\"\n\nA yes to a pilot or to sharing data is your strongest signal. \"Keep me posted\" is a polite no.\n\n---\n\n## Scorecard (fill in right after each call, while it's fresh)\n\n| | Mgr 1 | Mgr 2 | Mgr 3 | Mgr 4 | Mgr 5 | Mgr 6 |\n|---|---|---|---|---|---|---|\n| Pickers (normal / peak) | | | | | | |\n| Pick method (paper/RF/voice/light) | | | | | | |\n| Scans every pick? (Y/N) | | | | | | |\n| # bins | | | | | | |\n| Mis-picks raised **unprompted**? (Y/N, rank) | | | | | | |\n| Error rate (measured / guessed) | | | | | | |\n| Monthly cost of errors (their estimate) | | | | | | |\n| % errors that are wrong-bin | | | | | | |\n| Already spending on fix? ($) | | | | | | |\n| Hardware blockers named | | | | | | |\n| Preferred band or software in Q34? | | | | | | |\n| Reaction to $40/picker | | | | | | |\n| Commitment given (pilot / data / intro / none) | | | | | | |\n| Software problem they raised (pivot candidate) | | | | | | |\n\n## Making the decision after six calls\n\n**Keep building hardware if at least 3 of 6:**\n- raised mis-picks unprompted or in their top 3 problems, **and**\n- don't already scan-confirm every pick (or say scanning is too slow), **and**\n- say most errors are wrong-bin, **and**\n- have error costs clearly above pickers \u00d7 $40/month, **and**\n- didn't name a deal-breaking hardware blocker (bin tagging, charging, union, IT), **and**\n- agreed to a pilot or to share data.\n\n**Pivot to software if most of these show up:**\n- they already scan every pick, so errors are caught and the remaining pain is elsewhere\n- most errors are quantity or mis-slotting, which the band can't detect\n- they prefer \"no new device\" in Q34\n- bin tagging or charging/device management came up as a blocker\n- a software problem kept coming up across managers (slotting, labor planning, error analytics)\n\n**Neither (problem may not be real):** mis-picks rarely came up unprompted, nobody could estimate error cost, nobody has spent money on it. Then the question isn't hardware vs. software but whether to pick a different problem.\n\nSix interviews is a small sample, all from one meetup. If results come out 3\u20133, say so, don't force a call. Treat it as \"need 4\u20136 more interviews, aimed at the type of site that split.\"\n"}]}, "haiku-1": {"cost_usd": 0.0061, "seconds": 33, "output_tokens": 6271, "files": [{"name": "interview_guide.md", "text": "# Warehouse Manager Interview Guide\n\n**Calls:** 6 managers, Thursday, 45 minutes each, on site.\n**Goal:** Decide whether to keep building hardware or pivot to software only. Learn how mis-picks happen, what they cost, what has already been tried, and whether a wearable is the right fix.\n\nThis is a research call, not a sales call. We are not asking \"would you buy this?\" We are asking about what happened last time.\n\n## Before the calls\n\n- **Demo at the end, not the start.** The co-founder suggested a demo in the first five minutes. If they see the band first, they will react to the device and stop describing the problem. Most people are polite about a gadget they have just watched. We learn more from their own account of mis-picks first.\n- Bring: the prototype (for the last few minutes only), a notebook, a timer.\n- Ask permission to record or take notes before the first question.\n\n## Agenda (45 min)\n\n### 1. Opening (3 min)\n- Thank them. Say we are learning how picking works at their site, that there are no right answers, and that they can skip any question.\n- Confirm basics: number of pickers per shift, shifts per day, rough SKU count, and whether orders are mostly single-line or multi-line.\n\n### 2. Current picking process (7 min)\n- Walk me through how a picker goes from an order to a finished pick. What do they use: RF scanner, pick-to-light, voice, paper?\n- Does the picker scan or confirm the bin location before picking? What happens if they skip it?\n- Roughly how many picks per shift?\n\n### 3. Mis-picks (12 min) \u2014 the core of the call\n- Tell me about the last wrong-item pick you know of. Walk me through it from the picker to the customer.\n- How often does this happen per week? How do you know? Is it measured?\n- Where is it caught: at pick, at pack, at ship, or by the customer?\n- What does one mis-pick cost? Prompt for each: re-pick time, return shipping, refund, labor, lost customer. \"A rough number is fine.\"\n- Which mis-picks cause the most trouble? Similar SKUs, bin layout, new hires, peak season?\n- Who owns this problem? Who would have to approve a fix?\n\n### 4. What they have tried (5 min)\n- What have you already done to reduce mis-picks? What happened?\n- Do you have a system that already checks the item or bin? Does it work? Why did you keep it or drop it?\n- What would a fix have to take away or change on the floor?\n\n### 5. Floor observation (5 min, if allowed)\n- Ask to watch one picker for about 5 minutes. Do not interrupt.\n- Note: where they pause, how they confirm location, what they look at, how often they check the screen or label.\n- Afterwards, ask: \"Where in that would an alert have helped, and where would it have been in the way?\"\n\n### 6. Reaction to the concept, then the demo (7 min)\n- Describe the idea neutrally first: \"We are looking at ways to alert a picker when they reach for the wrong bin. One option is a band that vibrates.\"\n- Then ask:\n  - What would you expect this to change?\n  - What is your first worry?\n  - Would pickers wear it all shift? What about gloves, hygiene, sharing between shifts, charging?\n  - Would your pickers see it as help or as monitoring? What would HR or a union need to know? What would the band log?\n  - What would make you say no at the first meeting?\n- Demo the band last, briefly. Show it; do not pitch it.\n\n### 7. Price and pilot (5 min)\n- Before naming any price: \"If a fix cut mis-picks in half, what would that be worth to you per month?\" Then: \"What would be too expensive? What would be an obvious yes?\"\n- Then, if useful: \"We are thinking around USD 40 per picker per month. How does that compare?\"\n- Would you run a pilot on one shift? What would it need to prove, and by when?\n- Who else needs to be involved? Would you introduce us to another manager?\n\n### 8. Close (1 min)\n- \"Is there anything we should have asked and did not?\"\n- Thank them. Confirm any next step they offered.\n\n## Interviewer notes\n\n- Ask about the last time something happened, not what they would do. \"Would you use it?\" gets polite yeses.\n- If they criticize the idea, do not defend it. Ask: \"What would have to be true for you to use it?\"\n- Write down numbers with units and quotes word for word.\n- If they have no number, write \"no number, estimate: X\" so we can tell the difference later.\n\n## Right after each call (within 15 minutes)\n\nFill in one line per call:\n- Mis-picks per week (number, or \"unknown\")\n- Cost per mis-pick (number and what it includes)\n- Where mis-picks are caught\n- Current tools and whether they solve the problem\n- Who approves a fix\n- First worry about the wearable\n- Price anchor they gave\n- Pilot: yes, no, or maybe, with shift and timing if given\n\n## Thursday decision criteria (proposed; adjust before the calls)\n\nThese are starting thresholds, not rules. Agree on them with the co-founder before the calls so the outcome is not decided after the fact.\n\n- **Problem is real:** at least 4 of 6 name mis-picks as a top-three cost and give a number.\n- **Software may already solve it:** if most managers already use scan verification or pick-to-light that works, the band needs a clear edge over it. If it does not have one, that points to a software-only pivot.\n- **Wearable objections:** if hygiene, glove, shift-sharing, or monitoring objections come up in most calls with no workaround, that points to a software-only pivot.\n- **Pilot interest:** at least 2 of 6 name a shift and a timeline for a pilot. Fewer than that suggests the hardware is not wanted yet.\n- **Price:** at least 3 of 6 give an anchor that is near or above USD 40 per picker per month, or describe a value that clearly covers it.\n\nIf the problem is real but the wearable objections dominate, the likely answer is \"keep the problem, change the product.\" If the problem is weak, the answer is to stop, regardless of hardware.\n"}]}, "haiku-2": {"cost_usd": 0.00488, "seconds": 27, "output_tokens": 4331, "files": [{"name": "interview_guide.md", "text": "# Warehouse Manager Interview Guide\n\n**Calls:** 6 warehouse managers, Thursday, 45 minutes each, on site.\n**Goal for Thursday:** Decide whether to keep building hardware or pivot to software only.\n\n**What we need to learn**\n1. How much do mis-picks cost these sites, and how do managers know about them?\n2. How do pickers currently verify they are in the right bin, and where does that break down?\n3. Would a wearable alert change anything, compared with what they already use?\n4. What would stop a site from adopting a wearable (labor, safety, hygiene, charging, IT)?\n5. Who pays for error reduction, and how do they budget for it?\n\n**Ground rules for every call**\n- Ask about their operation before showing anything. Don't name a solution in the first 15 minutes.\n- Ask about specific recent events (\"Tell me about the last mis-pick you found\"), not opinions (\"Would you use this?\").\n- Don't pitch the price in the interview. Ask how they budget for per-head tools and what they pay now.\n- Ask what would make this a bad idea. Write down every objection, verbatim where you can.\n- Don't argue with objections. Ask a follow-up and move on.\n\n---\n\n## Timing (45 min)\n\n| Min | Section | Notes |\n|---|---|---|\n| 0-3 | Intro | Explain you're researching, not selling. Ask permission to take notes. |\n| 3-13 | Their operation | Scale, layout, pick method. |\n| 13-25 | Mis-picks | Frequency, cost, detection, consequences. Core section. |\n| 25-33 | Current tools | What they tried, what stuck, what they dropped. |\n| 33-40 | Prototype | Show the band and tags after the problem is covered. Watch reactions. |\n| 40-45 | Adoption and money | Objections, budget, next steps. Ask for a floor walk if time allows. |\n\n**Demo timing:** The co-founder suggested a demo in the first 5 minutes. I'd hold it until minute 33. If the demo comes first, managers will react to the band and describe the problem it solves, not the problem they have. You can't tell whether they'd use it or just like it.\n\n---\n\n## 1. Opening (0-3 min)\n\n- \"Thanks for the time. We're trying to understand how picking errors affect sites like yours. We're not selling anything today.\"\n- \"Is it OK if I take notes? I won't share names outside the team.\"\n\n## 2. Their operation (3-13 min)\n\n- How many people pick on a typical shift? How many shifts?\n- What do you ship, and roughly how many order lines a day?\n- How are picks released to pickers? (Batch, zone, wave, single order?)\n- What does a picker's day look like from start to finish?\n- What does the layout look like? How many bins or locations per picker?\n- How often do bins get relabeled or moved?\n- *Observe on the floor if you can:* aisle width, bin label size and placement, glove types, noise level, how pickers scan.\n\n## 3. Mis-picks (13-25 min) \u2014 core section\n\n- What counts as a mis-pick at your site? How do you define it?\n- Walk me through the last mis-pick you remember. How was it found? Who noticed? What happened next?\n- In a typical week, how many mis-picks do you catch? How many do you think you miss?\n- How do you find out about mis-picks? (Customer complaints, returns, cycle counts, audits?)\n- What does a mis-pick cost you? Ask for specifics: re-ship, labor, returns, chargebacks, lost customers.\n- Which mis-picks are the most expensive? Which are the most common?\n- Do mis-picks cluster by picker, shift, zone, product, or time of day?\n- Who is responsible when a mis-pick happens? How does that play out in practice?\n- Is the cost of mis-picks something you track formally? Where does that number live?\n\n## 4. Current tools (25-33 min)\n\n- What do pickers use right now to confirm location and item? (Handheld scanner, voice, pick-to-light, paper, label check?)\n- Where does that process fail? Ask for a recent example.\n- What have you tried to reduce mis-picks? What did you keep, and what did you drop? Why?\n- If you had a budget for this tomorrow, what would you spend it on?\n- Have you looked at a warehouse management system (WMS) or scanning upgrade? What stopped you from buying?\n- What did the vendor pitch miss about your site?\n\n## 5. Prototype (33-40 min)\n\nShow the band and bin tags. Keep it short: what it does, nothing more.\n\n- \"What do you think this does?\" (Let them describe it before you explain.)\n- Where would a picker wear it? What would happen during a shift?\n- What would happen when a picker wears gloves, moves fast, or changes shifts?\n- What would you need to see before you tried it on one team?\n- How would you measure whether it worked? What number would you look at?\n- If pickers were told to wear this, what would they say? How would you handle it?\n- Would a vibration alert be noticed in your noise levels? (Ask to test if possible.)\n- Ask for a floor walk: \"Can we walk the picking area for five minutes?\" Note where the band would and wouldn't work.\n\n## 6. Adoption and money (40-45 min)\n\n- Who would approve buying something like this? Who would have to sign off after that? (Ops, safety, IT, HR, union?)\n- What concerns would safety or HR raise about a wearable?\n- Pickers: who would have to charge, clean, and issue the device? How many devices per shift?\n- Your current spend on error reduction or picking tools: roughly what per picker per month?\n- If a tool cut mis-picks noticeably, what would the purchase look like? Pilot, per-head subscription, capital purchase?\n- What would make you say no, even if the results were good?\n- Who else should we talk to? Who in your network has tried something like this?\n- Can we follow up in two weeks with a short pilot proposal?\n\n---\n\n## Closing (last 1 min)\n\n- \"What's one thing we didn't ask about that we should have?\"\n- Thank them. Confirm the follow-up and who will send it.\n\n---\n\n## After each call (within 30 min)\n\nFill this in while it's fresh. The six entries should make the hardware-vs-software call easier.\n\n- **Site:** size (pickers, shifts), pick method, current error tool\n- **Mis-pick frequency and cost (their numbers, not ours):**\n- **Biggest recent mis-pick, and how it was found:**\n- **Would a bin-verification wearable change this?** Yes / No / Maybe, and why:\n- **Would software alone (WMS, scanner, pick-to-light) solve it?** Yes / No / Partly, and why:\n- **Strongest objection:**\n- **Adoption blockers (labor, safety, hygiene, IT):**\n- **Spend they described (per picker per month, if given):**\n- **Would they pilot? What would they need to see?**\n- **Quotes worth keeping, verbatim:**\n\n## Pattern check after all six\n\n- How many sites said mis-picks were a top-three cost? (Need most of them.)\n- How many already have a tool that addresses bin verification? If most do, the wearable needs to beat it, not just exist.\n- How many named a specific blocker for wearables? Any blocker raised by three or more sites is a serious risk.\n- Did any site name a price they'd pay? Compare against the USD 40 per picker per month idea in the plan.\n- Did anyone ask for something software-only that would remove the need for hardware? If several did, that's evidence for the pivot.\n"}]}, "haiku-3": {"cost_usd": 0.005, "seconds": 28, "output_tokens": 4520, "files": [{"name": "interview_guide.md", "text": "# Warehouse Manager Interview Guide\n\n**Purpose:** Decide whether to keep building hardware (wrist band + bin tags) or pivot to software only.\n**Format:** 6 calls, 45 min each, on site. Target: learn how mis-picks happen and what they cost, not sell.\n\n## Ground rules\n\n- **Talk less than they do.** Aim for under 30% of airtime.\n- **Ask for the last time it happened.** \"Tell me about the last mis-pick you dealt with\" beats \"How often do mis-picks happen?\"\n- **Don't pitch before the problem is confirmed.** Demo the band in the last section (around minute 33), not the first five minutes. If they see the device first, they react to the device and we learn nothing about whether the problem is real.\n- **Don't ask hypotheticals about price or interest.** \"Would you pay $40?\" gets polite yeses. Ask what they spend today on related problems instead.\n- **Write down numbers exactly as stated** (picks per hour, errors per week, cost per return). Mark estimates as estimates.\n- **Note who decides and who pays** for each manager.\n\n## Before the call (2 min)\n\n- Warehouse size: number of pickers, shifts, SKUs, daily order lines\n- Picking method: RF scanner, paper, voice, pick-to-light, or other\n- WMS in use, if known\n- Any prior notes from the logistics meetup\n\n## 1. Opening (3 min)\n\n- Thank them. Explain you're researching how warehouses handle picking accuracy. You're not selling anything today.\n- Ask permission to take notes and to walk the floor at the end if possible.\n\n## 2. Context (5 min)\n\n- Walk me through your operation: how many pickers, how many shifts, what you ship.\n- What does your team spend most of its time firefighting?\n- Where does picking accuracy rank among your KPIs, and who do you report it to?\n\n## 3. How picking works day to day (10 min)\n\n- Walk me through a single pick, from the moment a picker gets a task to the moment it's confirmed.\n- Where can the wrong item or wrong bin get picked? Where in that flow do the most errors happen?\n- How do pickers know they're at the right location? What happens when a bin label is hard to read or a location is moved?\n- What do new pickers struggle with most? How long until they're reliable?\n\n## 4. Mis-picks (10 min)\n\n- Think of the last mis-pick that cost you real money or time. What happened, step by step?\n- Roughly how many mis-picks per week? How do you know, and how sure are you of that number?\n- Are most errors the wrong bin, the wrong SKU in the right bin, or the wrong quantity? Which is the biggest?\n- Where do you usually catch them: at pick, at pack, at the customer? What does each one cost?\n- What happens to the picker involved? Does it affect them, their team, or their pay?\n- What have you already tried to reduce them? What happened to those efforts?\n\n## 5. Current spend and ownership (7 min)\n\n- What do you spend today on returns, re-picks, chargebacks, or overtime tied to picking errors? Ballpark is fine.\n- Which budget would fund a fix like this? Who signs off above you?\n- Have you bought any tool for this before (WMS modules, scanners, pick-to-light, voice picking)? How did it go?\n- What would make you say no to a new tool, even if it worked?\n\n## 6. Concept reaction (7 min)\n\nDemo the prototype at this point. Keep it to 2 minutes and then stop talking.\n\n- In your own words, what does this do? (Check that they understood it correctly.)\n- Would this have prevented the mis-pick you described earlier? Why or why not?\n- What would stop a picker from wearing it? (Gloves, charging, hygiene, safety rules, shift changes, lost devices, vibration distraction, union or labor concerns.)\n- Could a software-only change solve the same problem? For example, a scan-to-verify step in the WMS, a pick-to-light screen, or a voice prompt. What would that look like, and what would it not fix?\n- If we tested this, what would you need to see to call it a success? How many pickers and how many weeks?\n\n## 7. Next steps (3 min)\n\n- Is there someone else on your team (ops lead, safety, IT) we should talk to before any test?\n- Can we set up a small, time-boxed trial? Who would approve it, and what space or pickers could you offer?\n- Thank them and confirm how we'll follow up.\n\n## Floor walk (if time allows, after section 6)\n\n- Watch one pick from start to finish. Note where pickers hesitate, check labels, or re-scan.\n- Ask the picker (with the manager's OK) what slows them down most.\n- Note bin labels, lighting, glove use, and how far apart bins are.\n\n## After each call (10 min, same day)\n\nFill this in while it's fresh:\n\n- **Company / role / pickers / shifts:**\n- **Mis-pick rate (their words and number):**\n- **Error type:** wrong bin / wrong SKU / wrong quantity (rank them)\n- **Where caught and cost per error:**\n- **Current spend and budget owner:**\n- **Hardware objections raised:**\n- **Could software alone fix it? (yes / partly / no, and why):**\n- **Would they run a trial? (yes / maybe / no, and what they'd need):**\n- **Quotes worth keeping (verbatim):**\n\n## Decision check (after all 6 calls)\n\nKeep building hardware only if most managers:\n1. Name mis-picks from wrong bin location as a top cost, and\n2. Say a software-only fix (scan verification, pick-to-light, voice) would not solve it, and\n3. Raise no major objections to wearing the band, or say they would accept one in a trial, and\n4. Point to a budget and a person who could approve a trial.\n\nIf most of them say a software change would work, or can't name a budget, the software-only path is the better bet.\n"}]}}, "verdicts": [{"vs": "opus", "pair": "2", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "tie", "why": "A better grounds purchasing in past spending; B better probes error causes and floor constraints. Both narrow discovery prematurely and overinterpret weak evidence in decision rules. B equates headcount with paid seats and omits the $800\u2013$3,200 range."}, {"vs": "opus", "pair": "2", "judge": "gpt", "order": "opus shown first", "winner": "haiku+skill", "why": "B better separates evidence from demo reactions and investigates actual purchasing. A equates headcount with paid seats. Both narrow discovery prematurely, under-test detection reliability, and use weak software-pivot rules. Both agendas total 45 minutes."}, {"vs": "opus", "pair": "2", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "haiku+skill", "why": "A fully addresses pricing arithmetic and avoids leading questions; B introduces price early and misstates arithmetic."}, {"vs": "opus", "pair": "2", "judge": "deepseek", "order": "opus shown first", "winner": "opus", "why": "A includes the required price arithmetic and a more explicit, practical purchasing investigation."}, {"vs": "haiku", "pair": "1", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "tie", "why": "Both total 45 minutes but miss open problem prioritization and hardware detection feasibility. A separates evidence better but excludes commitments; B explores pilots but uses hypothetical savings and omits seat-cost arithmetic. Both offer weak software-pivot rules."}, {"vs": "haiku", "pair": "1", "judge": "gpt", "order": "haiku shown first", "winner": "haiku+skill", "why": "B better investigates actual spending, requests logs, and distinguishes evidence types and sample limitations. A relies on hypothetical value and price anchors. Both lack broad problem discovery, rigorous bin-tag feasibility checks, and a sound insufficient-evidence decision path."}, {"vs": "haiku", "pair": "1", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "haiku+skill", "why": "A fully addresses all criteria, especially protecting discovery, accurate pricing, and using observation. B's agenda is less structured, and it prematurely introduces price and concept."}, {"vs": "haiku", "pair": "1", "judge": "deepseek", "order": "haiku shown first", "winner": "haiku", "why": "A includes explicit price arithmetic and a pilot commitment probe, fully meeting c4; B omits these."}, {"vs": "opus", "pair": "1", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "opus", "why": "B better examines floor workflow, hardware constraints, software alternatives, and demo limitations. Both underdevelop open-ended problem discovery and uncertainty in synthesis. B weakens purchasing validation with hypothetical savings and omits the $800\u2013$3,200 monthly seat-cost illustration."}, {"vs": "opus", "pair": "1", "judge": "gpt", "order": "opus shown first", "winner": "opus", "why": "A provides stronger floor observation, hardware-fit probes, and concrete pilot steps. Its pricing omits seat-cost arithmetic and introduces hypothetical savings. B crowds observation, demo, and closing into five minutes. Both insufficiently explore other problems and overinterpret software-pivot signals."}, {"vs": "opus", "pair": "1", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "tie", "why": "Both are excellent, complete guides meeting all rubric criteria. Differences are stylistic, not substantive."}, {"vs": "opus", "pair": "1", "judge": "deepseek", "order": "opus shown first", "winner": "opus", "why": "A provides a complete, timed, ready-to-use guide with specific questions, observation prompts, and a clear decision framework. B is less structured and misses key details like pricing arithmetic and floor observation specifics."}, {"vs": "opus", "pair": "3", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "opus", "why": "B better investigates competing problems, error causes, and software alternatives, with a clearer insufficient-evidence outcome. Both under-test detection reliability and purchasing economics. B loses credit for allowing early demos and asserting unproven hardware capabilities; A\u2019s pivot rules lack software evidence."}, {"vs": "opus", "pair": "3", "judge": "gpt", "order": "opus shown first", "winner": "opus", "why": "A better investigates recurring errors, costs, error types, and software alternatives. Both under-test detection reliability and purchasing economics. A weakens discovery with its demo-first compromise; B\u2019s decision thresholds could justify hardware without feasibility or a software pivot without software evidence."}, {"vs": "opus", "pair": "3", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "tie", "why": "Both are excellent, complete guides meeting all rubric criteria. Differences are stylistic, not substantive."}, {"vs": "opus", "pair": "3", "judge": "deepseek", "order": "opus shown first", "winner": "opus", "why": "A more thoroughly addresses hardware feasibility, cost arithmetic, and synthesizes evidence for the pivot decision."}, {"vs": "haiku", "pair": "3", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "haiku+skill", "why": "A allocates floor observation within 45 minutes and better separates evidence from reactions. B better probes error types and software alternatives, but leaves observation optional. Both inadequately test installation feasibility and justify software pivots from weak evidence."}, {"vs": "haiku", "pair": "3", "judge": "gpt", "order": "haiku shown first", "winner": "haiku+skill", "why": "B reserves floor time within 45 minutes and distinguishes evidence types more carefully. Both inadequately test hardware detection feasibility and push software pivots from weak evidence. A leaves observation outside its full agenda; B lacks open exploration of competing problems."}, {"vs": "haiku", "pair": "3", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "haiku+skill", "why": "A explicitly addresses the co-founder's demo request and delays it, protecting discovery; B's demo timing is less robust against early pitch."}, {"vs": "haiku", "pair": "3", "judge": "deepseek", "order": "haiku shown first", "winner": "tie", "why": "Both are excellent, complete, and fully meet all rubric criteria."}, {"vs": "haiku", "pair": "2", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "haiku+skill", "why": "A has a complete 45-minute agenda, stronger purchasing evidence, clearer observation prompts, and separate post-demo notes. Both narrow discovery prematurely. A\u2019s decision thresholds weakly justify software; B crowds observation, lacks firm commitments, and omits sampling limits."}, {"vs": "haiku", "pair": "2", "judge": "gpt", "order": "haiku shown first", "winner": "haiku+skill", "why": "B provides a complete 45-minute agenda, stronger purchasing evidence, concrete commitments, and separated demo reactions. Both narrow discovery prematurely, under-test hardware reliability, and lack a sufficiently grounded hardware-versus-software decision rule."}, {"vs": "haiku", "pair": "2", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "haiku+skill", "why": "A's guide is more disciplined, explicitly addresses the co-founder's demo request, and more rigorously separates problem discovery from prototype."}, {"vs": "haiku", "pair": "2", "judge": "deepseek", "order": "haiku shown first", "winner": "tie", "why": "Both are excellent, complete, and meet all rubric criteria. They differ in structure but are equally effective."}]}, {"id": "int-3", "family": "interviews", "prompt": "Draft what I should ask in these discovery calls. Notes in calls.md.", "input": "calls.md", "skill": "fitzpatrick-mom-test", "input_text": "Product idea: a tool for freelance bookkeepers that collects missing receipts from their small-business clients automatically (texts the client, matches the photo to the bank line).\nI'm a former bookkeeper. Calls: 10 freelance bookkeepers, 25 minutes, booked through a bookkeepers' Slack community. Two of them are friends.\nQuestions I care about: how much time chasing receipts really costs them, whether they'd pay or expect their clients to pay, what they use today.\nLast round of calls everyone said \"that would be amazing\" and then nobody signed up for the waitlist. I don't want that again.\n", "rubric": [{"id": "c1", "criterion": "Provides a call-ready discovery script suited to the booked calls.", "full_marks_when": "Gives concrete questions and follow-up prompts in a usable sequence that fits 25 minutes, with a brief opening and closing, focused on freelance bookkeepers serving small-business clients.", "zero_when": "Offers only general interviewing advice, lacks actual questions, or proposes a script that cannot reasonably fit 25 minutes."}, {"id": "c2", "criterion": "Establishes the real cost and frequency of chasing missing receipts using recent examples.", "full_marks_when": "Asks the bookkeeper to walk through a recent receipt-chasing episode and quantifies time, frequency, number of affected clients or transactions, and consequences; distinguishes chasing time from other bookkeeping work and bases any estimates on reported evidence.", "zero_when": "Relies on whether receipt chasing is annoying or asks only for hypothetical time savings without establishing actual effort."}, {"id": "c3", "criterion": "Reveals the current workflow, alternatives, and practical requirements for automation.", "full_marks_when": "Asks what tools and communication channels they use today, how they identify missing receipts, chase clients, receive photos, match receipts to bank transactions, and handle failures; probes what works, what breaks, and what would prevent automated texting or matching from fitting their workflow.", "zero_when": "Only asks which software they use or assumes automated texts and photo matching will work without examining the existing process."}, {"id": "c4", "criterion": "Investigates payment and purchasing decisions through evidence rather than hypothetical enthusiasm.", "full_marks_when": "Includes questions about existing spending, previous purchases or attempts to solve the problem, who chooses and pays for tools, and how costs are passed to clients; distinguishes bookkeeper-funded from client-funded adoption without treating a guessed price or stated willingness to pay as proof.", "zero_when": "Only asks 'Would you pay for this?' or 'How much would you pay?' and treats the answers as reliable evidence of demand."}, {"id": "c5", "criterion": "Addresses the previous gap between praise and action while reducing biased responses.", "full_marks_when": "Keeps the product pitch out of the initial problem questions, uses neutral prompts about actual behavior, invites accounts where the problem is minor or already solved, and accommodates the two friends without soliciting reassurance; closes with an optional, concrete next step requiring meaningful effort or commitment and makes clear that praise or a waitlist promise alone does not validate demand.", "zero_when": "Leads with the proposed solution, seeks agreement that it would be amazing, or uses compliments and hypothetical signup intentions as the main success signal."}, {"id": "c6", "criterion": "Uses the supplied facts accurately and keeps conclusions proportional to the sample.", "full_marks_when": "Accurately works from 10 freelance bookkeepers, 25 minutes per call, recruitment through a bookkeepers' Slack community, two friends, the asker's bookkeeping experience, and prior praise followed by no waitlist signups; avoids invented findings and acknowledges that this recruitment and friendship mix can bias responses. If totals are given, 10 calls equal 250 minutes (4 hours 10 minutes), and two friends constitute 20% of the sample.", "zero_when": "Misstates the supplied facts, makes arithmetic errors, invents measured costs or willingness to pay, or treats these 10 community-recruited calls as representative proof of market demand."}], "runs": {"haiku-skill-1": {"cost_usd": 0.00764, "seconds": 36, "output_tokens": 5972, "files": [{"name": "discovery-questions.md", "text": "# Discovery call guide: missing receipts for freelance bookkeepers\n\n10 calls, 25 minutes each. Recruited through a bookkeepers' Slack community; two calls are with friends.\n\n## Goals\n\n1. How much time and money chasing receipts costs them, using a recent real month.\n2. Who currently pays for that time (them, the client, nobody), and how they've bought tools before.\n3. What they use today and what they tried before.\n\nSuccess is not \"they like the idea.\" Success is a specific past episode with numbers, and a commitment that costs them something (or a clear reason they won't).\n\n## Rules for the call\n\n- No pitch for the first 20 minutes. Do not describe the product, and do not answer \"would this help?\" with a demo.\n- Never ask \"would you use / pay / like this?\" Those get polite yeses. Ask about the last time it happened.\n- If they say \"that would be amazing,\" go back to the last time the problem happened: \"Tell me about the most recent one.\"\n- Do not swap in your own bookkeeping stories. Let them talk.\n- Write down the answer before interpreting it. Mark each note as **observed** (they showed you), **reported** (they told you about a specific past event), or **opinion** (a prediction, a compliment, a generalization).\n- Friend calls: use the first one or two as dry runs to test the script. Record the relationship in the notes and ask them to be blunt. Don't count their enthusiasm as evidence.\n\n## Opening (2 min)\n\n\"I'm trying to understand how bookkeepers handle missing receipts from clients. I'm not selling anything today. Can you walk me through the last time a client's receipts were missing at month-end?\"\n\n## 1. The last episode (10 min)\n\n- What month was it, and which client? What first told you something was missing?\n- Walk me through what you did after that, step by step, until you had what you needed.\n- How many requests did it take? Who did you contact, and by what method: email, text, portal, phone?\n- Roughly how long did that take you in total? Do you track time on this? Could you check last month's time entries?\n- What happened when a receipt never arrived? Did you estimate it, leave the line uncategorized, or chase again?\n- What did the delay cost in the end? Late close, unbilled hours, a client complaint, a fee, anything else?\n- Across your clients, how many months last quarter had this problem? Which clients are the repeat cases?\n- Has missing paperwork ever cost you a client or a dispute? Tell me about that one.\n\n## 2. Today's tools and workarounds (5 min)\n\n- Could you show me the last request you sent? A screen share or a template is fine.\n- Where do clients upload receipts? Do they actually use it? About what share of clients do?\n- Who on the client side is responsible for finding receipts: the owner, an office manager, someone else?\n- How do receipts get matched to bank lines today? Which accounting software and bank feed?\n- Have you tried anything else for this? What happened, and why did you stop?\n\n## 3. Money and purchasing (5 min)\n\n- Does your fee cover chasing time, or is it unbilled? Have you ever charged a client for it, and how did they respond?\n- What is the last paid tool you added to your practice? Walk me through how you decided: who was involved, how long it took, what you compared it against.\n- Have you looked for something that automates this? What did you find, what did it cost, and why isn't it in use?\n- If you bought something for this, who would have to approve it: you alone, a partner, the client?\n\n## 4. Fit check (only if time remains, and only after sections 1 to 3)\n\nBefore this section, say: \"I'm considering a tool that texts clients for missing receipts and matches the photo to the bank line. I'm trying to see where it would break in your work.\"\n\n- Looking at the last episode, which step would the tool have changed, and which step would it have left alone?\n- What would you need to keep control of? Client relationship, tone of the chase, sign-off before anything goes out?\n- What would make you stop using it after the first month?\n\nMark any reactions in this section as **influenced by the pitch**.\n\n## 5. Commitment ladder (3 min, only if section 1 showed a real, recurring cost)\n\nAsk for one step at a time. Record each as requested, agreed, completed, or declined, plus the reason if declined.\n\n1. Send me an anonymized copy of your last receipt request email or template. *(Low cost. Time.)*\n2. Pick two or three clients with this problem for a test on one month's close, and make the introduction. I'd need about 30 minutes to set up and a 15-minute check-in afterwards. *(Time and client reputation.)*\n3. Only after 1 and 2 are done: \"If this works, what would a fair price be for your practice?\" Then make an honest offer, not a hypothetical. *(Money.)*\n\nReferral (optional): \"Who else in the community deals with this the way you do? Would you introduce me?\" An introduction is not complete until it is made.\n\n## Closing (1 min)\n\n- \"What should I have asked you today that I didn't?\"\n- \"Is there anyone who handles this differently from you that I should talk to?\"\n\n## Do not ask\n\n- Would you use / buy / pay for this?\n- Is this a problem for you? (Ask how often it happened instead.)\n- Do you think this is a good idea?\n- What features would you want?\n- Would $X a month be fair? (Ask what they last paid for a comparable tool.)\n\n## Notes template (one per call)\n\n- Participant role, practice size, relationship to me, recruiting route:\n- Last episode (client, month, steps, hours, cost, outcome):\n- Current tools and workarounds:\n- Who pays and how they buy tools:\n- Observed / reported / opinion / interpretation / unknown:\n- Commitment status (requested / agreed / completed / declined, with reason):\n- Pitch-influenced reactions (if any):\n"}]}, "haiku-skill-2": {"cost_usd": 0.01064, "seconds": 53, "output_tokens": 9946, "files": [{"name": "discovery-call-guide.md", "text": "# Discovery call guide: missing receipts for freelance bookkeepers\n\n**Mode:** Learning only. No offer exists yet, so these calls establish behavior and costs. The product comes up only in the final section, after the learning questions.\n\n## Three learning objectives\n\n1. **Cost of chasing.** How much time and friction missing receipts cause, measured from the last real episode.\n   - Assumption: chasing receipts is a recurring, expensive part of closing books.\n   - Disconfirming answer: it's rare, quick, or already handled by bank feeds or a client portal.\n   - If disconfirmed: the problem is smaller than the idea assumes. Narrow or stop.\n\n2. **Who pays.** Whether the bookkeeper pays for a fix, whether they can pass the cost to clients, and whether clients have ever paid for admin work.\n   - Assumption: bookkeepers would pay, or clients would accept a fee.\n   - Disconfirming answer: they have never bought a tool for this, they absorb the cost, or clients refuse add-ons.\n   - Decision consequence: determines the buyer (bookkeeper or client) and whether a price is viable at all.\n\n3. **Current tools and fit.** What they use now, what they dropped, and whether clients will take a new step such as photographing a receipt.\n   - Assumption: clients will send a receipt photo if asked.\n   - Disconfirming answer: a past portal or app saw little client use after a month, or existing tools already cover this.\n   - Decision consequence: whether the product must fit inside their existing practice tools or can stand alone.\n\n## Before the calls\n\n- **Two of the ten are friends.** Run them first as practice, tag their notes separately, and don't count their praise as evidence. Ask them directly to be blunt about where they'd push back.\n- **Set kill thresholds now**, before any call. Write down what monthly chasing time, or what client willingness, would mean the idea isn't worth building.\n- **Record how each person was found** (Slack post, DM, friend). Community volunteers may over-represent people who already feel this pain or like helping founders.\n- **No product description until Section E.** Describing it early turns the rest of the call into reactions to a pitch.\n\n## Opening (2 min)\n\n> \"I'm researching how bookkeepers handle missing receipts. I'd rather hear about specific recent cases than opinions, so I'll ask about those. I'm not pitching anything today. If I think there's something worth building, I'll tell you at the end.\"\n\nFact check: \"How many clients do you do monthly books for, and what do you run them in?\"\n\n## Section A: Last episode (5 min)\n\n**A1.** \"Think of the last time a client's missing receipt held up their books. Walk me through it from the start.\"\n- Probes: How did you find out it was missing? Who did you contact, and how: email, text, portal? How many times did you chase? What happened next? When did it finally close?\n- Listen for: the trigger, number of rounds, and whether a deadline was involved.\n\n**A2.** \"How long did that one take from first chase to closed? How much of that was actual hands-on work?\"\n- Record elapsed time and working time separately.\n\n**A3.** \"What did you put off or drop to deal with it?\"\n- Reveals the real cost, not the annoyance.\n\n**A4.** \"What would have happened if you hadn't chased it?\"\n- Shows whether the consequence is painful (late filing, unreconciled balances) or only irritating.\n\n## Section B: Time across a month (4 min)\n\n**B1.** \"Over the last month, roughly how much time went to chasing receipts? How did you work that out?\"\n- If it's a guess, record it as an estimate. If they can't estimate, ask when they last tracked it.\n\n**B2.** \"Which clients cause the most chasing, and what do they have in common?\"\n\n## Section C: Money (5 min)\n\n**C1.** \"What's the last tool or service you paid for that handles client admin? How did you decide to buy it, and who paid?\"\n- Probes: What did it cost? Did you pass that on to clients? Are you still using it? What replaced it?\n\n**C2.** \"Has anyone ever charged clients for admin work like chasing documents? What happened?\"\n- If no: \"Why not? Has it ever come up?\"\n\n**C3.** \"Have you ever paid for something meant to fix receipt chasing? What happened to it after three months?\"\n- If no: \"What did you do instead, and what did that cost you?\"\n\n**Avoid** \"Would you pay $X?\" and \"Would your clients pay for this?\" Both invite guesses. Ask about past purchases instead.\n\n## Section D: Tools and client behavior (5 min)\n\n**D1.** \"Walk me through where a receipt goes, from the client to your books, today. Where does it usually get stuck?\"\n- If they're willing, ask them to show you the screen. Observed behavior beats description.\n\n**D2.** \"Which tools do you use for this? Which have you stopped using, and why?\"\n- Failed alternatives are often more informative than current tools.\n\n**D3.** \"When you've asked clients to send things a new way, such as an app, a portal, or a photo, what happened?\"\n- Probe: how many were still doing it a month later?\n\n**D4.** \"Which parts of client admin have you tried to automate, and where did you stop?\"\n\n## Section E: Summary and next step (3 min)\n\n**E1.** Summarize back: \"So the expensive part is X, and it hits when Y. Did I get that right?\"\n- Correct the summary if they disagree. Don't argue for your version.\n\n**E2.** \"Who else should I talk to? Is there someone who had a bad month of chasing?\"\n\n**E3. Optional, only after the learning questions and only if there's something to show:** \"I'm working on a way to match client receipts to bank lines. Would you test it on one client's real files for two weeks?\"\n- Record as **requested**. Mark as **agreed** only if they name a client and a start date. Mark as **declined** with their reason if they say no.\n- Don't use a waitlist signup as the ask. Last round, praise did not turn into signups, and a waitlist costs the prospect nothing.\n\n## Notes to capture per call\n\n- Role, number of clients, and how they were found (Slack post, DM, friend)\n- Last episode: trigger, rounds of chasing, elapsed days, working hours\n- Monthly chasing time: reported or estimated\n- Tools used now, and tools dropped with reasons\n- Who paid for past tools, and whether they're still in use\n- Whether each claim is **observed**, **reported past**, **opinion**, **commitment**, or **unknown**\n- Pitch timing and any reaction that came after it, marked as contaminated\n- Commitment status: requested, agreed, completed, or declined, with specifics\n\n## Avoid during calls\n\n- Treating \"that sounds great\" or \"I'd use that\" as evidence\n- Describing the product before Section E\n- Counting \"send me the details\" as a commitment\n- Asking how many people have this problem. They'll guess.\n- Answering their story with your own story\n\n## After the ten calls\n\n- How many gave a specific last episode? If most gave opinions only, the calls didn't answer the question.\n- How many commitments are agreed versus merely requested?\n- Did the cost of chasing clear the kill thresholds set before the calls?\n- Friend notes stay tagged and are reported separately from the others.\n"}]}, "haiku-skill-3": {"cost_usd": 0.0088, "seconds": 42, "output_tokens": 7813, "files": [{"name": "discovery-questions.md", "text": "# Discovery call guide: missing receipts for freelance bookkeepers\n\n## Goal for this round\n\nLearn three things, in this order:\n\n1. How much time and money chasing receipts actually costs, from one real recent episode.\n2. Who pays for that work: the bookkeeper absorbs it, bills it, or the client pays.\n3. What they use today and what they have already tried.\n\nNo product pitch in the first 20 minutes. Ask about their last real episode, not whether the idea sounds good.\n\n## Three learning objectives\n\nEach has an answer that would change the plan. Write these down before the first call.\n\n1. **Pain size.** If they chase only a handful of receipts a month, or already have a routine that works, the problem is small and this segment may be wrong.\n2. **Who pays.** If bookkeepers say clients should pay but have never billed for chasing, bookkeepers will not pay for this themselves, and a client-side model is needed.\n3. **Client tolerance.** If they would not let a tool text their clients under their name, the product needs another channel.\n\n## Participants\n\n- Ten bookkeepers from the Slack community.\n- The two friends: run them first as practice for the script. Mark them as friend contacts in the notes. Their answers are still useful, but check whether they match the other eight. Do not count them as evidence of demand.\n\n## Recording\n\nAfter each call, label every claim:\n\n- **Observed:** they showed you an artifact (a message, invoice line, export, timesheet).\n- **Reported past:** a specific episode they described.\n- **Opinion:** a preference, compliment, or prediction.\n- **Commitment:** a time, reputation, or money action they agreed to.\n- **Interpretation:** your inference.\n- **Unknown:** what you still do not know.\n\nAlso note whether any praise came before or after you described the product.\n\n## Opening (about 2 minutes)\n\n\"I'm looking at how bookkeepers handle missing receipts at month-end. I'm not selling anything today. I want to hear how it actually works for you. Can you walk me through the last time you were chasing receipts?\"\n\n## Section 1: Last episode (about 7 minutes)\n\n- Which client, which month was the most recent time you chased receipts?\n- How did you first notice something was missing?\n- What did you do next? And then what?\n- Who did you contact, how, and how many times?\n- How long from first noticing to having what you needed? How much of that was your time, and how much was waiting?\n- What happened to the items still missing at the end? (Written off, coded to suspense, client paid late?)\n- Can you show me the message or list you used? (Observed, if they can.)\n\n## Section 2: Cost (about 4 minutes)\n\n- Roughly how many clients do you chase receipts for each month? How many hours does that take? How did you estimate that?\n- If you check last month's timesheet or calendar, how much of it went to chasing?\n- What did that time displace? What did you not get to?\n- Has a missing receipt ever led to an error, a late filing, or a question from the tax authority or a client? Tell me about the most recent one.\n\n## Section 3: Tools and past attempts (about 4 minutes)\n\n- What do you use today to collect receipts? (Email, WhatsApp, a portal in your practice software, a shoebox, an app?)\n- Where do they end up? (Xero, QuickBooks, a spreadsheet, something else?)\n- Have you tried anything to fix this? What, when, and why did you stop? What did it cost?\n- What is the last software you cancelled or did not renew? Why?\n- If you have never tried anything, what kept you from looking? (Unknown, but useful.)\n\n## Section 4: Who pays (about 4 minutes)\n\n- When you chase receipts, is that included in your fee, billed separately, or absorbed?\n- Show me an invoice line if you have one.\n- Have you ever charged a client for chasing admin? What happened?\n- What is the last tool you paid for that handled client admin? Who approved it, and how long did that take?\n- What do you pay today for anything close to this?\n\nDo not ask \"Would you pay $X a month?\" It produces an imagined answer.\n\n## Section 5: Client-facing (about 3 minutes)\n\n- Last time one of your clients got an automated reminder from anyone (you, your practice software, their bank), how did they react? Did you hear about it?\n- Which clients would be least likely to respond to a text about receipts? Why?\n- Who at the client usually handles receipts: the owner, an employee, or you?\n\n## Section 6: Matching (about 2 minutes, only if time allows)\n\n- When a receipt arrives, how do you match it to the bank line today? How long does that take?\n- What is the most recent match that went wrong, and how did you catch it? (Split payments, cash, shared cards, foreign currency.)\n\n## Closing: the commitment ask (about 3 minutes)\n\nOnly ask this after Sections 1 to 5, and only if they described a real episode.\n\nAsk for something specific and small. Choose one:\n\n- \"Could you send me the unmatched-transactions list for one client, anonymized? I want to try a manual version this week.\" (Observed, if they send it.)\n- \"Would you try a manual test on one client's missing receipts next month? It's about 30 minutes of your time, and I'd show you the results.\" Get a date.\n- \"Which client should we start with?\"\n\nDo not ask for money. Pricing is not set, and the purpose here is to test whether they will spend time.\n\nRecord each ask as **requested**, **agreed**, **completed**, or **declined**, with the reason for a decline.\n\nDo not accept \"sounds great,\" \"add me to the list,\" or \"let me know when it's ready\" as commitments.\n\n## Questions and phrasings to avoid\n\n- \"Would you use / pay for / recommend this?\"\n- \"Does this sound useful?\"\n- \"Would you want a feature that...?\" Ask instead: \"What's the last time that happened? How did you deal with it?\"\n- Describing the product before Section 5.\n- Taking a feature wish list at face value. If they propose one, ask: \"What would that let you do? How do you handle it today?\"\n\n## After each call (about 10 minutes)\n\nFill in:\n\n- Role and number of clients they chase for\n- Date and client of the last episode, with the time estimate and its source\n- Tools used now and past attempts\n- Who pays for the work\n- Commitment status (requested, agreed, completed, declined)\n- Unknowns that still matter\n- Whether any praise came after you described the product\n\n## Decision thresholds\n\nSet these before the first call so the results cannot be argued into a different answer afterwards:\n\n- **Pain:** what number of hours per month, from a real episode, counts as enough? Write the number down.\n- **Payment:** if the cost is absorbed and never billed, note that bookkeepers are unlikely to pay for this themselves.\n- **Commitment:** how many of the eight non-friend bookkeepers must agree to a dated test before you continue? Write the number down.\n"}]}, "opus-1": {"cost_usd": 0.14166, "seconds": 51, "output_tokens": 4297, "files": [{"name": "discovery-call-guide.md", "text": "# Discovery Call Guide: Receipt-Chasing Tool for Freelance Bookkeepers\n\n10 calls, 25 minutes each, booked through the bookkeepers' Slack community.\n\n## Why last round went wrong, and the rules this time\n\nEveryone saying \"that would be amazing\" and then nobody joining the waitlist usually means the calls were about **the idea** when they should have been about **the person's life**. Asking someone to react to your idea gets you compliments. Asking about what they actually did gets you facts.\n\n1. **Don't describe the product until the last 5 minutes, if you describe it at all.** Once they know what you're building, everything they say is a reaction to it.\n2. **Ask about the past, not the future.** \"Last month-end, what happened?\" beats \"Would you use...?\" Ask for specific times and real numbers, not what they usually do.\n3. **Treat compliments as noise.** If you hear \"that's a great idea,\" steer back: \"Thanks. Can I ask how you handled this last week?\"\n4. **Dig into emotion and workarounds.** If they've built a spreadsheet, a template text, or a rule about it, the pain is real. If they shrug, it isn't.\n5. **End every call by asking for something that costs them a little.** A yes that costs nothing tells you nothing. (See the closing section.)\n6. **Let them talk most of the time.** Aim for them speaking about 80% of the call. Don't fill silences.\n7. **Being a former bookkeeper helps and hurts.** It builds trust fast, but you'll fill in answers yourself. When you catch yourself thinking \"yeah, I know how that goes,\" ask \"walk me through it anyway.\"\n\n---\n\n## Call script (25 min)\n\n### 1. Opening (2 min)\n\n> \"Thanks for making time. I used to be a bookkeeper and I'm trying to understand how other people run their practices, especially the messy parts of month-end. I'm not selling anything today. The most useful thing you can do is tell me how things really work, including the boring parts. OK if I take notes?\"\n\nDon't mention receipts or a tool yet.\n\n### 2. Their practice (3 min)\n\nQuick context so you can segment answers later:\n\n- How many clients do you have right now? What kinds of businesses?\n- Solo, or do you have help (a VA, a subcontractor, a partner)?\n- How do you bill: hourly, fixed monthly fee, or a mix?\n- Which software do you work in day to day (QBO, Xero, Dext, Hubdoc, something else)?\n\n### 3. Walk through the last month-end (8 min). This is the core of the call.\n\n> \"Take me back to your most recent month-end close. Pick one client. What happened, step by step?\"\n\nLet them tell it. Only bring up receipts if they never do, and then ask: *\"Were there any transactions you couldn't categorize or document?\"*\n\nWhen missing receipts come up, dig in:\n\n- How many transactions were missing documentation for that client? Across all your clients?\n- What did you do first? Then what? (Email, text, call, shared folder, a list in a portal?)\n- How many times did you have to follow up? How long until you got everything back?\n- Did you ever get all of them? What happened to the ones you didn't get?\n- What does the client send back: a photo, a forwarded email, a PDF, a shoebox?\n- Once you had it, how did you match it to the bank line?\n- Who on the client side do you chase: the owner, an office manager, employees with cards?\n\n**If they don't bring up receipts at all, or describe them as a minor annoyance, that's a real finding.** Write it down. Don't push.\n\n### 4. What it costs and what they've tried (5 min)\n\nTime:\n- Roughly how many hours did chasing take you last month? *(Get a number for last month specifically, not \"usually.\")*\n- Is that time billable? Do you actually bill it, or eat it?\n- Has a client's missing paperwork ever delayed a close, a tax filing, or a deadline? What happened?\n\nWorkarounds (the strongest signal):\n- What have you tried to make this easier? *(Templates, scheduled reminders, Dext/Hubdoc client upload, a portal, a fee for late docs, firing a client?)*\n- What happened with that? Why did you stop or keep using it?\n- Have you searched for or paid for anything to fix this? What did you look at? Why didn't you buy it?\n- Do you have a policy for clients who don't send docs, like a catch-up fee or \"uncategorized\" left in the books?\n\n### 5. Money: who pays (4 min)\n\nDon't ask \"would you pay?\" Ask what they already pay for and how.\n\n- What tools do you pay for in your practice today? About how much a month total?\n- Which of those do you pay for yourself, and which do you pass on to clients or bundle into your fee?\n- Have you ever added a tool to a client's bill? How did the client react?\n- Last time you started paying for a new tool, what made you do it? Who decided?\n- If you charge a catch-up or clean-up fee, how much? How often do clients pay it?\n\nWhat this tells you: if they already pass software costs to clients, client-pays might work. If they absorb everything, the bookkeeper is the buyer, and the price has to beat their hourly rate times the hours they eat.\n\n### 6. Close and commitment (3 min)\n\nNow, briefly, you can describe the idea in one sentence:\n\n> \"I'm exploring something that texts clients for missing receipts and matches the photo to the bank line automatically.\"\n\nThen, instead of asking \"would you use it?\", **ask for one concrete next step**, sized to how much pain you heard:\n\n| Pain you heard | Ask |\n|---|---|\n| Strong (hours a month, workarounds, frustration) | \"Would you try an early version with 1 or 2 real clients next month?\" Or, stronger: \"I'm taking a small deposit or paid pilot at $X. Want in?\" |\n| Medium | \"Could you send me your list of missing items from last month-end, anonymized? I want to see the real volume and formats.\" |\n| Any | \"Who else in the community deals with this the most? Could you introduce me?\" |\n| Any | \"Can I follow up in 3 weeks with something to look at, 20 minutes?\" (Book it right then.) |\n\n**If they say yes, that's real interest. If they hesitate, they were being polite. Both are useful data.**\n\nFinish with:\n- \"Is there anything I should have asked but didn't?\"\n\n---\n\n## The two friends\n\n- Their answers will lean positive. **Count their facts (hours, tools, workarounds), but don't count their enthusiasm or commitments** when you decide whether to build.\n- Ask them for something only a friend can give you: honest criticism. \"Be blunt: what about this would make you *not* use it?\"\n- Think about doing them last, once your questions are polished, or first as practice runs.\n\n---\n\n## Questions to avoid, and what to ask instead\n\n| Don't ask | Ask instead |\n|---|---|\n| Would you use a tool that...? | How did you handle this last month? |\n| How much would you pay for this? | What do you pay for today, and who covers it? |\n| Is chasing receipts a big problem? | How many hours did chasing take last month? |\n| Do you think clients would reply to texts? | How do your clients send you documents now? What gets the fastest reply? |\n| Would your clients pay for it? | Have you ever passed a tool cost on to a client? What happened? |\n| Do you like this idea? | What have you already tried to fix this? |\n\n---\n\n## After each call: 2-minute scorecard\n\nFill this in right away, before you talk to anyone else.\n\n| | |\n|---|---|\n| Name / friend? | |\n| # clients, billing model | |\n| Missing receipts last month (count) | |\n| Hours spent chasing last month | |\n| Billed for it? (yes / no / partly) | |\n| Workarounds built or tools tried | |\n| Currently pays for tools? Passes cost to clients? | |\n| How clients send docs today | |\n| Did the pain come up **before** you prompted it? (Y/N) | |\n| Commitment given (pilot / deposit / data / intro / follow-up / none) | |\n| Best quote (their exact words) | |\n| What surprised me | |\n\n## What counts as success across all 10 calls\n\nDecide these thresholds now, before the calls, so you can't talk yourself into a yes afterwards:\n\n- **Go signal:** at least 4 of 10 raise receipt-chasing without prompting, report 2+ unbilled hours a month, and **at least 3 agree to a pilot with real clients or put down money.**\n- **Rethink signal:** most people mention it only after prompting, already have a \"good enough\" fix (Dext/Hubdoc plus a reminder template), or no one commits to anything beyond \"keep me posted.\"\n- **Who-pays answer:** a clear majority pattern (bookkeepers absorb tool costs vs. pass them to clients). If it's split, segment by billing model (hourly vs. fixed fee).\n\n*(The numbers above are placeholders. Change them to what you'd honestly need to see, but write them down before the first call.)*\n"}]}, "opus-2": {"cost_usd": 0.13677, "seconds": 50, "output_tokens": 4098, "files": [{"name": "discovery-call-guide.md", "text": "# Discovery Call Guide: Missing-Receipt Chasing\n\n**Format:** 25 min \u00b7 10 freelance bookkeepers (via Slack community, 2 are friends)\n**Goal:** Find out whether chasing receipts is a problem people already spend time and money on, before I mention the tool.\n\n---\n\n## Why the last round produced \"that would be amazing\" and no signups\n\nPeople are polite about ideas. They're accurate about what they've already done. So for this round:\n\n1. **Don't pitch until the last 5 minutes, or at all.** Once I describe the tool, every answer after that is about my idea, not their week.\n2. **Ask about the past, not the future.** \"Last month, how many\u2026\" instead of \"Would you\u2026\". Hypothetical questions get hypothetical answers.\n3. **Get numbers and stories.** \"A lot of time\" isn't data. \"Six hours across 4 clients in March\" is.\n4. **End with a real ask.** Something that costs them a bit of time, reputation, or money. How they respond to it is the result of the call. Compliments don't count.\n5. **Don't mention I used to be a bookkeeper until the end.** If I lead with it, they'll agree with what they assume I think. Use it later to build rapport.\n\n---\n\n## Call structure (25 min)\n\n| Time | Section | Goal |\n|---|---|---|\n| 0\u20132 | Opener | Set expectations; ask for honesty |\n| 2\u20137 | Their practice | Context for weighting answers |\n| 7\u201315 | Last time they chased receipts | **Core.** Cost in time, pain, workarounds |\n| 15\u201319 | Current tools & spend | What they use today; what they pay for |\n| 19\u201322 | Who pays | Their money vs. client's money |\n| 22\u201325 | Commitment ask + referrals | The actual signal |\n\n---\n\n## 0\u20132 min \u00b7 Opener\n\n> \"Thanks for making time. I'm researching how freelance bookkeepers handle month-end. I'm not selling anything today. I mostly want to hear how you actually work. The most useful thing you can do is be blunt, including if something I ask about isn't a problem for you at all.\"\n\n---\n\n## 2\u20137 min \u00b7 Their practice (keep it quick)\n\n- How many active clients do you have right now? What kind of businesses?\n- Roughly how are you paid: monthly retainer, hourly, per-transaction?\n  - *(This matters: hourly bookkeepers may **bill** for chasing time, so it's not their loss. Retainer bookkeepers absorb it.)*\n- What does your month-end close look like, start to finish?\n- Which software do you work in? (QBO, Xero, etc.)\n\n---\n\n## 7\u201315 min \u00b7 The last time they chased receipts (core section)\n\n**Anchor on one recent, specific instance.**\n\n- \"Think about your last month-end close. Walk me through what happened with missing receipts or documentation.\"\n- How many transactions were missing support? Across how many clients?\n- What did you do first? Then what? (Email, text, a shared folder, a spreadsheet of missing items?)\n- How many times did you have to follow up before you got them?\n- How long did that take you in total that month? *(Push for a number. \"Is that closer to 1 hour or 10?\")*\n- What happened to the ones you never got? (Uncategorized? Coded to \"ask my accountant\"? Written off?)\n- Which client is the worst at this? What's it like dealing with them?\n- Have you ever lost a client, raised a fee, or dropped a client over this?\n\n**Probe whatever they bring up:**\n- \"Why was that annoying?\" / \"What did that cost you?\"\n- \"Have you tried anything to fix that?\" \u2190 **Key question.** If they haven't tried anything, it probably isn't a big enough problem.\n- \"Where does this rank against your other month-end headaches?\" *(Ask them to list their top 3 before I suggest anything.)*\n\n**\ud83d\udea9 Watch for:** \"It's annoying but it's just part of the job.\" That's a real answer. Write it down; don't argue with it.\n\n---\n\n## 15\u201319 min \u00b7 What they use today\n\n- What tools do you use for collecting documents from clients today? (Dext, Hubdoc, QBO receipt capture, Client Hub, Karbon, TaxDome, Google Drive, email, WhatsApp\u2026)\n- How did you choose it? What did you use before?\n- What do you pay for it? Who pays: you or the client?\n- What does it do badly? When did that last bite you?\n- Have you ever looked for something better? What did you find, and why didn't you switch?\n- Do any of your clients actually use the receipt-capture app you gave them? How many out of how many?\n\n*(That last question matters: if the existing tools already work and the real problem is client behavior, a texting tool might not fix it either. Find out why clients don't send receipts.)*\n\n---\n\n## 19\u201322 min \u00b7 Who would pay\n\nAsk about **what they've done**, not about my price.\n\n- What software do you pay for yourself, for the practice? Roughly how much per month in total?\n- Do you pass any software costs through to clients? How did clients react?\n- Have you ever asked a client to pay for or adopt a tool? What happened?\n- When a client is chronically missing receipts, do you charge for the extra work? *(If yes: how much? That's a natural price ceiling and a value anchor.)*\n- Who decides on new software in your practice? Just you, or a partner/firm?\n\n**Don't ask:** \"Would you pay $X for this?\" Everyone says yes, and it means nothing.\n\n---\n\n## 22\u201325 min \u00b7 Pitch briefly (optional), then the commitment ask\n\nOnly now, in one or two sentences:\n\n> \"What I'm thinking about building: it texts your clients for the specific missing receipts and matches the photo they send back to the bank line automatically. I used to do this job myself, which is why I'm interested.\"\n\nThen **stop talking** and watch their reaction. Follow-ups:\n- \"What would have to be true for you to actually switch to that from [what they use now]?\"\n- \"What's the first thing that worries you about it?\"\n\n**The commitment ask. Pick the one that fits the call:**\n\n| Strength | Ask |\n|---|---|\n| Weak | \"Can I follow up in 3 weeks with a mockup?\" |\n| Medium | \"Would you send me your current missing-receipts list (anonymized) so I can see the real format?\" |\n| Medium | \"Could you introduce me to 1\u20132 other bookkeepers who'd have strong opinions on this?\" |\n| Strong | \"I'm running a manual pilot: I'll chase receipts for one of your clients myself next month. Would you pick a client for it?\" |\n| Strongest | \"Pilot slots are $X/month, refunded if it doesn't save you time. Want one?\" |\n\n**How to score it:** a yes to a medium or strong ask is a real signal. \"Sure, keep me posted\" is a polite no.\n\n**Always close with:**\n- \"Who else should I talk to about this?\"\n- \"Is there anything I should have asked but didn't?\"\n\n---\n\n## Handling the two friends\n\n- **Run their calls last.** By then I'll know what typical answers sound like, so I'll notice if theirs are just kind.\n- Say it out loud: \"You're my friend, so you're the worst person to ask, which is why I need you to be harsh. Tell me why this won't work.\"\n- **Count their data separately.** Don't let their enthusiasm into the \"would pay\" count.\n- They're good for the strong commitment asks (pilot, sharing real data) because they trust me. Use that.\n\n---\n\n## Things to avoid saying\n\n| \u274c Don't ask | \u2705 Ask instead |\n|---|---|\n| \"Would you use a tool that\u2026?\" | \"What did you do the last time\u2026?\" |\n| \"Is chasing receipts a big pain?\" | \"How long did it take last month?\" |\n| \"Would you pay $20/month?\" | \"What do you pay for now? Who pays?\" |\n| \"Don't you hate when clients\u2026?\" | \"Tell me about your most difficult client.\" |\n| \"That would save you time, right?\" | *(silence, let them talk)* |\n\nAlso: avoid jargon-matching. Being a former bookkeeper makes it easy to finish their sentences. Don't.\n\n---\n\n## Notes template (fill in during or right after each call)\n\n```\nName / practice size / billing model:\nFriend? Y/N\n\nHours spent chasing receipts last month (their number):\n# clients affected:\nWhat they do today (tools + process):\nWhat they pay today, and who pays:\nHave they tried to fix it? What:\nRank of this problem among their top 3:\nBest verbatim quote:\n\nReaction to pitch (if given):\nCommitment ask made:\nTheir response: [declined / weak yes / medium / strong]\nReferrals given:\n\nSurprises / things I was wrong about:\n```\n\n---\n\n## Scoring after all 10 calls\n\nDecide the thresholds **before** the calls so I can't move them afterwards:\n\n- **Problem is real if** \u2265 5 of 8 non-friends report \u2265 2 hrs/month chasing **and** have already tried something to fix it.\n- **Willingness to pay is real if** \u2265 3 non-friends accept a medium or strong commitment ask.\n- **Who pays:** tally \"me\" vs. \"pass to client\" vs. \"billable anyway\". If most are hourly and bill the chasing time, the pitch has to be about something other than saving them time (cash flow, cleaner books, fewer annoyed clients).\n- **Kill / pivot signal:** most people say it's \"just part of the job,\" haven't tried anything, and nobody takes even a medium ask.\n"}]}, "opus-3": {"cost_usd": 0.1452, "seconds": 51, "output_tokens": 4452, "files": [{"name": "discovery-call-guide.md", "text": "# Discovery Call Guide: Missing-Receipt Chasing\n\n**Format:** 25 min \u00b7 10 freelance bookkeepers (booked via the bookkeepers' Slack) \u00b7 2 are friends\n\n## Why the last round produced \"that would be amazing\" and no signups\n\nPeople are polite about ideas. When you describe the product, you get compliments, and compliments cost nothing. To avoid that this time:\n\n1. **Don't describe the product until the last 5 minutes, or at all.** Don't say \"I'm building a tool that\u2026\" in the intro.\n2. **Ask about what already happened, not what they'd do.** \"Walk me through last month\" gets facts. \"Would you use\u2026\" gets guesses.\n3. **Ask for something specific at the end.** Make it a concrete next step that costs them a little time, money, or reputation, not \"can I add you to the waitlist?\"\n4. **Count what they did, not what they said.** Score each call right after (template at the bottom).\n\n## Before the calls: decide what counts as a yes\n\nWrite these down now so you can't move the bar after the calls. Suggested starting points (adjust them):\n\n| Question | Promising if\u2026 | Worrying if\u2026 |\n|---|---|---|\n| Is chasing receipts a real cost? | \u22655 of 8 non-friends can give a specific recent number (hours, receipts, days late) without being prompted, and it bothers them | Answers are vague (\"it's annoying sometimes\"), or they've already solved it with a template email |\n| Will someone pay? | Several already pay for tools out of their own pocket for this kind of work, **or** already pass software costs to clients | Nobody pays for anything beyond their ledger software, and nobody has looked for a fix |\n| What's the competition? | They use workarounds (email, texts, spreadsheets, \"uncategorized\" left open) | They're happy with an existing tool and can't name something it does badly |\n| Commitment | \u22653 non-friends take the concrete next step (see section 5) | Lots of \"send me info when it's ready\" |\n\n---\n\n## The script (25 min)\n\n### 1. Opening (2 min)\n\n> \"Thanks for doing this. I used to be a bookkeeper, and I'm trying to understand how freelance bookkeepers run month-end with their clients today. No pitch. I'm mostly going to ask about how you actually work. It's fine to say something's boring or doesn't matter to you. That's useful too.\"\n\nDon't mention receipts yet. If they bring up missing documents without you asking, that tells you something.\n\n### 2. Their practice in general (3 min)\n\n- How many clients do you have right now? Roughly what kind of businesses?\n- What's your stack: ledger (QBO, Xero, other), plus anything else you pay for?\n- Walk me through your month-end close for a typical client. What are the steps?\n- **What's the most annoying part of that?** *(Note whether receipts or missing docs come up before you mention them.)*\n\n### 3. The last time they chased receipts (10 min, the most important part)\n\nKeep pulling them back to **one specific, recent instance.**\n\n- Think about the last client you closed. Were any receipts or documents missing? How many?\n- What did you do to get them? *(Email? Text? Call? A shared folder? A list in a spreadsheet?)*\n- How many times did you have to follow up? How long until you had everything?\n- Did you ever get all of them? What happens to the ones you don't get? *(Leave uncategorized? Ask-my-accountant account? Write a note in the file?)*\n- **Roughly how much time did that take you, for that client, that month?** Then: how many clients is it like that for?\n- Is any of that time billable? Do you charge for it, or eat it?\n- Has a missing receipt ever caused a real problem: a late close, a tax-time scramble, an audit, a client you dropped?\n- Which clients are the worst for this, and why? *(Industry, how old they are, how they pay, whether they use a card)*\n\n**Follow-ups to use when answers get vague:**\n- \"Can you give me an example from the last month?\"\n- \"Do you remember roughly how many?\"\n- \"What did you do next?\"\n- \"Why does that bother you?\" / \"Why doesn't that bother you?\"\n\n**Watch for:** \"I always\u2026\", \"Usually\u2026\", \"I'd probably\u2026\". Turn these back into a specific instance.\n\n### 4. What they've tried and what they pay for (5 min)\n\nWhat they use today:\n- Have you tried anything to make this easier? What? *(Receipt capture apps like Dext or Hubdoc, the ledger's own receipt feature, client portals, email templates, a VA)*\n- Is it still in use? If they stopped: **why did they stop?**\n- If they never tried anything: why not? *(If it's a real pain and they've never looked for a fix, it's probably not as painful as it sounds.)*\n- What do your clients actually do with the receipt-capture app you give them? Do they use it?\n\nPayment (ask what they already do, not what they'd do):\n- What software do you pay for yourself each month? Roughly how much in total?\n- Is any of that passed on to clients, either as a line item or built into your fee?\n- Who pays for the ledger subscription: you or the client?\n- Last time you started paying for a new tool for the business, what was it? How did you decide? Did you need anyone's OK?\n- Have you ever asked a client to pay for or install a tool? How did that go?\n\nDon't ask \"how much would you pay for this?\" The answers aren't reliable.\n\n### 5. Closing and the ask (5 min)\n\nOnly now, and briefly, if it fits:\n\n> \"One thing I'm exploring is automatically texting clients for the missing receipts and matching the photo to the bank line.\"\n\nThen **ask for something concrete right away**, not \"what do you think?\" Pick the ask based on how strong the pain sounded:\n\n- **Strong pain:** \"I'm running a hands-on pilot next month for a few bookkeepers. I'd do the chasing for one of your real clients. Would you pick one and introduce me?\" *(Introducing you to a client puts their reputation on the line. That's a real signal.)*\n- **Strong pain, plus they already pay for tools:** \"The pilot is $X/month and I'm taking the first 5. Do you want one of the spots?\" *(A pre-order or deposit is the strongest signal you can get.)*\n- **Medium pain:** \"Could you send me a redacted export of last month's missing items / uncategorized transactions for one client? I want to see what the real list looks like.\" Or: \"Can we do a 20-minute screen share at your next month-end so I can watch you chase?\"\n- **Always:** \"Who else should I talk to? Can you introduce me?\"\n\nIf they say \"send me info when it's ready,\" count that as a polite no.\n\n---\n\n## The two friends\n\n- Interview them **first**, as practice, to tighten the script.\n- Their opinions about the idea don't count toward your decision. Their **facts** do (hours, number of clients, tools they pay for).\n- Score them separately in the tally.\n\n## Keep in mind: the sample\n\nPeople who answer a call request in a bookkeepers' Slack are probably more engaged and more open to new tools than average. If even this group won't commit, that's a strong negative signal. If they do commit, check whether it holds with bookkeepers outside the community before building much.\n\n---\n\n## After each call: fill this in right away (5 min)\n\n```\nName / friend? (Y/N):\nClients:            Ledger:            Tools paid for (monthly $):\n\nDid they bring up missing receipts before I did? Y/N\nLast concrete instance: (client type, # receipts missing, # follow-ups, time to resolve)\nTheir estimate of time per month on chasing:\nBillable or eaten?\nConsequences they've actually had:\nWhat they've tried / why they stopped:\nWho pays for software today (them / client / mix):\n\nPain (1\u20135, based on facts, not tone):\nAsked for:                       Result:\n  [ ] Pre-pay/deposit   [ ] Intro to a client   [ ] Data/screen share   [ ] Referral   [ ] Nothing\n\nQuotes worth keeping (exact words):\nCompliments / hypotheticals I heard (to ignore):\nWhat surprised me:\n```\n\n## Final tally (after all 10)\n\n| | Non-friends (8) | Friends (2) |\n|---|---|---|\n| Mentioned receipts unprompted | | |\n| Gave a specific recent time number | | |\n| Median hrs/month chasing | | |\n| Already pay for a related tool | | |\n| Pass software costs to clients | | |\n| Took a concrete next step | | |\n| Pre-paid / deposited | | |\n\nCompare this table against the \"what counts as a yes\" thresholds you wrote down before the calls.\n"}]}, "haiku-1": {"cost_usd": 0.00489, "seconds": 25, "output_tokens": 4337, "files": [{"name": "discovery-call-questions.md", "text": "# Discovery call questions: freelance bookkeepers\n\n## Goal of each call\n\nLearn three things, backed by specifics:\n\n1. What receipt chasing actually costs them, in hours and dollars, from recent events.\n2. Who pays for tools today, and whether they would pay.\n3. What they use now and what they have tried before.\n\nA good call ends with a concrete next step. \"That would be amazing\" does not count.\n\n## Ground rules\n\n- Do not pitch in the first 15 minutes. Describe the product only in section 4.\n- Ask about the last time something happened, not what they would do. \"Would you use this?\" gets a yes. \"Tell me about the last missing receipt you chased\" gets facts.\n- Do not ask \"would you pay for this?\" Ask what they pay for similar things now.\n- Ask for numbers: hours a month, clients affected, dollars lost or billed.\n- Write down their exact words, especially anything with a number in it.\n\n## Questions\n\n### 1. Their practice (3 min)\n\n- How many clients do you handle, and how many of them send you receipts regularly?\n- How do you bill: hourly, monthly retainer, per transaction?\n\n### 2. Receipt chasing today (8 min)\n\n- Walk me through the last missing receipt you chased, from the moment you noticed it to the moment it was matched.\n- How often does that happen in a typical month?\n- How many hours a month does it take? How did you arrive at that number? Can you check it against a recent month?\n- What did the last one cost you? Did it delay a close, a tax filing, or a client conversation?\n- Who handles it on your side: you, an assistant, or the client?\n- What do you do when a client never sends the receipt?\n- Do you absorb that time or bill it back? Has any client ever paid you for it?\n\n### 3. Current tools and past attempts (5 min)\n\n- What tools touch this today? Probably email, texts, portals, receipt apps, spreadsheets. Which of them did you pay for?\n- What did they cost, and who paid: you or the client?\n- Have you tried to fix this before? What happened, and why did you stop?\n- What did you like and dislike about the last tool you used for it?\n\n### 4. The idea (5 min, only after sections 1 to 3)\n\n- Here is what I am building: it texts the client for the missing receipt and matches the photo to the bank line. Where in your workflow would that help most?\n- What would you not trust it to do on its own?\n- What would stop you from using it? Client privacy, texting clients under your name, accuracy, bank access?\n- Which client would be the first to try it? Which would be the hardest?\n- Who would need to approve it: you, your clients, or your firm?\n\n### 5. Money and commitment (4 min)\n\n- If this saved the time you described, what is that worth to you per month?\n- Would you pay for it, or would your clients? What have clients paid for similar services?\n- Would you try it with two or three clients in the next month? What would you need from me to do that?\n- Who else do you know who has this problem? Could you introduce me to two bookkeepers you do not know well?\n\nClose with one specific ask, such as \"Can I send you a one-page pilot outline this week?\" or \"Can you introduce me to one client who would try it?\" A yes to a named next step counts. \"Send me more info\" does not.\n\n## Reading the answers\n\n**Strong signals:** a quantified problem (hours or dollars per month), a recent example, a workaround they already pay for, questions about price or timing, an offer of an introduction or a pilot.\n\n**Weak signals:** \"that's a nice idea,\" \"I'd use it,\" talk about the problem in general terms with no recent example, \"just send me the link.\"\n\n## Notes on the two friends\n\n- Their feedback will lean kind. Ask the hard questions directly: \"What would stop you from bothering?\" and \"What is the one thing you would least want me to change?\"\n- Log their calls separately in calls.md and report them apart from the other eight so they do not flatter the pattern.\n- Ask each of them for two introductions to bookkeepers they do not know. Those calls are the ones that count as independent.\n\n## After each call\n\nAdd to calls.md:\n\n- Hours a month spent on receipt chasing (their number, and how they got it)\n- Dollars lost or billed for it\n- The most recent specific example\n- Tools used and what they paid for them\n- Next step agreed: yes or no, and exactly what\n"}]}, "haiku-2": {"cost_usd": 0.00426, "seconds": 21, "output_tokens": 3317, "files": [{"name": "discovery-questions.md", "text": "# Discovery call guide: freelance bookkeepers and missing receipts\n\nGoal: find out whether chasing receipts is a costly, recurring problem they've already tried to solve, and whether they'll take a concrete next step. Compliments don't count. Past behavior and specific commitments do.\n\nTime: 25 minutes. Let them talk about their own experience for most of it.\n\n## Rules for the call\n\n- Ask about the last time it happened, not what they'd do in the future. \"Would you use this?\" always gets \"yes.\"\n- Don't pitch until the last 5 minutes. Pitching early turns the call into feedback on your idea.\n- Don't ask \"would you pay $X?\" Ask what they pay today for something similar, and who approves that spend.\n- When they say something positive, ask \"what makes you say that?\" or \"when did that last happen?\"\n- Write down exact numbers and quotes. \"It's a pain\" is not data. \"I sent 14 reminder texts last month and it took about 3 hours\" is.\n\n## 1. Context (3 min)\n\n- How many clients do you do monthly bookkeeping for, roughly?\n- Walk me through how you get receipts from clients today.\n- Who on the client side usually handles this: the owner, an office manager, or nobody?\n\n## 2. Past behavior: receipt chasing (8 min)\n\n- When did you last have to chase a missing receipt? Walk me through it from the first message to getting it, or giving up.\n- How many missing receipts did you have at your last month-end or quarter-end close? Roughly how many were never resolved?\n- In a typical month, how many hours go to chasing receipts and matching them to bank lines? Which client has been the worst, and why?\n- How do you match a receipt to a transaction once you have it? What goes wrong?\n- What happens when a receipt never shows up? Do you book it to a suspense account, ask the client about it, or let it go?\n- Has a client ever been penalized or had a problem with their books because of a missing receipt? What happened?\n\n## 3. What they've already tried (5 min)\n\n- What tools do you use for this today? Which ones are paid, and which are free or built into your accounting software?\n- Have you tried anything that didn't stick? What was it, and why did you stop using it?\n- What do your clients think about the current process? Have any of them complained, or pushed back on being asked?\n- If something saved you an hour a week on this, what would you have to change about how you work to use it?\n\n## 4. Money and who pays (3 min)\n\n- What did you last pay for a tool that saved you time on client work? How did you decide to buy it, and who had to agree?\n- Do you bill clients for admin or chasing time today? If not, why not?\n- If a tool cost you money per client, would you pass that cost on, absorb it, or stop offering the service? (Ask this only if they've already said they bill for admin time.)\n\n## 5. Pitch and test (4 min, after everything above)\n\nDescribe the idea in two or three sentences: it texts the client for the missing receipt and matches the photo to the bank line. Then ask for something specific:\n\n- \"Would you let me try this with one of your clients for the next 30 days? I'd set it up and check in with you each week.\" Then wait. Silence is useful information.\n- \"Can you introduce me to two other bookkeepers you know who deal with this?\"\n- \"Can I follow up with you on [specific date] to show you a working version?\"\n- \"Would you put your name on the waitlist today, and what's the one thing that would make you move off the waitlist and try it?\"\n\nRecord what they actually agree to, not what they say they'd consider.\n\n## 6. Close (1 min)\n\n- \"What's the thing I should have asked you but didn't?\"\n- \"Who else should I talk to about this, especially someone who would disagree with me?\"\n\n## Notes on the two friends\n\nTheir calls are useful but they're the most likely to be polite. Ask them the same questions, and ask them to be blunt: \"I need you to tell me if this is a bad idea. I won't be offended.\" Log their answers separately, and don't count their enthusiasm toward the total. Ask each of them for one referral to a bookkeeper they don't know well.\n\n## After each call: log these fields\n\n- Client count and how they collect receipts today\n- Number of hours per month on receipt chasing (their estimate, with the basis for it)\n- Tool(s) tried and why they stopped\n- Who pays for admin time today\n- Commitment reached, scored as:\n  - 0: compliments only\n  - 1: asked for a follow-up\n  - 2: agreed to a referral or introduction\n  - 3: agreed to a pilot or to a specific dated next step\n- Exact quote that best captures their pain\n"}]}, "haiku-3": {"cost_usd": 0.00421, "seconds": 19, "output_tokens": 3237, "files": [{"name": "discovery-questions.md", "text": "# Discovery call questions: freelance bookkeepers\n\nGoal: learn how much chasing missing receipts costs them, who would pay, and what they use today. Get a real commitment at the end, not compliments.\n\nGround rules\n- Ask about what they have done, not what they would do. \"Would you use this?\" gets \"yes\" every time.\n- Don't pitch. Don't describe the product until the last 3 minutes, and only if they ask.\n- Ask for specific examples: the last client, the last month, the last time it went wrong.\n- Let silence sit after answers. The useful detail often comes second.\n- Friends (2 of 10): ask them to be harsher than they want to be. Ask \"what would make you not bother?\" Mark their answers as lower-confidence if they only agree.\n\n## Timing (25 min)\n\n| Block | Time | Purpose |\n|---|---|---|\n| Context | 3 min | Who they serve, how many clients |\n| Receipt workflow today | 8 min | Actual process and time spent |\n| Cost and pain | 5 min | Quantify it, and what it displaces |\n| Money | 5 min | Who pays, and whether they have paid for tools |\n| Commitment | 4 min | Concrete next step |\n\n## 1. Context (3 min)\n\n- Walk me through a typical client engagement. What do you handle for them?\n- How many clients do you have right now, and what share of them are chasing receipts or statements?\n- How did you end up with this kind of work?\n\n## 2. Receipt workflow today (8 min)\n\n- Tell me about the last time a client's receipts were missing at month or quarter end. What happened, step by step?\n- How do you usually ask for them? Email, text, a shared folder, a portal, a phone call?\n- How many reminders does it usually take? What was the longest it's gone on?\n- What happens when a client never sends one? How do you code that transaction?\n- Where do the photos and PDFs end up? How do you match them to bank lines?\n- What did you last do to fix this for a client, if anything?\n\nProbes:\n- \"What did you do next?\"\n- \"How long did that take, roughly?\"\n- \"Who else was involved?\"\n\n## 3. Cost and pain (5 min)\n\n- In a typical month, how many hours go to chasing documents across all clients? Roughly.\n- Which part of that is the most annoying? Which part do you dread?\n- Has missing receipts ever cost you a client, a late filing, a penalty, or a fight with a client? Tell me about it.\n- What did you stop doing, or charge less for, because of this chasing?\n- If you could get that time back, what would you do with it?\n\nProbes:\n- \"Is that every month, or only at year end?\"\n- \"What's the worst month you've had for this?\"\n\n## 4. Money (5 min)\n\n- Do you currently pay for any tool to help with this, or with document collection in general? Which one, and what do you pay?\n- When you last bought a tool for your practice, how did you decide? Who signed off?\n- Who usually pays for document-chasing time today: you, through your fee, or the client directly?\n- Do you bill clients for chasing time, or absorb it? Have you ever tried to bill it?\n- If a tool cut that chasing time, would you pay for it, or would you expect your clients to pay? What's the difference to you?\n\nDo not ask: \"How much would you pay for this?\" Ask what they pay for similar things now, and what they think a reasonable price would be for something that already saves them an hour a month.\n\n## 5. Current tools and workarounds (covered across sections 2 and 4)\n\n- What do you use today to track receipts? Spreadsheet, the accounting software, email folders, a shared drive?\n- Which accounting software do your clients use? Does it have a receipt-capture feature, and do you use it?\n- What have you tried that didn't work? Why did you stop?\n\n## 6. Commitment (4 min)\n\nThis is the part that failed last time. Don't end on \"this sounds great.\" Ask for something specific that costs them time or reputation.\n\nAsk one or more of these, in order of how much they cost the person:\n1. \"Can I send you a one-page summary to forward to a colleague who might also be interested?\"\n2. \"Would you be willing to try a pilot with one of your clients over the next two weeks? I'd set it up and check in with you.\"\n3. \"Can I book a 20-minute follow-up next week to show you a prototype and get your feedback on the matching step?\"\n4. \"Would you put your email on the early-access list today? I'll only contact you when it's ready to test.\"\n\nRecord what they agreed to, not what they said they liked. A \"yes\" to a forwarded summary is a weak commitment. A booked follow-up or a named client for the pilot is a strong one.\n\nIf they refuse everything, ask: \"What would have to be true for you to say yes to a pilot?\" Their answer is the most useful thing you'll hear on the call.\n\n## Do not ask\n\n- \"Would you use this?\" / \"Would you pay for this?\" (hypothetical, always yes)\n- \"Does this sound like a good idea?\" (invites politeness)\n- \"Do you think other bookkeepers would like it?\" (speculation)\n- Long product walkthroughs before they've described their own week\n\n## After each call\n\nFill in one line per call in calls.md:\n- Name, practice size, friend or stranger\n- Hours per month chasing documents (their number, not yours)\n- Current tool and what they pay for it\n- Who pays for chasing time today\n- Commitment reached (none / forward / pilot client / follow-up booked / early access)\n- Quote worth keeping, word for word\n\nAfter 10 calls, count how many reached a pilot client or follow-up. If fewer than 3 of 10 did, the problem is likely the idea or the customer, not the questions.\n"}]}}, "verdicts": [{"vs": "opus", "pair": "2", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "haiku+skill", "why": "A distinguishes working time from elapsed time and tracks concrete commitments more carefully. Both underprobe matching and automation constraints. B incorrectly treats medium commitments as payment validation and overlooks Slack recruitment bias when declaring demand real."}, {"vs": "opus", "pair": "2", "judge": "gpt", "order": "opus shown first", "winner": "haiku+skill", "why": "B separates working from elapsed time, recognizes recruitment bias, and requires specific commitments. A incorrectly equates nonfinancial commitments with willingness to pay. Both underprobe matching and automation constraints; B also misses explicit monthly transaction counts and frequency."}, {"vs": "opus", "pair": "2", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "tie", "why": "Both provide excellent, complete, and factually accurate scripts meeting all rubric criteria."}, {"vs": "opus", "pair": "2", "judge": "deepseek", "order": "opus shown first", "winner": "tie", "why": "Both provide excellent, concrete, 25-minute scripts that fully meet all rubric criteria. Differences are minor."}, {"vs": "haiku", "pair": "1", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "haiku+skill", "why": "A better separates evidence from praise and tracks completed commitments. Its required sections total 26 minutes before the optional fit check. Both omit explicit chasing-time isolation and client-funded purchasing evidence. B adds leading hypotheticals and overvalues agreed next steps."}, {"vs": "haiku", "pair": "1", "judge": "gpt", "order": "haiku shown first", "winner": "haiku+skill", "why": "B better examines workflow, purchasing history, and completed commitments while controlling pitch and friendship bias. Its timed sections total 26 minutes before the optional fit check. Both underquantify affected clients or transactions and insufficiently distinguish client-funded adoption."}, {"vs": "haiku", "pair": "1", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "haiku+skill", "why": "A provides a more concrete, time-boxed script with a commitment ladder to address past praise-action gap."}, {"vs": "haiku", "pair": "1", "judge": "deepseek", "order": "haiku shown first", "winner": "tie", "why": "Both provide excellent, concrete, time-bound scripts that fully meet all rubric criteria."}, {"vs": "opus", "pair": "1", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "haiku+skill", "why": "A better probes automation constraints and distinguishes agreed from completed commitments. Its timed sections total 26 minutes before the optional fit check. B fits 25 minutes but overinterprets yeses, hesitation, and small-sample thresholds. Both leave cost quantification incomplete."}, {"vs": "opus", "pair": "1", "judge": "gpt", "order": "opus shown first", "winner": "opus", "why": "A fits 25 minutes and better investigates existing spending and client cost allocation. B totals 26 minutes before optional questions, but better probes automation controls and tracks completed commitments. Both inadequately isolate chasing effort and acknowledge recruitment bias."}, {"vs": "opus", "pair": "1", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "tie", "why": "Both provide excellent, call-ready scripts that fully meet all rubric criteria with accurate use of facts."}, {"vs": "opus", "pair": "1", "judge": "deepseek", "order": "opus shown first", "winner": "tie", "why": "Both provide excellent, call-ready scripts that fully meet all rubric criteria with accurate use of facts."}, {"vs": "haiku", "pair": "2", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "haiku+skill", "why": "A separates elapsed from working time, supports disconfirmation, demands specific commitments, and flags Slack recruitment bias. Both omit affected-client counts and automation constraints. B combines chasing with matching time and reintroduces waitlist promises despite the prior failure."}, {"vs": "haiku", "pair": "2", "judge": "gpt", "order": "haiku shown first", "winner": "haiku+skill", "why": "B better separates working time from elapsed time, examines actual purchases, defines concrete commitments, and acknowledges recruitment bias. Both lack explicit automation-fit probes; B misses monthly client/transaction counts, while A combines chasing with matching and includes weak waitlist signals."}, {"vs": "haiku", "pair": "2", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "tie", "why": "Both provide complete, call-ready scripts that accurately use the input and address all rubric criteria effectively."}, {"vs": "haiku", "pair": "2", "judge": "deepseek", "order": "haiku shown first", "winner": "tie", "why": "Both provide excellent, call-ready scripts that fully meet all rubric criteria, accurately using the input facts."}, {"vs": "haiku", "pair": "3", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "haiku+skill", "why": "A better grounds costs, purchases, and commitments in actual behavior. Its required blocks total 27 minutes, and its payment conclusions overreach. B weakens discovery with hypothetical pricing, waitlist commitments, insufficient automation probes, and an unsupported three-of-ten cutoff."}, {"vs": "haiku", "pair": "3", "judge": "gpt", "order": "haiku shown first", "winner": "haiku+skill", "why": "B better grounds costs in recent evidence, examines automation failures, and tracks completed commitments. Its script totals 29 minutes and makes unsupported payment inferences. A leaves workflow gaps, includes hypothetical pricing, and invents a three-of-ten decision threshold."}, {"vs": "haiku", "pair": "3", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "haiku+skill", "why": "A's script is more structured, avoids hypothetical payment questions, and includes stronger commitment tests."}, {"vs": "haiku", "pair": "3", "judge": "deepseek", "order": "haiku shown first", "winner": "tie", "why": "Both provide excellent, call-ready scripts that fully meet all rubric criteria."}, {"vs": "opus", "pair": "3", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "opus", "why": "B fits 25 minutes, invites low-pain answers, and acknowledges recruitment bias. A better separates active work from waiting but overruns time and wrongly infers nonpayment from unbilled chasing. Both inadequately probe automation requirements; B omits matching."}, {"vs": "opus", "pair": "3", "judge": "gpt", "order": "opus shown first", "winner": "opus", "why": "A fits 25 minutes and better investigates client-funded purchasing. B budgets 27\u201329 minutes and wrongly infers nonpayment from unbilled chasing. A underexplores matching and automation constraints; B omits missing-receipt quantities."}, {"vs": "opus", "pair": "3", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "tie", "why": "Both provide excellent, call-ready scripts that fully meet all rubric criteria with no significant flaws."}, {"vs": "opus", "pair": "3", "judge": "deepseek", "order": "opus shown first", "winner": "tie", "why": "Both provide excellent, call-ready scripts that fully meet all rubric criteria, accurately using the input facts."}]}, {"id": "sal-2", "family": "founding-sales", "prompt": "Help me set up our sales process, I'm the only one selling. Notes in notes.md.", "input": "notes.md", "skill": "kazanjy-founding-sales", "input_text": "Company: Tallyroom, software for independent hotels (20-150 rooms) to manage group bookings (weddings, conferences) instead of email + spreadsheets.\nMe: CEO, ex-hotel revenue manager. I've done 22 demos in 3 months from my network: 5 closed (USD 400/month), 9 \"interested, follow up in Q1\", 8 went silent.\nI don't have a CRM; I track deals in my head and a Notion page. I don't know why the 5 bought and the others didn't.\nAverage deal took 7 weeks. The decision maker is usually the general manager, but the sales manager uses it.\nGoal: close 15 more hotels in the next 4 months, and figure out what to hand over to a first salesperson later.\n", "rubric": [{"id": "c1", "criterion": "Fits the sales process to Tallyroom and a CEO selling alone.", "full_marks_when": "Provides a prioritized, manageable four-month plan for selling group-booking software to independent hotels with 20\u2013150 rooms. Uses the CEO's hotel revenue experience and existing network, and gives concrete actions and a sustainable weekly routine.", "zero_when": "Offers generic sales advice or a process that depends on a sales team, extensive tooling, or resources the CEO does not have."}, {"id": "c2", "criterion": "Uses accurate arithmetic and a credible plan for the growth target.", "full_marks_when": "Correctly calculates the observed demo-to-close rate as 5/22 = 22.7%. Fifteen additional hotels at USD 400/month add USD 6,000 MRR, bringing the stated customer base to 20 hotels and USD 8,000 MRR. Explains that matching the observed rate would imply about 66 demos over four months, while treating the small network-sourced sample as uncertain and accounting for the seven-week sales cycle and existing pipeline without double-counting.", "zero_when": "Misstates the input or arithmetic, treats interested hotels as committed customers, or promises the target based on an unsupported conversion assumption without allowing time for deals to close."}, {"id": "c3", "criterion": "Creates a practical way to learn why hotels buy, delay, or disappear.", "full_marks_when": "Specifies focused conversations with the five buyers and selected delayed or silent prospects, with questions about purchase triggers, previous workflows, measurable impact, objections, decision steps, and reasons for delay. Shows where to record findings and how those findings will change targeting, messaging, demos, or qualification.", "zero_when": "Assumes why prospects bought or disappeared, or merely recommends collecting feedback without questions, a recording mechanism, or decisions the findings will inform."}, {"id": "c4", "criterion": "Makes deal tracking and follow-up usable immediately.", "full_marks_when": "Provides a lightweight Notion setup or simple CRM structure with clearly defined stages and essential fields, including hotel, contacts and roles, need, stage, last interaction, next action, and due date. Gives distinct actions for the nine Q1 prospects and eight silent prospects, verifies what Q1 means for their timing, and includes a regular pipeline review and a rule for parking unresponsive deals.", "zero_when": "Only says to adopt a CRM or follow up, leaves deals without dated next actions, or treats all 17 unresolved prospects identically."}, {"id": "c5", "criterion": "Helps the CEO move qualified hotels through a real buying decision.", "full_marks_when": "Provides concrete qualification questions and a discovery-to-demo-to-decision sequence focused on group-booking problems and customer outcomes. Accounts for the sales manager as user and the general manager as usual decision maker, establishes how both participate, and ends meetings with an agreed next step, owner, and date.", "zero_when": "Relies on feature demonstrations and repeated check-ins, ignores either stakeholder, or assumes user enthusiasm establishes purchase authority and readiness."}, {"id": "c6", "criterion": "Builds evidence and materials that a first salesperson can actually inherit.", "full_marks_when": "Identifies a small set of records and reusable materials to build during the next four months: target-hotel criteria, discovery questions, demo flow, objections and responses, follow-up examples, stage definitions, deal histories, and observed conversion and cycle data. Explains how to turn tested practices into an onboarding playbook and distinguish validated findings from assumptions.", "zero_when": "Suggests hiring a salesperson as the solution, or recommends documenting a playbook without specifying its contents or how evidence from current selling will populate it."}], "runs": {"haiku-skill-1": {"cost_usd": 0.01227, "seconds": 63, "output_tokens": 13268, "files": [{"name": "sales-process.md", "text": "# Tallyroom Sales Process (working packet)\n\nSource: notes.md, dated 2026-10-08. Evidence labels: **O** = observed (counted in your notes), **R** = reported (your account, not yet verified), **H** = hypothesis, **U** = unknown.\n\n## 1. Assignment and evidence\n\n- **Deliverable:** A working founder-led sales process to win 15 more hotels in 4 months (by about 2027-02-08), and to learn why 5 hotels bought and 17 did not, so a first seller can later inherit a proven motion.\n- **Product and price:** Group-booking software for independent hotels with 20\u2013150 rooms. The 5 closes are at USD 400/month (R). Unknown whether price varies by room count (U).\n- **Available evidence:**\n  - O: 22 demos in about 3 months, all sourced from your network.\n  - O: Outcomes of 5 closed, 9 \"interested, follow up in Q1\", 8 silent.\n  - R: Average sales cycle is 7 weeks. The GM usually decides; the sales manager uses the product.\n  - U: Whether the 5 are paying, using the product, or achieving a measurable result.\n  - U: Why the 5 bought and the others did not.\n- **Constraints:** Solo founder. No CRM; deals tracked in your head and a Notion page. 7-week average cycle, so deals must start soon to close in the window.\n- **Missing inputs:** Room count, contact roles, and first-contact source for each of the 22 demos. Reasons for the 8 silent and the 9 interested. Current time cost of group bookings at a hotel.\n\n## 2. Stage and next proof\n\n- **Current demonstrated stage:**\n  - Value exchange is partly shown: 5 hotels closed at USD 400/month (R). Payment and implementation are not verified.\n  - Value creation is unverified (U). No measured outcomes are recorded.\n  - Repeated exchange is untested. All 22 demos came from your network, so the close rate may overstate what cold prospects will do.\n- **Missing proof:** First, whether the 5 paying hotels get measured value. Second, whether you can win hotels outside your network.\n- **Next experiment (owner: you):**\n  1. Within 2 weeks, interview each of the 5 customers. Record a baseline (e.g., staff hours per group booking, number of group events handled, missed or slow replies) and the current figure for each.\n  2. From weeks 3\u20136, work a batch of non-network prospects (section 3) through the discovery script in section 6, and record every step (section 7).\n- **Exit condition:** All 5 customers interviewed, with a baseline and a current measure recorded for each, and a stated answer to \"what changed?\" Only then move to a wider outreach push.\n- **Review dates and fallback:**\n  - 2026-10-22: 5 customer interviews complete; 9 interested and 8 silent tagged with reasons.\n  - 2026-11-19: first cohort of non-network conversations reviewed.\n  - 2026-12-17: bottleneck diagnosed (section 8).\n  - 2027-01-14: pipeline checked against the 15-hotel target.\n  - 2027-02-08: 4-month review. If the target is out of reach, reset it using observed conversion, not the hoped-for rate.\n\n## 3. Ideal customer and account qualification\n\nAll items below are hypotheses until the 5 customer interviews and first outside conversations confirm them.\n\n- **Organization conditions (H):** Independent hotel, 20\u2013150 rooms, that takes group bookings (weddings, conferences) and currently manages them by email and spreadsheet.\n- **Exclusions (H):** Hotels under 20 rooms; hotels with no regular group business; chains with a central group-sales system. Revise after the 5 interviews.\n- **People:**\n  - User: sales manager (R).\n  - Decision maker: GM (R, \"usually\").\n  - Budget authority: presumably the GM (H). Confirm in discovery.\n- **Current approach:** Email and spreadsheets (R).\n- **Timing:** Unknown (U). Ask each prospect for a specific upcoming event or a recent booking problem. Do not treat fit as urgency.\n\nSegment actions for the existing 22 demos:\n\n| Segment | Count | Action |\n| --- | --- | --- |\n| Closed | 5 | Interview each (section 2). Record why they bought, in their words. |\n| Interested, follow up Q1 | 9 | Ask each for a concrete trigger and date. Q1 2027 falls near the end of the 4-month window, so re-engage in November instead if a trigger exists. |\n| Silent | 8 | Record a one-line reason for each. Re-contact once, only with a new reason. |\n\n## 4. Master narrative and economics\n\n- **Current state (R):** Group bookings run through email and spreadsheets.\n- **Problem and consequences (U \u2192 H):** Discovery must establish these. Hypotheses to test: slow replies lose group leads; double-entry causes errors; staff time is absorbed by admin.\n- **Alternatives and gap (R):** Email, spreadsheets, and doing nothing. Gap: no single view of group bookings.\n- **Proposed change (R):** Tallyroom manages group bookings in one place.\n- **Evidence (O):** 5 hotels paying USD 400/month. Outcome evidence is unknown (U).\n- **Why act now (U):** Ask for the trigger in each conversation.\n- **Commercial proposal:** USD 400/month subscription (R). Confirm onboarding scope and terms before quoting.\n\nEconomics template (fill from the 5 interviews; do not present forecasts as results):\n\n| Business measure | Baseline and source | Proposed change | Calculation and period | Costs and assumptions | Validation plan |\n| --- | --- | --- | --- | --- | --- |\n| Staff hours per group booking | U (interview) | H (measure after onboarding) | hours saved \u00d7 bookings per month \u00d7 loaded hourly rate | Onboarding time, adoption | Owner: you. Re-measure at 60 days. |\n| Group leads lost to slow reply | U (hotel's own figures) | H | leads per month \u00d7 lost share \u00d7 average group value | Only if the hotel tracks it | Owner: you. |\n\nClassify each benefit as released capacity unless the hotel will actually reduce spend or add revenue. Do not add overlapping benefits.\n\n## 5. Requested selling assets\n\nNot requested yet. Outreach and demo drafts should follow the 5 customer interviews, so they use verified outcomes rather than hypotheses.\n\n## 6. Discovery and commitment plan\n\n- **Pre-call:** Check hotel size, group-event listings, and the GM and sales manager's names. Do not spend call time on facts already public.\n- **Meeting objective:** Confirm one current-state problem and agree on a specific next step.\n- **Opening question:** \"Walk me through the last group booking you handled, from first inquiry to confirmed.\"\n- **Branches:**\n  - Works fine: ask when something last went wrong. If nothing did, do not invent a gap.\n  - Manual workaround: who does it, how often, and what happens when it is delayed.\n  - Pain acknowledged, low priority: what would change its priority, and is any event coming up?\n  - Sales manager without budget authority: how is spend approved, and what would the GM need to see?\n- **Concrete incident:** The most recent group booking that went wrong. Ask for the hours it cost and any revenue lost.\n- **Problem summary:** Read the problem back in their words and get confirmation before any demo.\n- **Demo selection:** The group-booking workflow from inquiry to signed contract.\n- **Objection response:** If you do not know, say so. Offer the 5 customers as references only after their outcomes are recorded.\n- **Commitment request:** A 30-minute follow-up with the GM present, or a trial with a named hotel owner for the next step.\n- **If no commitment:** Record the reason and a revisit date. Revisit only when a real trigger is named.\n\n## 7. Opportunity and customer-success record\n\nReplace the Notion page and memory with one table (Notion database or spreadsheet). Fields:\n\n- Account, room count, source (network or outside), and contacts with roles (user, GM, budget).\n- Stage, first demo date, and the narrative version used.\n- Acknowledged pain (in the buyer's words) and current approach.\n- Objections, and the next action with owner and due date.\n- Value estimate, labelled as an estimate.\n- Customer-only: payment date, implementation owner, baseline, outcome measure, and validation date.\n\nStage definitions:\n\n1. Discovery held (pain acknowledged in their words)\n2. Demo held\n3. Proposal sent with price\n4. Verbal yes\n5. Signed and paid\n6. Implemented (first group booking run in Tallyroom)\n7. Outcome validated with the customer\n\nStages 5\u20137 are kept separate so payment is not counted as delivered value.\n\n## 8. Motion review and revision\n\nWeekly review, 30 minutes, every Friday:\n\n- **Counts by step and source:** conversations, discovery calls, demos, proposals, wins, losses. Keep denominators and the period.\n- **Current baseline (O):** 5 closes from 22 demos over about 3 months, a 23% close rate. Small sample, all from network; treat as uncertain.\n- **Pace needed for the goal:** 15 closes in about 4 months is about 3.75 per month. Current pace is about 1.7 per month, so the goal needs about 2.2x.\n- **Demo volume needed:** At 23%, 15 closes needs about 66 demos. A 7-week cycle means demos must start before about 2026-12-20 to close by 2027-02-08. Current pace is about 1.7 demos per week, so you would need roughly 4x. Decide whether the target or the conversion assumption needs revising after the outside-network test.\n- **Bottleneck hypothesis, to test first:**\n  - Prospects do not acknowledge pain \u2192 targeting problem.\n  - Pain acknowledged, no next step \u2192 discovery, explanation, or priority.\n  - Closes but results disappoint \u2192 product or onboarding.\n- **Alternative explanation:** Network buyers are people you already know well, which would inflate the close rate relative to cold prospects.\n- **Next review:** Founder, 2026-11-19, decision: keep, change, or stop the discovery script.\n\n## 9. Transfer plan\n\nOut of scope now. Repeatable selling beyond your network is not yet shown, and the method warns against hiring a first seller to find the motion. Revisit at the 2027-02-08 review once the outside-network test and customer outcomes are in.\n\n## 10. Review result\n\n| Criterion | Status | Note |\n| --- | --- | --- |\n| Scope | PASS | Deliverable and intended use stated. |\n| Maturity | PASS | Earliest missing proof is customer value. |\n| Evidence integrity | PASS | Labels used throughout. |\n| Next experiment | PASS | Owner, action, dates, exit condition. |\n| Customer profile | PARTIAL | Hypotheses only; confirm with interviews. |\n| Timing | PASS | Labelled as unknown. |\n| Narrative | PARTIAL | Consequences not yet confirmed. |\n| Economics | PARTIAL | Inputs missing. Do not quote ROI yet. |\n| Discovery | PASS | Branches defined. |\n| Commitment | PASS | Specific next step defined. |\n| Record quality | FAIL \u2192 fixed by section 7 | No CRM today. |\n| Customer success | FAIL \u2192 fixed by section 2 | No outcomes recorded today. |\n| Diagnosis | PARTIAL | Needs 2 more months of data. |\n| Metrics | PASS | Denominators kept; sample is small. |\n| Transfer readiness | N/A | Premature (section 9). |\n\n**Overall:** Not ready to scale outreach. Ready to start the customer interviews and the opportunity table now.\n\n**Immediate next actions:**\n\n1. 2026-10-10 (you): Build the opportunity table from the Notion page, with all 22 demos.\n2. 2026-10-22 (you): Interview all 5 customers. Record a baseline and outcome for each.\n3. 2026-10-22 (you): Tag the 9 interested with a trigger and date; record a reason for each of the 8 silent.\n"}]}, "haiku-skill-2": {"cost_usd": 0.01715, "seconds": 98, "output_tokens": 19797, "files": [{"name": "deal-tracker.csv", "text": "account,city,rooms,source,gm_name,sales_manager_name,decision_maker_confirmed,icp_fit_evidence,pain_in_their_words,current_process,stage,stage_entered_date,first_contact_date,discovery_date,demo_date,next_action,next_action_date,proposed_price_usd_month,closed_date,lost_or_nurture_reason,re_engage_date,baseline,outcome_result,outcome_validated_by,notes\n"}, {"name": "sales-process.md", "text": "# Tallyroom Sales Working Packet\n\nDraft dated 2026-10-08. Owner: founder (CEO).\nMethod basis: Peter Kazanjy's public founding-sales material (not affiliated or endorsed). The structure, evidence labels, and tracker are this document's own working conventions.\n\nEvidence labels: **Observed** (from your sales data), **Reported** (your statement, not yet checked), **Hypothesized**, **Unknown**.\n\n## 1. Assignment and evidence\n\n- **Deliverable:** A sales process you can run yourself now, a deal tracker to replace memory and Notion, and a definition of what a first salesperson would need to be handed later.\n- **Product and price:** Group booking software for independent hotels with 20-150 rooms, replacing email and spreadsheets for weddings and conferences. USD 400/month. **Reported.** Which features are shipped today is **Unknown**.\n- **Sales history:**\n  - 22 demos in about 3 months, all from your network. **Observed.**\n  - 5 closed at USD 400/month. **Observed.**\n  - 9 \"interested, follow up in Q1.\" **Observed.**\n  - 8 went silent. **Observed.**\n  - Average cycle: 7 weeks. **Reported.**\n- **Unknown:**\n  - Why the 5 bought.\n  - Whether any of the 5 gets measurable value from the product.\n  - Whether the 5 are fully live or still onboarding.\n  - Why the 8 went silent.\n  - The specific Q1 event, if any, behind each of the 9 follow-ups.\n- **Constraints:** You are the only seller. No CRM. The general manager (GM) usually decides, and the sales manager uses the product day to day.\n\n## 2. Stage and next proof\n\n| Stage | Status | Evidence |\n| --- | --- | --- |\n| Problem discovery | Partly shown | 14 of 22 demos led to a purchase or a follow-up. The pain they named is **Unknown**. |\n| Value creation | **Unproven** | No customer outcome recorded. |\n| Value exchange | Shown, inside your network | 5 paid. Procurement and onboarding details are **Unknown**. |\n| Repeated exchange | **Not shown** | Every demo came from your network. No cold or referred sourcing yet. |\n| Seller transfer | Not started | Correct for now. See section 9. |\n\n- **Current demonstrated stage:** Value exchange inside your network. Repeated exchange is not yet demonstrated.\n- **Earliest missing proof:** Whether the 5 customers get the result that made them buy. Without that, you cannot say why they bought or whether they will renew or refer.\n- **Second missing proof:** Whether hotels outside your network buy through a process you can repeat.\n\n### Experiment A: Interview the 5 paying hotels (weeks 1-4, owner: you)\n\n- Send the requests by 2026-10-15. Book 30 minutes each, and use the questions in section 5, \"Customer interview,\" below.\n- **Exit condition:** All 5 interviews logged. For each: the purchase trigger, who decided, what they did before, and whether a baseline and a change exist (measured, or estimated and labelled).\n- **Decision after exit:** If a trigger repeats across interviews, it becomes the timing signal in section 3. If not, the profile stays broad and the motion is not yet repeatable.\n- **Review:** 2026-11-05. **Fallback:** If fewer than 5 can be interviewed, record the outcome as Unknown and continue.\n\n### Experiment B: Sourcing outside the network (weeks 1-4, owner: you)\n\n- Ask each of the 5 customers for introductions to 2 peer hotels. Also build a list from hotel association directories and \"weddings\" or \"events\" pages on hotel websites.\n- Research and contact 40 accounts that match section 3. Log each one's source, reply, first meeting, and discovery call in the tracker.\n- **Exit condition:** 40 accounts contacted. Reply, meeting, and discovery rates by source are recorded. These become the baseline. They are not a pass or fail gate.\n- **Review:** 2026-11-05.\n\n### Goal check: 15 more hotels in 4 months\n\n- Historical demo-to-close: 5 of 22, or **23%**. With 22 demos the 95% range is about 10% to 43%, and the sample is network-sourced, so expect outside-network conversion to be lower.\n- At 23%, 15 closes need about **65 demos**.\n- A 7-week average cycle means demos must start by about **2026-12-21** to close before **2027-02-08**. That leaves about 10.5 weeks, or about **6 demos per week**.\n- Your recent pace: 22 demos in about 13 weeks, or **about 1.7 per week**.\n- The 9 Q1 follow-ups add roughly 2 closes at the same rate, if they convert and move earlier.\n\n**Conclusion:** At current conversion and sourcing, 15 in 4 months needs roughly 3.5 times your demo rate, from sources you have not yet proven. Treat it as a stretch target and revisit it at the 8-week review (2026-12-03), once the stage data exists.\n\n## 3. Ideal customer and qualification\n\nEach item is a hypothesis unless marked.\n\n- **Organization:** Independent hotel, 20-150 rooms. **Reported.**\n- **Event business:** Hosts weddings, conferences, or group events. Check the hotel's website for event or wedding space before first contact. **Hypothesized.**\n- **Current process:** Group enquiries handled by email, with room, date, and catering details tracked in spreadsheets. Holds and deposits tracked by hand. **Hypothesized.** Confirm in discovery.\n- **Exclusions:**\n  - Hotels with little or no group business.\n  - Hotels whose chain or PMS already covers group bookings. **Hypothesized.**\n- **People:**\n  - User: sales manager. **Reported.**\n  - Decision maker and budget holder: GM. **Reported.** Confirm on each deal.\n- **Pain indicators to listen for:** Slow replies to group enquiries, lost or double-booked holds, manual rooming or catering lists, sales manager time spent on coordination.\n- **Timing signals:** Unknown. The interviews in Experiment A should reveal any.\n\nQualification decision for each account: pursue, discover, nurture (confirmed pain, wrong timing), or exclude.\n\n## 4. Narrative and economics\n\n- **Current state:** Group enquiries arrive by email. The sales manager coordinates them across emails and spreadsheets. **Hypothesized.**\n- **Problem and consequences:** **Unknown.** Discovery must establish these in the buyer's own words, with a concrete example.\n- **Alternatives:** Email and spreadsheets. Any PMS or chain group module, if they have one. **Unknown.**\n- **Proposed change:** Tallyroom holds group bookings in one place. Describe only the features confirmed as shipped.\n- **Evidence:** 5 paying customers. Outcomes **Unknown**.\n- **Why act now:** **Unknown.** Do not claim urgency without a named event.\n- **Commercial proposal:** USD 400/month. Each deal needs a defined evaluation scope and success measure agreed with the GM before the proposal.\n\n### Economics: formulas to fill from interviews\n\nDo not add these together. Capacity and revenue are separate claims.\n\n| Benefit | Formula | Classification | Inputs needed |\n| --- | --- | --- | --- |\n| Coordination time | Hours per week saved x 52/12 x loaded hourly rate | Released capacity, not cash | Hours, role, rate, adoption |\n| Lost or slowed group revenue | Enquiries per month that are lost or slowed x expected uplift from faster response x average group booking value x margin | Incremental revenue, only once measured | Enquiry volume, conversion, average value, margin |\n\n- Record the source of every input. Estimates stay labelled as estimates.\n- Net against the USD 400/month fee and the onboarding effort.\n\n## 5. Discovery and demo\n\n### Before every call\n\n- Check the hotel's website for event, wedding, and conference space.\n- Confirm the GM and sales manager names.\n- Note room count. Use facts only; do not pitch before the call.\n\n### Customer interview (Experiment A)\n\n- \"Walk me through what happened from the moment a group enquiry arrived to the deposit being paid, the last time you did one.\"\n- \"What were you doing before? What did that cost in time, and in lost bookings?\"\n- \"What made you decide now? Who was involved? Who signed off?\"\n- \"What changed after you started? What would you be doing without it?\"\n- \"Who else in hotels like yours should I talk to?\"\n\n### First meeting with a prospect\n\n**Objective:** Agree on the current group enquiry process and one concrete recent failure. Not a demo.\n\n- **Opening:** \"Walk me through a group enquiry, from the first email to the deposit.\"\n- **If the current process works well:** Ask where it is slowest or where a group was ever lost. If there is no real gap, stop. Do not manufacture one.\n- **If it is manual:** Ask who does it, how often, and what happens when it is delayed.\n- **If the pain is real but low priority:** Ask what would change that, and whether a date is coming up.\n- **If the contact has no budget authority:** Ask how the GM decides on software, and what they would need to see.\n- **Concrete incident:** \"Tell me about the last group you nearly lost, or one that went wrong. What happened, and what did it cost?\"\n- **Summarize:** Repeat the problem back in their words and get confirmation before showing anything.\n- **Demo:** Show only the workflow that matches the problem they confirmed.\n- **Ask for:** A second meeting with the GM, or an evaluation with a defined success measure and a date. \"Sounds good\" is not a commitment.\n- **Objections:** Log each one verbatim in the tracker. Write responses from real calls, not in advance.\n\n### Pipeline clean-up (this week)\n\n- **9 \"follow up in Q1\":** Send each one a specific question tied to the reason they gave, and ask for a date now. If the reason was real timing, record the date and the condition.\n- **8 silent:** Send one short message: \"Did we miss what you needed, or is the timing off?\" No pitch. Log the reply, or \"no response\" as the outcome.\n\n## 6. Deal stages and exit criteria\n\n| Stage | Exit when | Next ask |\n| --- | --- | --- |\n| 1. Qualified | Section 3 criteria checked; GM or sales manager named | Discovery call |\n| 2. Discovery held | Current process described; one concrete incident or consequence recorded, or no-pain documented | Demo or second meeting |\n| 3. Demo held | Problem confirmed in their words | Evaluation or proposal, with a date |\n| 4. Evaluation | Scope and success measure agreed in writing; participants named | Review on the agreed date |\n| 5. Proposal sent | Price, start date, and decision-maker named | Decision date |\n| 6. Closed won | Signed and first payment received | Onboarding start |\n| 7. Live | Onboarding complete; baseline recorded | Outcome review date |\n| 8. Outcome validated | Customer confirms a measured result | Referral request |\n\n- **Closed lost:** A reason is required: no pain, timing, budget, chose another option, product gap, no response.\n- **Nurture:** Confirmed pain with wrong timing. A re-engagement date is required.\n- \"Interested\" is not a stage. Convert it to Nurture with a date, or to Qualified with a next action.\n\n## 7. Deal record\n\nUse `deal-tracker.csv` in this folder. Import it into Notion, Airtable, or a CRM when you are ready. Every open deal needs a dated next action. Stage changes happen only when the exit criteria are met. Stage dates are what let you diagnose where deals stall.\n\nSource values: `network`, `referral`, `association-list`, `outbound`, `inbound`.\n\n## 8. Review rhythm\n\n- **Weekly (30 minutes):** Check every open deal for a dated next action. Update stages. List the week's demos and interviews.\n- **2026-11-05 (4 weeks):** Review Experiments A and B. Decide whether to keep or narrow the profile in section 3.\n- **2026-12-03 (8 weeks):** Bottleneck diagnosis. Where are deals stalling?\n  - Fit: few accounts have the pain.\n  - Discovery: pain acknowledged, but no next step.\n  - Explanation: demo before the problem was confirmed.\n  - Buying process: GM not engaged.\n  - Product or onboarding: customers do not get value.\n  Also revise the goal in section 2.\n- **2026-12-21:** Last date a demo can start and plausibly close before the window ends, at a 7-week cycle.\n- **2027-02-08:** Four-month review.\n\n## 9. Handing selling to a first salesperson: not yet\n\nTransfer is not justified now. Repeated exchange outside your network is not shown, outcomes are not validated, and the motion is not yet written down. Hiring now would ask a new person to invent the motion.\n\nWhat the packet must contain before a first hire:\n\n- Profile confirmed from interviews, including the timing signal if one exists.\n- Discovery guide and branches taken from real calls.\n- Demo script built around the workflow customers value most.\n- Pricing, proposal template, and discount rules.\n- Objection responses from real calls.\n- Onboarding checklist and baseline template.\n- Tracker data covering at least two cohorts of deals.\n\nReadiness criteria (no numbers set yet):\n\n- You close deals from sources other than your network on a steady basis.\n- Some customers from those sources report measured outcomes.\n- You can say which stage each stalled deal stopped at.\n\nProfile for a first hire: Hotel sales or group-sales experience. Your buyer, the GM, and your user, the sales manager, both expect hospitality fluency. A seller comfortable with an early, unfinished product and a 7-week cycle matters more than a famous employer. You are the one who can pass on what makes the motion work, because you have done this from the hotel side.\n\n## 10. Review result\n\nChecked against the review criteria in the founding-sales checklist.\n\n- **Scope:** PASS. One packet. No outreach drafted, because each account needs verified facts first.\n- **Maturity:** PASS. Value exchange shown in network; value creation marked Unknown.\n- **Evidence integrity:** PASS. All figures are from your data and labelled.\n- **Next experiment:** PASS. Owners, dates, and exit conditions set.\n- **Customer profile:** PARTIAL. Profile is hypothesized until interviews confirm it.\n- **Timing:** PASS. Marked Unknown; no urgency claimed.\n- **Narrative:** PARTIAL. Consequences unknown until discovery.\n- **Economics:** PASS. Formulas only; no benefit figures invented.\n- **Discovery and demo:** PASS. Demo follows the confirmed problem.\n- **Commitment:** PASS. Asks are specific.\n- **Record quality:** PARTIAL. Tracker needs backfilling for the 22 existing deals.\n- **Customer success:** FAIL today. No outcome records. Experiment A addresses this.\n- **Diagnosis:** PARTIAL. Stage dates are needed before a bottleneck can be identified.\n- **Metrics:** PASS. Numerators, denominators, and a sample-size caveat included.\n- **Transfer readiness:** PASS. Transfer correctly deferred.\n- **Seller fit, coaching, balanced scale:** N/A until transfer is justified.\n\n**Immediate next action:** Send the 5 interview requests by 2026-10-15 (owner: you). Send the 8 \"silent\" and 9 \"Q1\" messages this week.\n"}]}, "haiku-skill-3": {"cost_usd": 0.02131, "seconds": 125, "output_tokens": 26986, "files": [{"name": "sales-process.md", "text": "# Tallyroom Sales Working Packet\n\nDraft dated 2026-10-08, built from notes.md. Evidence labels: **observed** (seen in records), **reported** (founder's account, not yet checked), **hypothesized** (to test), **unknown** (no data). Method basis: Kazanjy's public founding-sales method. Layouts, thresholds, and the 7-day hygiene rule are this packet's conventions.\n\n## 1. Assignment and evidence\n\n- **Requested deliverable:** A working sales process for a founder-only seller: qualification, discovery, value case, pipeline stages, records, and review. Plus a plan toward 15 new hotels by about 2027-02-08. Handing off to a first seller is deferred (section 9).\n- **Product and price:** Software for independent hotels with 20\u2013150 rooms to manage group bookings (weddings, conferences) that currently run on email and spreadsheets. USD 400/month. Unknown: which group-booking workflows are live today, whether price is flat, and whether annual terms exist.\n- **Available evidence:**\n\n| Fact | Label | Source |\n| --- | --- | --- |\n| 22 demos in 3 months, all from the founder's network | reported | founder |\n| 5 hotels bought at USD 400/month | reported | founder |\n| 9 said \"interested, follow up in Q1\" | reported | founder |\n| 8 went silent | reported | founder |\n| Average deal took 7 weeks | reported | founder |\n| GM usually decides; sales manager uses it | reported | founder |\n| Deals tracked in memory and a Notion page, no CRM | reported | founder |\n| Why the 5 bought and the others did not | unknown | \u2014 |\n| Whether the 5 customers achieve measurable results | unknown | \u2014 |\n| Conversion from prospects outside the network | unknown | \u2014 |\n\n- **Constraints:** Founder is the only seller. No CRM. 7-week average cycle. Target: 15 new hotels in 4 months.\n- **Missing inputs:**\n  - For the 5 customers: hotel size, first-contact date, what they did before buying, and who approved.\n  - For the 22 demos: the stage each reached, the date, and why it stopped.\n  - For the 9 Q1 accounts: what is happening in Q1 at each hotel.\n  - Any before-and-after numbers from the 5 customers.\n  - Whether the 5 wins share traits (room count, city, group volume, existing system, who runs events).\n\n## 2. Stage and next proof\n\n- **Current demonstrated stage:** Value exchange is partly shown: 5 hotels have paid (reported). Value creation is not shown: no measured result is recorded for any customer (unknown). Repeated exchange is not shown: all wins came through the founder's network, and outside conversion is unknown.\n- **Missing proof:** Value creation for the 5 paying customers. Until we know what changed for them and by how much, we cannot tell whether the sales reflect the product or the founder's relationships.\n- **Next experiment (Phase 1, 2026-10-08 to 2026-11-19):**\n  - **Owner: founder.** Enter all 22 demos into one database with the section 7 fields, by 2026-10-15. Mark each field as recorded or recalled.\n  - Interview the 5 paying customers, 30 minutes each, by 2026-10-22. Cover the trigger for buying, the previous process, the first booking in Tallyroom, any measured change, and who approved.\n  - Send the re-engagement note to the 9 Q1 accounts by 2026-10-15 (section 5).\n  - Send one reopen note to the 8 silent accounts. If there is no reply after two touches two weeks apart, close them with reason \"went silent.\"\n  - Start the small outside-network test batch (section 8) in parallel.\n- **Exit condition (review 2026-11-19):** All 5 customers interviewed. Each has a documented trigger and either a baseline with a measured change or a note that none exists. All 22 demos have a stage and reason code. At least one written hypothesis for why the 5 bought, checked against the 8 stalled accounts.\n- **Fallback:** If fewer than 2 of the 5 can name a measurable change, keep outside prospecting to the test batch. Set baselines with 2 willing customers before expanding. Scaling outreach around an unproven outcome would multiply the wrong message.\n\n## 3. Ideal customer and qualification\n\nHypotheses from the notes, to confirm against the 5 wins:\n\n- **Organization conditions:** Independent hotel, 20\u2013150 rooms, with recurring group business handled by email and spreadsheets. Confirm: room count, group bookings per month, city, existing systems, and who runs events.\n- **Exclusions (hypothesis):** Hotels with little group business; hotels where a third party handles all group sales; hotels where group requests already run in a system that works. Test these against the 8 silent accounts before adopting them.\n- **People:**\n  - GM: economic buyer (reported).\n  - Sales manager: daily user and probable champion (reported; confirm).\n  - Who first raised the problem in each win: unknown.\n- **Current approach:** Email and spreadsheets (reported).\n- **Timing:** Unknown. Check whether the 5 wins had a trigger, such as a bad booking, a staff change, a season, or a new group offering. Do not treat the 9 Q1 accounts as timed to a real event until they name one.\n\n| Decision | When |\n| --- | --- |\n| Pursue | Meets organization conditions; user or champion acknowledges a current gap; GM reachable |\n| Discover | Fits conditions; gap unclear; one discovery call to find out |\n| Defer | Real gap but no decision timing; record a dated trigger and re-engagement date |\n| Exclude | Outside conditions, or no meaningful group business |\n\n## 4. Master narrative and economics\n\n- **Current state (hypothesis):** Group inquiries arrive by email, get tracked in spreadsheets, and proposals and holds are coordinated by hand between sales and events staff.\n- **Problem and consequences (hypothesis, test in discovery):** Slow replies to inquiries, unconfirmed holds, errors in terms, and time spent coordinating. Personal stakes: the sales manager's workload and reliability; the GM's group revenue and visibility into it.\n- **Alternatives and gap:** Email and spreadsheets (reported). Gap (hypothesis): no single record per group booking and no pipeline view.\n- **Proposed change:** One shared record per group booking, from first inquiry to contract, replacing email threads and spreadsheets. Confirm this matches what is live.\n- **Evidence:** 5 paying customers (reported). No measured result yet. Make no outcome claims until the section 2 interviews are done.\n- **Why act now:** Unknown. Test against the Q1 accounts and the GM's planning calendar.\n- **Commercial proposal:** USD 400/month. Next decision: a demo with the GM and sales manager, then a trial or purchase.\n\n| Business measure | Baseline and source | Proposed change | Calculation and period | Costs and assumptions | Validation plan |\n| --- | --- | --- | --- | --- | --- |\n| Staff hours on group inquiries per month | Unknown; ask in interviews | Hypothesized reduction | Hours saved \u00d7 loaded hourly rate = capacity value, not cash | Onboarding time; staff adoption | Customer estimate at 30 days, checked against interview notes |\n| Group bookings won per month | Unknown; hotel's own count | Hypothesized increase from faster replies | Extra bookings \u00d7 average group contribution, per month | Count only if the hotel can attribute the wins to faster replies | Compare 3 months before and after |\n| Break-even | \u2014 | \u2014 | $400 \u00f7 loaded hourly rate = hours per month needed to cover the fee (e.g., $25/hour \u2192 16 hours/month) | \u2014 | Use in calls to test whether the saved time is plausible |\n\nClassify each benefit as cash savings, released capacity, incremental revenue, or qualitative. Do not count the same hours as both time saved and extra revenue.\n\n## 5. Outreach drafts\n\nVerify every personalized detail before sending.\n\n**Outside-network first message:**\n\n> Subject: Group inquiries at [Hotel]\n>\n> Hi [Name], I used to run revenue at hotels, and I'm curious how [Hotel] handles group inquiries for weddings and conferences today. Is it mostly email and spreadsheets? We help independent hotels with 20\u2013150 rooms keep each group booking in one record. Would a 20-minute call about your current process be useful? [Verified detail about the hotel.]\n\n**Re-engagement for the 9 \"Q1\" accounts:**\n\n> Subject: Following up on Q1\n>\n> Hi [Name], when we spoke in [month], you said following up in Q1 made sense. What is changing at [Hotel] in Q1 on group bookings? If it's still a priority, I can walk you through [workflow they discussed] in 20 minutes. If the timing has moved, tell me when and I'll check back then.\n\n**Reopen for the 8 silent accounts:**\n\n> Hi [Name], are group inquiries at [Hotel] still handled by email and spreadsheets, or has something changed? Happy to share what we've learned since we last spoke.\n\n## 6. Discovery and commitment plan\n\n- **Pre-call:** Check the hotel's website for room count, event spaces, and group offer. Note sources. Do not spend discovery time on facts that are public.\n- **Meeting objective:** Confirm the current group process and one recent failure. Agree the next step: a demo with the GM or sales manager.\n- **Opening question:** Walk me through the last group booking you took, from first inquiry to signed contract. Where did the details live at each step?\n- **Branches:**\n  - *Current process works:* Where does it fall short, if anywhere? If nothing, stop and mark Exclude or Defer.\n  - *Manual workaround:* Who does it, how often, and what happens when it's late or missed?\n  - *Pain, low priority:* What would change its priority? Is an event coming?\n  - *Champion without budget:* Who approves spend, and what would they need to see?\n- **Concrete incident:** Tell me about the last group booking that went wrong or nearly did. What happened, who noticed, and what did it cost in time or revenue?\n- **Problem summary:** Before showing the product, restate the problem in the hotel's words and ask them to confirm it.\n- **Demo selection:** One real group inquiry from their own process, from first request to proposal. Not a feature tour.\n- **Objection response:** Ask what would make them skeptical. Answer with a fact you have, or say you don't know yet.\n- **Commitment request by stage:**\n  - After discovery: a dated demo with the GM or sales manager.\n  - After demo: a trial with a named user and start date, or a proposal review with the GM.\n  - After proposal: signed order, payment set up, and a named onboarding owner.\n- **If no commitment:** Record a reason code and a dated re-engagement trigger, or close the deal.\n\n## 7. Pipeline and records\n\n**Stages and exit criteria**\n\n| Stage | Exit criterion (evidence required) |\n| --- | --- |\n| 1. Lead | Account and contact identified; fit not yet checked |\n| 2. Qualified | Meets section 3 conditions; contact's role known |\n| 3. Discovery held | Current process written up; gap confirmed or explicitly absent |\n| 4. Demo held | Demo covered the confirmed workflow; dated next step agreed in writing |\n| 5. Trial or proposal | Trial start date set, or proposal sent with the GM named as approver |\n| 6. Won | Signed; payment set up; onboarding owner named |\n| 7. Live | First real group booking run in Tallyroom; baseline recorded; first-value check scheduled |\n| Deferred | Real gap, no decision timing; dated trigger and re-engagement date |\n| Lost | Reason code recorded |\n\n**Required fields** (one database; Notion works if it is a database, not a page):\n- Account, city, room count, group types, and source (network, outbound, or inbound)\n- Contacts with roles (GM, sales manager, other) and whether the decision maker is confirmed\n- Stage and date entered\n- Acknowledged pain, in the buyer's words, and current approach\n- Next action, owner, and due date (required for every open deal)\n- Monthly value, start date, and objections\n- Reason code (lost, deferred, or stalled)\n- After winning: baseline measure and first-value date\n\n**Reason codes:** no pain acknowledged; pain but low priority; no decision maker reached; chose current approach; timing later (with date); went silent; price; product gap; other.\n\n**Hygiene:** Every open deal has a dated next action. A deal with no action for 7 days gets reviewed. Deferred deals leave the active pipeline count.\n\n## 8. Motion review and goal math\n\n**Weekly metrics:** new leads, qualified, discovery held, demos held, next steps agreed, trials or proposals, won, live. Report each conversion with its numerator, denominator, source, and period.\n\n**Goal math** (15 new wins, window ends about 2027-02-08):\n\n- Current close rates: 5 of 13 resolved demos (38%); 5 of 22 demos including open ones (23%). The 9 open demos are unresolved, so the 23% figure is a floor, not a settled rate.\n- With a 7-week average cycle, a deal that starts after about mid-December is unlikely to close inside the window.\n\n| Demo-to-win conversion | Demos needed for 15 wins |\n| --- | --- |\n| 38% (resolved, network) | about 40 |\n| 23% (all demos, network) | about 65 |\n| 15% (illustrative, outside network) | about 100 |\n\n- Current pace: 22 demos in about 13 weeks, or about 1.7 per week. Reaching the table's range in the roughly 10 weeks before mid-December means about 4 to 10 demos per week.\n- The 9 open accounts and the 8 silent ones add a finite pool. They may contribute some wins, but the network cannot supply repeated new starts.\n\n**Conclusion:** 15 in this window needs either several times the current demo pace or higher conversion than the network has shown. Treat 15 as a stretch target, and decide on 2026-11-19 whether to keep it, using Phase 1 results and first outside-network conversion data.\n\n**Review cadence:**\n- **2026-11-19 (about 6 weeks):** Phase 1 exit review. Decide the target and how far to scale outbound.\n- **2026-12-17 (about 10 weeks):** First full-cycle read. Check whether the pipeline can still deliver in the window.\n- **2027-01-14:** Final pipeline review before the window closes.\n\n**Bottleneck hypotheses** (method diagnostics, applied to Tallyroom):\n- *Most prospects lack the pain:* targeting problem. Check outside-network discovery answers.\n- *Pain acknowledged but no next step:* discovery, explanation, or buying process. Check whether the GM was in the call.\n- *Customers sign but do not use it:* product or onboarding gap. Check first-booking dates and usage.\n- *Customers get results but sales are weak:* messaging gap. Ask the 5 customers how they would describe the result to a peer.\n\n## 9. Transfer plan (premature)\n\nNot justified yet. No non-network wins are documented, no customer outcome is measured, and this process has not been tested.\n\n**Readiness criteria before hiring:**\n- Discovery approach, demo path, pricing, and objection responses, updated from real calls.\n- The founder can name the top reasons deals win and stall, backed by the records.\n- Wins from outside the network, each with a documented trigger and baseline.\n- Customers reporting a measured change.\n\n**Seller profile when the time comes (hypothesis):** Comfortable selling an early product with incomplete resources. Familiar with hotel or events buyers. Able to run a 7-week cycle at USD 400/month, which implies high prospect volume. The founder must reserve coaching time for the first hire.\n\n## 10. Review result\n\n- **Scope:** PASS. Sales process only; transfer deferred with reasons.\n- **Maturity:** PASS. Earliest missing proof named.\n- **Evidence integrity:** PASS. Every fact labeled.\n- **Next experiment:** PASS. Owner, action, dates, and exit condition set.\n- **Customer profile:** FAIL until the 5 wins and 8 silent accounts are analyzed. Remains a hypothesis.\n- **Timing:** FAIL until a real trigger is confirmed.\n- **Narrative:** PASS as a hypothesis draft. Needs interview validation.\n- **Economics:** FAIL until inputs are collected. Formulas are ready.\n- **Discovery and solution framing:** PASS.\n- **Commitment:** PASS. Requested next steps are stage-specific.\n- **Record quality:** FAIL until the 22 demos are in a database.\n- **Customer success:** FAIL. No baseline or measured outcome yet.\n- **Diagnosis:** PASS as hypotheses. No conclusions drawn yet.\n- **Metrics:** PASS. Denominators retained; sample too small to show causality.\n- **Transfer, seller fit, coaching, balanced scale:** N/A, premature (section 9).\n- **Attribution:** PASS.\n\n**Remaining uncertainties:** why the 5 bought; whether they get value; outside-network conversion; whether the 9 Q1 accounts have real timing; price structure.\n\n**Immediate next action:** Founder enters the 22 demos into a database with the section 7 fields and sends the 9 Q1 notes, both by 2026-10-15. Expected evidence: 22 complete records and 9 sent messages.\n"}]}, "opus-1": {"cost_usd": 0.27974, "seconds": 120, "output_tokens": 9246, "files": [{"name": "4-month-plan.md", "text": "# 4-Month Plan: 15 New Hotels by early February 2027\n\nWindow: **12 Oct 2026 \u2013 5 Feb 2027** (17 weeks). Goal: 15 new hotels \u00d7 USD 400 = **USD 6,000 more MRR** (from USD 2,000 today to USD 8,000).\n\n---\n\n## 1. The math (read this first)\n\nWhat you've done so far: 22 demos \u2192 5 wins = **23% demo-to-win**, about **7 weeks** from first call to signature.\n\nTwo things follow from that:\n\n1. **A 7-week cycle means new demos must happen by about mid-December** to close inside the window. Anything first demoed after that probably closes after February. So **Oct\u2013Dec is for filling the pipeline, Jan\u2013early Feb is for closing it.**\n2. **The existing 17 open deals won't get you there by themselves.** A realistic guess:\n\n| Source | Deals | Assumed win rate | Expected wins |\n|---|---|---|---|\n| \"Follow up in Q1\" | 9 | ~30% (some will be polite no's) | ~3 |\n| Went silent | 8 | ~10% | ~1 |\n| **New demos needed** | **~48** | 23% | **~11** |\n| **Total** | | | **~15** |\n\n**~48 new demos in ~10 weeks = about 5 demos a week**, alongside running the company.\n\n**Be honest with yourself about the risk:** the 23% rate came from your personal network. Cold or referred leads usually close at a lower rate. At 15% you'd need ~73 demos (7/week). Treat **15 as the stretch target and 10\u201312 as the realistic one** unless the first month's numbers say otherwise. Check this at the end of each month (section 4).\n\n---\n\n## 2. Where ~48 new demos come from\n\nRoughly, warm intros turn into a demo about half the time; cold outreach is closer to 1 in 10. Rough target mix:\n\n| Source | Outreach needed | \u2192 Demos |\n|---|---|---|\n| **Referrals from your 5 customers** (ask each for 2\u20133 GM intros) | 10\u201315 intros | ~6 |\n| **Your remaining hotel network** (ex-colleagues, revenue managers, GMs you know) | 30\u201340 | ~15 |\n| **Second-degree intros** (ask every demo, won or lost: \"who else has this problem?\") | 20\u201330 | ~10 |\n| **Targeted cold outreach** to independent hotels with wedding/event pages in your best region | 150\u2013200 | ~15 |\n| **Hotel associations / local events** (one talk or table) | \u2014 | a few, bonus |\n\nThe 5 current customers matter most: they are your references, your referrals and your source of \"why we win\".\n\n---\n\n## 3. Weekly targets\n\n| Weeks | Dates | Focus | New outreach / wk | Demos / wk | Proposals / wk | Wins (cumulative) |\n|---|---|---|---|---|---|---|\n| 1\u20132 | 12\u201323 Oct | Set up, win/loss calls, reactivate old deals | 15 | 2\u20133 | 1 | 0\u20131 |\n| 3\u20136 | 26 Oct \u2013 20 Nov | Fill the pipeline | 40 | 5 | 1\u20132 | 3 |\n| 7\u201310 | 23 Nov \u2013 18 Dec | Last demos that can close in the window | 40 | 5 | 2\u20133 | 6 |\n| 11\u201312 | 21 Dec \u2013 1 Jan | Holidays: light touch only | 0\u201310 | 0\u20131 | 0\u20131 | 6\u20137 |\n| 13\u201317 | 4 Jan \u2013 5 Feb | Close: proposals and the Q1 nurture deals | 15 | 2\u20133 | 3\u20134 | **15** |\n\nHotels plan next year's budgets in Q4, and wedding/conference season planning starts in winter. Use both as the \"why now\" in your outreach.\n\n---\n\n## 4. Monthly checkpoints\n\nAt the end of each month, compare against this and adjust.\n\n| Checkpoint | On track if\u2026 | If behind\u2026 |\n|---|---|---|\n| **6 Nov** | Win/loss calls done, all 17 old deals sorted (live / Nurture with date / Lost), 12+ demos booked or done | Lean harder on customer referrals; cut cold outreach that isn't producing replies |\n| **4 Dec** | ~30 new demos done, 4+ wins, demo-to-win still \u226520% | Lower the target to 10\u201312 now rather than in February; look for the stage where deals stall |\n| **8 Jan** | 7+ wins, 10+ deals in Evaluation/Proposal | Focus only on deals where the GM is already involved |\n| **5 Feb** | 15 wins | Write down what blocked it. That goes into the playbook and the hiring decision |\n\n---\n\n## 5. The first two weeks, step by step\n\n**Week 1 (12\u201316 Oct)**\n- [ ] Import `pipeline.csv` into Notion. Put all 22 past deals in it, with whatever you remember (it doesn't need to be perfect).\n- [ ] Book 20-minute calls with all 5 customers (see `win-loss-interviews.md`). End each one by asking for 2\u20133 intros.\n- [ ] Email the 8 silent deals with the \"close the file\" template from the playbook. Their replies tell you who is still alive.\n- [ ] Start recording your demos (with permission).\n\n**Week 2 (19\u201323 Oct)**\n- [ ] Contact the 9 \"Q1\" deals now, not in January: confirm a specific date and reason, and try to get the GM into a call before the holidays. A Q1 decision needs Q4 groundwork.\n- [ ] Try 2\u20133 short loss calls with silent or lost prospects who'll talk.\n- [ ] Write down \"why we win\" from the customer calls and update section 4 of the playbook.\n- [ ] Build a list of 100 target hotels for cold outreach in the region where your customers are.\n"}, {"name": "pipeline.csv", "text": "Hotel,Rooms,City/Region,Source,GM name,Sales/events manager,Stage,First contact,Last contact,Next step,Next step date,Pain (their words),Groups per month,Win/loss reason,Notes\n[Customer 1],,,,,,Won,,,Win/loss call + ask for 2-3 intros,2026-10-16,,,,\n[Customer 2],,,,,,Won,,,Win/loss call + ask for 2-3 intros,2026-10-16,,,,\n[Customer 3],,,,,,Won,,,Win/loss call + ask for 2-3 intros,2026-10-16,,,,\n[Customer 4],,,,,,Won,,,Win/loss call + ask for 2-3 intros,2026-10-16,,,,\n[Customer 5],,,,,,Won,,,Win/loss call + ask for 2-3 intros,2026-10-16,,,,\n[Q1 follow-up 1],,,,,,Nurture,,,Confirm specific date & reason; get GM on a call before holidays,2026-10-23,,,,\"Said 'interested, follow up in Q1'\"\n[Q1 follow-up 2],,,,,,Nurture,,,Confirm specific date & reason; get GM on a call before holidays,2026-10-23,,,,\"Said 'interested, follow up in Q1'\"\n[Q1 follow-up 3],,,,,,Nurture,,,Confirm specific date & reason; get GM on a call before holidays,2026-10-23,,,,\"Said 'interested, follow up in Q1'\"\n[Q1 follow-up 4],,,,,,Nurture,,,Confirm specific date & reason; get GM on a call before holidays,2026-10-23,,,,\"Said 'interested, follow up in Q1'\"\n[Q1 follow-up 5],,,,,,Nurture,,,Confirm specific date & reason; get GM on a call before holidays,2026-10-23,,,,\"Said 'interested, follow up in Q1'\"\n[Q1 follow-up 6],,,,,,Nurture,,,Confirm specific date & reason; get GM on a call before holidays,2026-10-23,,,,\"Said 'interested, follow up in Q1'\"\n[Q1 follow-up 7],,,,,,Nurture,,,Confirm specific date & reason; get GM on a call before holidays,2026-10-23,,,,\"Said 'interested, follow up in Q1'\"\n[Q1 follow-up 8],,,,,,Nurture,,,Confirm specific date & reason; get GM on a call before holidays,2026-10-23,,,,\"Said 'interested, follow up in Q1'\"\n[Q1 follow-up 9],,,,,,Nurture,,,Confirm specific date & reason; get GM on a call before holidays,2026-10-23,,,,\"Said 'interested, follow up in Q1'\"\n[Went silent 1],,,,,,Nurture,,,Send 'close the file' email; move to Lost if no reply in 1 week,2026-10-16,,,,Went silent after demo\n[Went silent 2],,,,,,Nurture,,,Send 'close the file' email; move to Lost if no reply in 1 week,2026-10-16,,,,Went silent after demo\n[Went silent 3],,,,,,Nurture,,,Send 'close the file' email; move to Lost if no reply in 1 week,2026-10-16,,,,Went silent after demo\n[Went silent 4],,,,,,Nurture,,,Send 'close the file' email; move to Lost if no reply in 1 week,2026-10-16,,,,Went silent after demo\n[Went silent 5],,,,,,Nurture,,,Send 'close the file' email; move to Lost if no reply in 1 week,2026-10-16,,,,Went silent after demo\n[Went silent 6],,,,,,Nurture,,,Send 'close the file' email; move to Lost if no reply in 1 week,2026-10-16,,,,Went silent after demo\n[Went silent 7],,,,,,Nurture,,,Send 'close the file' email; move to Lost if no reply in 1 week,2026-10-16,,,,Went silent after demo\n[Went silent 8],,,,,,Nurture,,,Send 'close the file' email; move to Lost if no reply in 1 week,2026-10-16,,,,Went silent after demo\n"}, {"name": "sales-playbook.md", "text": "# Tallyroom Sales Playbook\n\nHow we sell, written so a founder can run it now and a first salesperson can take it over later.\nVersion 1, October 2026. **Update it every month** with what you learn. Sections marked *[VALIDATE]* are my best guesses until the win/loss calls (`win-loss-interviews.md`) confirm or correct them.\n\n---\n\n## 1. Who we sell to (ideal customer profile)\n\n| | Good fit | Poor fit |\n|---|---|---|\n| Hotel | Independent, 20\u2013150 rooms | Brand-managed or chain (group tools come from HQ) |\n| Group business | Hosts weddings, conferences, retreats or sports teams **regularly** (rough bar: 2+ groups a month) *[VALIDATE]* | Groups are rare or one-offs |\n| Current setup | Email + spreadsheets, and it hurts: double-booked room blocks, missed cut-off dates, rooming list chaos | Already happy with a sales and catering tool |\n| People | A GM who can sign, plus a sales or events manager who owns groups | No one owns group sales |\n\n**Two people are always involved:**\n- **General manager (GM)**: decides and pays. Cares about revenue, fewer mistakes and staff time.\n- **Sales/events manager**: uses it every day. Cares about less admin and not losing a booking. They usually feel the pain first and become your inside champion.\n\nA deal that has only one of these two involved is at risk. Get both in front of you before the proposal.\n\n---\n\n## 2. Pipeline stages\n\nEvery deal is in exactly one stage. A deal moves on only when it meets the **exit criteria**, not when it \"feels\" ahead.\n\n| # | Stage | What happens | Exit criteria (all must be true to move on) |\n|---|---|---|---|\n| 1 | **Lead** | You know the hotel exists and it may fit | Fits the profile in section 1 |\n| 2 | **Contacted** | You reached out | They replied and agreed to a call |\n| 3 | **Discovery / demo** | First call: their process, their pain, then the demo | Pain confirmed in their own words, GM identified, next meeting **in the calendar** |\n| 4 | **Evaluation** | Both GM and sales manager have seen it; questions, references, trial if any | GM has seen it, the user wants it, no open blocker |\n| 5 | **Proposal** | Price and start date sent | Verbal yes from the GM, start date agreed |\n| 6 | **Won** | Signed / first payment | \u2014 |\n| \u2014 | **Lost** | Said no, or chose something else | Reason recorded (section 7) |\n| \u2014 | **Nurture** | Real fit but real timing reason (e.g. \"budget opens January\") | A **specific** date and reason to come back. \"Q1\" alone is not enough |\n\n**Rule:** a deal with no next step in the calendar is not a live deal. After every call, book the next one before you hang up.\n\n**Rule:** if a deal hasn't moved in 3 weeks, it goes to Nurture or Lost. No exceptions. That keeps your forecast honest.\n\n---\n\n## 3. Qualification: what to learn on the first call\n\nAsk these before you demo, not after. If you can't answer most of them, the demo will be generic and the deal will drift.\n\n1. **Volume:** \"How many groups did you host last year? What share of revenue was that?\"\n2. **Current process:** \"Walk me through the last wedding block you took, from first email to check-in.\"\n3. **Pain:** \"What went wrong the last time a group went badly?\" Get a specific story and a cost (lost rooms, comped nights, staff hours).\n4. **People:** \"Who handles groups day to day? Who signs off on new software?\"\n5. **Timing:** \"Is there a reason to fix this now, like a busy wedding season, a new sales hire or a budget cycle?\"\n6. **Alternatives:** \"What else have you looked at? What would you do if you didn't buy anything?\"\n7. **Process to buy:** \"If you liked it, what happens next? Does anyone else need to approve it?\"\n\nWrite the answers into the pipeline tracker (`pipeline.csv`) on the same day.\n\n---\n\n## 4. Demo structure (30\u201340 min)\n\n1. **Recap their pain (3 min).** \"Last time you said X. Is that still the main issue?\"\n2. **Show their story, not a feature tour (15 min).** Rebuild the exact group they described in question 2: create the block, track the pickup, handle the cut-off, send the rooming list. Use their hotel name and room types if you can.\n3. **Your credibility (2 min).** You've done revenue management in hotels. Say it once, briefly.\n4. **Proof (3 min).** One sentence on what a similar customer gets out of it. *[VALIDATE: fill this in after the win/loss calls]*\n5. **Price and next step (5 min).** Say the price (USD 400/month) on the call. Book the next meeting with the person who wasn't there (GM or sales manager).\n\n---\n\n## 5. Follow-up rules\n\n| When | Do this |\n|---|---|\n| Same day as the call | Short recap email: their pain, what you showed, the agreed next step and date |\n| Next step missed | Follow up within 2 working days with one specific question |\n| No reply after 3 attempts over 2 weeks | Send a \"close the file\" email (template below), then move to Nurture or Lost |\n| Nurture date arrives | Reach out with something new: a feature, a customer story, their season |\n\n### Templates\n\n**Recap after a call**\n> Hi [Name], thanks for the time today. My notes: today groups are handled through [process], and the biggest headache is [their pain in their words]. I showed you how Tallyroom would handle [their example]. Next step: [meeting] on [date] with [person]. Anything I got wrong?\n\n**Close the file (for deals that went quiet)**\n> Hi [Name], I haven't heard back, so I'm guessing group bookings aren't a priority right now. Fair enough. I'll close this on my side. If that changes, or if I misread it, just reply and we'll pick it up.\n\n**Nurture check-in (for the \"follow up in Q1\" deals)**\n> Hi [Name], when we spoke in [month] you mentioned [timing reason]. Before [their season/budget date], is this worth a 20-minute look at how [specific pain] would work for you? I've also added [new thing] since we last spoke.\n\n---\n\n## 6. What to record for every deal\n\nUse `pipeline.csv` (it imports straight into Notion as a database, or into any CRM later). The minimum for every deal:\n\n- Hotel, rooms, city, source (who introduced you)\n- GM name and sales manager name\n- Stage, date of last contact, **next step and its date**\n- Pain in their words\n- For won/lost: the reason (section 7)\n\nIf it isn't written down, it doesn't exist. This tracker is what a future salesperson inherits.\n\n---\n\n## 7. Win/loss reasons (use these exact labels)\n\nUsing fixed labels lets you count them later and answer \"why do we win?\"\n\n**Won because:** `pain-acute`, `founder-trust`, `price-ok`, `sales-mgr-champion`, `season-urgency`, `other`\n**Lost because:** `no-pain`, `no-budget`, `gm-not-involved`, `timing`, `chose-competitor`, `stayed-with-spreadsheets`, `went-silent`, `bad-fit`, `other`\n\nReview the counts once a month. Change the playbook based on what they show.\n\n---\n\n## 8. Weekly routine (about 1 hour, every Monday)\n\n1. Update every open deal in the tracker. Any deal with no next date gets one, or moves to Nurture/Lost.\n2. Check your numbers against the targets in `4-month-plan.md`: new leads, demos, proposals, wins.\n3. Pick the 3 deals that most need you this week.\n4. Book this week's prospecting time in the calendar.\n\n---\n\n## 9. Handing over to a first salesperson\n\nHire once the following are true (aim for month 4\u20136):\n- You have about 20+ customers and a clear \"why we win\" (section 7 data).\n- The demo in section 4 works when somebody else runs it. Test this by having a friend run it from this document.\n- The tracker is complete and current.\n\n**What they take over first:** prospecting, first calls and demos for new leads.\n**What you keep for now:** closing calls with GMs, pricing exceptions, and customers who came through your personal network.\n**What they inherit:** this playbook, the tracker, call recordings of your best demos (start recording now, with permission), and the email templates.\n"}, {"name": "win-loss-interviews.md", "text": "# Win/Loss Interviews\n\nYou don't know why 5 hotels bought and 17 didn't. These calls are the fastest way to find out. Do them in the first two weeks: everything else in the playbook gets better once you know.\n\n## Who to call\n- **All 5 customers.** Talk to the GM if you can, and separately to the sales manager who uses it. They often bought for different reasons.\n- **3\u20135 of the 17 that didn't buy.** Choose a mix of \"Q1\" and silent deals, picking people who know you well enough to be honest.\n\n## How to run it\n- 20 minutes, by phone or video. Say clearly: \"This isn't a sales call. I want to learn.\"\n- Ask, then stay quiet. Write down their exact words. Their phrases become your outreach copy.\n- Ask \"why?\" or \"tell me more\" at least once per answer.\n\n## Questions for customers\n1. What was going on at the hotel when you first heard about Tallyroom?\n2. What were you doing for groups before? What was the worst part?\n3. What made you decide to actually do something about it, and why then?\n4. Did you look at anything else, or consider just keeping the spreadsheets?\n5. Who was involved in the decision? Was anyone unsure? What convinced them?\n6. How much did it matter that I used to be a revenue manager?\n7. What almost stopped you from buying?\n8. What has changed since you started using it? Any numbers, like hours saved or blocks not lost?\n9. How would you describe Tallyroom to another GM in one sentence?\n10. **Who are 2\u20133 other hotels that have this problem? Would you introduce me?**\n\n## Questions for prospects who didn't buy\n1. When we spoke, what was the main reason you took the call?\n2. What did you think after the demo?\n3. What would have had to be true for you to go ahead?\n4. Was the GM / owner involved? What did they think?\n5. How are you handling group bookings today? Is it painful enough to fix?\n6. Was price a factor? (Ask directly. They usually won't bring it up.)\n7. Is there a point where it would make sense to look again?\n\n## Record the results\n\n| Hotel | Won/Lost | Who you spoke to | Main reason (label from playbook \u00a77) | Exact quote worth keeping | Intros given |\n|---|---|---|---|---|---|\n| | | | | | |\n| | | | | | |\n| | | | | | |\n| | | | | | |\n| | | | | | |\n\n## After all the calls, answer these\n- What do the 5 customers have in common (size, group volume, region, who championed it, the trigger)? **That's your real customer profile.** Update the playbook \u00a71.\n- What was the trigger to buy? **That's your \"why now\" message.**\n- What's the one sentence customers used to describe Tallyroom? **That's your pitch.**\n- Where did lost deals stall: no pain, GM never involved, timing, or price? **That's the stage to fix first.**\n"}]}, "opus-2": {"cost_usd": 0.23659, "seconds": 104, "output_tokens": 7933, "files": [{"name": "pipeline.csv", "text": "hotel,city,rooms,group_mix,contact_user,contact_gm,gm_involved,source,current_tool,stage,first_demo_date,next_step,next_step_date,mrr_usd,win_loss_reason,notes\nCustomer 1 (fill in),,,,,,,network,,Won,,Win interview + ask for 2 intros,,400,,\nCustomer 2 (fill in),,,,,,,network,,Won,,Win interview + ask for 2 intros,,400,,\nCustomer 3 (fill in),,,,,,,network,,Won,,Win interview + ask for 2 intros,,400,,\nCustomer 4 (fill in),,,,,,,network,,Won,,Win interview + ask for 2 intros,,400,,\nCustomer 5 (fill in),,,,,,,network,,Won,,Win interview + ask for 2 intros,,400,,\nInterested 1 (fill in),,,,,,,network,,Nurture,,Send pre-budget email (scripts.md \u00a73),,400,,said follow up in Q1\nInterested 2 (fill in),,,,,,,network,,Nurture,,Send pre-budget email (scripts.md \u00a73),,400,,said follow up in Q1\nInterested 3 (fill in),,,,,,,network,,Nurture,,Send pre-budget email (scripts.md \u00a73),,400,,said follow up in Q1\nInterested 4 (fill in),,,,,,,network,,Nurture,,Send pre-budget email (scripts.md \u00a73),,400,,said follow up in Q1\nInterested 5 (fill in),,,,,,,network,,Nurture,,Send pre-budget email (scripts.md \u00a73),,400,,said follow up in Q1\nInterested 6 (fill in),,,,,,,network,,Nurture,,Send pre-budget email (scripts.md \u00a73),,400,,said follow up in Q1\nInterested 7 (fill in),,,,,,,network,,Nurture,,Send pre-budget email (scripts.md \u00a73),,400,,said follow up in Q1\nInterested 8 (fill in),,,,,,,network,,Nurture,,Send pre-budget email (scripts.md \u00a73),,400,,said follow up in Q1\nInterested 9 (fill in),,,,,,,network,,Nurture,,Send pre-budget email (scripts.md \u00a73),,400,,said follow up in Q1\nSilent 1 (fill in),,,,,,,network,,Demo done,,Send breakup email (scripts.md \u00a74),,400,,went silent\nSilent 2 (fill in),,,,,,,network,,Demo done,,Send breakup email (scripts.md \u00a74),,400,,went silent\nSilent 3 (fill in),,,,,,,network,,Demo done,,Send breakup email (scripts.md \u00a74),,400,,went silent\nSilent 4 (fill in),,,,,,,network,,Demo done,,Send breakup email (scripts.md \u00a74),,400,,went silent\nSilent 5 (fill in),,,,,,,network,,Demo done,,Send breakup email (scripts.md \u00a74),,400,,went silent\nSilent 6 (fill in),,,,,,,network,,Demo done,,Send breakup email (scripts.md \u00a74),,400,,went silent\nSilent 7 (fill in),,,,,,,network,,Demo done,,Send breakup email (scripts.md \u00a74),,400,,went silent\nSilent 8 (fill in),,,,,,,network,,Demo done,,Send breakup email (scripts.md \u00a74),,400,,went silent\n"}, {"name": "sales-playbook.md", "text": "# Tallyroom Sales Playbook (v1, Oct 2026)\n\nOwner: CEO (only seller for now). This is written so it can be handed to a first salesperson later. Anything marked **[ASSUMPTION]** is a guess to check against your real deals.\n\nFiles that go with this:\n- `pipeline.csv` is the tracker. It replaces \"in my head + Notion\". It already has one row for each of the 22 demos, ready for you to fill in.\n- `scripts.md` has the word-for-word emails and call questions.\n\n---\n\n## 1. The math: what 15 hotels in 4 months takes\n\n| Input | Number | Source |\n|---|---|---|\n| Demo \u2192 close rate | 23% (5 of 22) | your notes |\n| Sales cycle | ~7 weeks | your notes |\n| Target | 15 new hotels by ~8 Feb 2027 | your notes |\n| New MRR at target | 15 \u00d7 $400 = **$6,000/mo** | |\n\n**The 7-week cycle is the real constraint.** A deal needs its first demo by about **mid-December 2026** to close by early February. That gives you roughly **10 weeks of demo time, not 17.** Hotels also go quiet over the holidays **[ASSUMPTION]**, so treat 20 Dec\u20135 Jan as close to dead time.\n\nWhere the 15 could come from:\n\n| Source | Count | Assumed rate | Expected closes |\n|---|---|---|---|\n| 9 \"follow up in Q1\" | 9 | ~30% (they're already warm) | ~3 |\n| 8 gone silent | 8 | ~10% | ~1 |\n| **New demos needed** | **~48** | 23% | ~11 |\n| **Total** | | | **~15** |\n\n48 new demos in 10 weeks is about **5 demos a week**. You did 22 in 3 months (~2/week), all from your network. So:\n1. Your network is probably close to used up. You need a repeatable source of new demos (see \u00a73), and that's the biggest gap.\n2. A higher close rate is the cheapest lever. Going from 23% to 33% drops the new demos needed from ~48 to ~33. Qualification (\u00a75) and multi-threading the GM (\u00a76) are how you get there.\n3. A shorter cycle is the second lever. Going from 7 weeks to 5 opens up 2 more weeks of usable demo time.\n\nCheck this every week with the numbers in \u00a79. If you're under 4 demos a week by mid-November, the target is at risk. Change the plan then instead of finding out in February.\n\n---\n\n## 2. Do this first (weeks 1\u20132)\n\n1. **Backfill `pipeline.csv`** for all 22 demos (about 2 hours). Fill in what you remember: hotel size, type (wedding/conference/mixed), who was on the call, what tool they use today, and the reason they gave.\n2. **Run win interviews with all 5 customers** (`scripts.md` \u00a71). This answers \"I don't know why the 5 bought.\" Look for what they have in common: size, group mix, who championed it, what pain made them act, what tool they replaced.\n3. **Compare winners and losers in the CSV.** The patterns become your ICP (\u00a73) and your qualification rules (\u00a75).\n4. **Re-contact the 8 silent ones** with the breakup email (`scripts.md` \u00a74). Any reply teaches you something, even a \"no\".\n5. **Don't wait for Q1 on the 9 \"interested\" ones.** \"Q1\" often means \"next budget year\". Find out whether budgets get set in Nov/Dec. If they do, you want to be in the budget (`scripts.md` \u00a73).\n6. **Ask each of the 5 customers for 2 introductions** to GMs they know. This is your fastest demo source.\n\n---\n\n## 3. Who to sell to (ICP), draft v0\n\nUpdate this once the win interviews are done.\n\n**Fit:**\n- Independent hotel (not chain-managed), 20\u2013150 rooms\n- Hosts groups regularly, e.g. weddings, conferences, sports teams, tour groups **[ASSUMPTION: around 10+ group bookings a year]**\n- Groups run today on email + spreadsheets, maybe with Word/PDF contracts\n- Has a sales manager or events coordinator, i.e. someone whose day job is groups\n\n**Stronger fit (check these against the winners):**\n- Lost revenue recently because of a group mistake: double-booked block, attrition not tracked, rooming list errors\n- Sales manager is new or overloaded\n- GM came from a chain and misses chain tools\n\n**Poor fit:**\n- Chain-managed or brand-mandated tech stack\n- Under ~5 groups a year\n- No one owns groups (GM does everything ad hoc)\n\n**Where new demos come from, in priority order:**\n1. Customer referrals (ask every customer, every quarter)\n2. Your ex-revenue-manager network, 2nd degree: ask contacts \"who else should I talk to?\"\n3. Targeted outbound to sales managers at hotels that advertise weddings/meetings on their website (`scripts.md` \u00a75). Build lists of 50 at a time by region.\n4. Regional hotel associations and independent hotel groups (talks, newsletters)\n\n---\n\n## 4. Pipeline stages\n\nEach stage has an **exit criterion**, something that has to be true before the deal moves on. That keeps the pipeline honest and makes it something you can hand over.\n\n| # | Stage | Exit criterion (must be true to move on) | Typical time |\n|---|---|---|---|\n| 1 | **Lead** | Fits ICP, and you have a named contact | |\n| 2 | **Discovery call booked/held** | Pain confirmed, group volume known, GM and sales manager identified | ~1 wk |\n| 3 | **Demo done** | Demo shown to the **sales manager AND the GM** (live or recorded), and the next step is in the calendar | ~1\u20132 wk |\n| 4 | **Proposal / trial** | Written proposal sent, decision date agreed, any objections listed | ~2 wk |\n| 5 | **Verbal yes** | GM says yes, and the signer and start date are known | ~1 wk |\n| 6 | **Won** | Signed, paid, onboarding booked | |\n| \u2014 | **Nurture** | Real fit, but timing is wrong. Has a specific re-contact date and reason | |\n| \u2014 | **Lost** | Clear no. Log the reason (\u00a78) | |\n\nRules:\n- **Every open deal has a next step with a date.** If it doesn't, it isn't a deal.\n- **No reply after the cadence in \u00a77 \u2192 move to Nurture or Lost.** Don't let deals sit in limbo; that's how the 8 went silent.\n- \"Interested, follow up in Q1\" goes in **Nurture** with a date and a reason, not in the active pipeline.\n\n---\n\n## 5. Discovery and qualification\n\nDo a short discovery call (20 min) before the demo, or the first 10 minutes of the demo. Full questions are in `scripts.md` \u00a72. You need answers to these:\n\n1. **Volume:** How many group bookings a year? What share of revenue?\n2. **Pain:** What went wrong with a group in the last 6 months? What did it cost?\n3. **Current way:** Who handles groups and how? Which spreadsheets, which PMS?\n4. **People:** Who would use it day to day? Who decides? Who signs?\n5. **Timing:** What's driving a change now? Is there a busy season to be ready for?\n6. **Budget:** How do software purchases get approved? Is there an annual budget cycle?\n\n**Disqualify early** when volume is too low, there's no named pain, or there's a chain mandate. A fast \"no\" is better than a 7-week \"maybe\".\n\n---\n\n## 6. The demo, and the GM problem\n\nGM decides, sales manager uses. **[ASSUMPTION]** A lot of the silent and \"Q1\" deals were probably demoed to only one of the two. Check this in the CSV backfill (`gm_involved` column).\n\n**Plan for both people:**\n- Sales manager: show their daily workflow (create block, track pickup, rooming list, contract). It should feel like time saved.\n- GM: show money and risk (revenue at risk from attrition, group pace vs. last year, the visibility they lack today). It should feel like revenue protected.\n- If the GM can't join, send them a 3-minute recorded summary afterwards, plus a one-page ROI note. Ask the sales manager: \"What will [GM] ask you about this?\"\n\n**Demo structure (30\u201340 min):**\n1. 5 min: recap their pain in their words (from discovery)\n2. 20 min: show only the 2\u20133 workflows tied to that pain, using their kind of group (wedding vs. conference)\n3. 5 min: ROI on their numbers, e.g. \"You said ~40 groups a year; one missed attrition clause costs you ~$X\"\n4. 5 min: agree the next step **before the call ends**: who else needs to see it, decision date, trial or proposal\n\n**Use your background.** You were a revenue manager, which counts for a lot of credibility with GMs. Lead with it.\n\n---\n\n## 7. Follow-up cadence\n\nAfter the demo:\n| Day | Action |\n|---|---|\n| 0 | Recap email: their pain, what you showed, the agreed next step (`scripts.md` \u00a73) |\n| 3 | Send the GM summary/ROI note if the GM wasn't on the call |\n| 7 | Check in on the next step, with one useful item (case study, template) |\n| 14 | Direct question: \"Is this a priority for this season, or should I check back in [month]?\" |\n| 21 | Breakup email (`scripts.md` \u00a74) \u2192 Nurture or Lost |\n\nNurture: one touch a month with something useful (a group-revenue tip, a customer story), plus a call on the agreed re-contact date.\n\n---\n\n## 8. Win/loss tracking\n\nEvery closed deal (won or lost) gets a reason in `pipeline.csv`, chosen from a fixed list so you can count them:\n\n**Lost reasons:** `no_budget` \u00b7 `timing` \u00b7 `no_gm_buyin` \u00b7 `low_group_volume` \u00b7 `chose_competitor` \u00b7 `pms_integration` \u00b7 `price` \u00b7 `no_response` \u00b7 `other`\n\n**Win reasons:** `pain_event` \u00b7 `new_sales_mgr` \u00b7 `gm_champion` \u00b7 `referral` \u00b7 `roi_clear` \u00b7 `other`\n\nReview once a month. Write one sentence on what you'll change.\n\n---\n\n## 9. Weekly rhythm (about 1 hour every Monday)\n\n1. Update every open row in `pipeline.csv` (stage, next step, date)\n2. Count these numbers and log them:\n   - New leads added\n   - Discovery calls held\n   - Demos held (**target: 5/week**)\n   - Proposals out\n   - Closes this week / total (**target: 15 by early Feb**)\n   - Pipeline value (open deals \u00d7 $400)\n3. Find every deal with no next step or an overdue date, and fix it today\n4. Block demo slots in the calendar for the week\n\nMonthly: win/loss review (\u00a78), update the ICP (\u00a73), check stage conversion rates.\n\n---\n\n## 10. Handover to a first salesperson (later)\n\nWhat should be ready before you hire, and what this playbook is building towards:\n- [ ] ICP proven by data from ~40+ demos, not by assumptions\n- [ ] Stage conversion rates known (e.g. demo\u2192proposal, proposal\u2192won)\n- [ ] A lead source that works without your personal network (outbound or referrals)\n- [ ] Demo script and recorded example demos\n- [ ] Objection list with answers (build it from the win/loss data)\n- [ ] `pipeline.csv` moved into a real CRM (HubSpot free or Pipedrive is plenty) once you pass ~50 active rows\n\n**What a first rep could take over first:** prospecting, discovery calls, and follow-up cadence. **What you should keep for now:** GM conversations and closing, where your revenue-manager credibility matters most.\n"}, {"name": "scripts.md", "text": "# Tallyroom Sales Scripts\n\nTemplates for `sales-playbook.md`. Replace the [brackets]. Keep emails short, since GMs and sales managers read on their phones.\n\n---\n\n## 1. Win interview (the 5 customers), 20 min call\n\nFrame it as: \"I'm trying to understand what made Tallyroom worth it for you, so I can find more hotels like yours.\"\n\n1. Before Tallyroom, how did you handle group bookings? What was the worst part?\n2. What happened that made you start looking? Why then and not 6 months earlier?\n3. Who else was involved in deciding? What did the GM need to hear?\n4. What almost stopped you from buying?\n5. What else did you consider, including \"keep using spreadsheets\"?\n6. What has changed since you started? Any numbers: hours saved, a group you didn't lose?\n7. If you were selling Tallyroom to a hotel like yours, what would you say?\n8. Who are 2 other GMs or sales managers you think would want to see it?\n\nLog the answers in the `win_loss_reason` and `notes` columns. Look for answers that repeat across all 5.\n\n---\n\n## 2. Discovery questions (20 min)\n\n**Groups today**\n- Roughly how many group bookings do you run a year? Weddings vs. corporate/conferences?\n- Walk me through the last group you booked, from first enquiry to checkout.\n- Where does that live today: email, Excel, PMS, paper?\n\n**Pain**\n- What's gone wrong with a group in the past 6 months?\n- How do you track pickup against the block and attrition dates today?\n- How long does a rooming list take you?\n\n**People and process**\n- Who handles groups day to day? Who would make a call on new software like this?\n- How does a software purchase usually get approved here?\n\n**Timing**\n- Is there a season you want to be ready for?\n- When do you set next year's budget?\n\nClose: \"Based on that, I'd like to show you [2\u20133 specific things]. Can we get [GM name] on that call too?\"\n\n---\n\n## 3. Emails: after the demo and nurture\n\n**Demo recap (Day 0)**\n> Subject: Tallyroom \u2013 recap and next step\n>\n> Hi [Name],\n>\n> Thanks for the time today. Here's what I heard:\n> - [Pain 1 in their words]\n> - [Pain 2]\n>\n> What we looked at: [workflow 1], [workflow 2].\n>\n> Next step, as agreed: [action] on [date]. [If GM absent: I'll send a 3-minute summary you can forward to [GM].]\n>\n> [Your name]\n\n**The 9 \"follow up in Q1\" (send now, not in January)**\n> Subject: Before your 2027 budget is set\n>\n> Hi [Name],\n>\n> When we spoke in [month] you mentioned looking at this again in Q1. Quick question: does next year's budget get set before then? If it does, I'd rather help you get a line in for group-booking tools now than have you wait until next year.\n>\n> Also: [one new thing, e.g. a new feature, or \"[Customer hotel] cut rooming-list time from X to Y\"].\n>\n> Worth 15 minutes in the next two weeks?\n\n---\n\n## 4. Breakup / re-engagement email (the 8 silent, and Day 21)\n\n> Subject: Close the file?\n>\n> Hi [Name],\n>\n> I haven't heard back, which usually means one of three things:\n> 1. Groups aren't a priority right now\n> 2. It's a fit, but the timing is wrong (tell me when to check back)\n> 3. Something about Tallyroom didn't fit. I'd honestly like to know what.\n>\n> Just reply with 1, 2, or 3 and I'll take it from there.\n>\n> [Your name]\n\nLog the reply in `win_loss_reason`. A \"3\" is worth a 10-minute call.\n\n---\n\n## 5. Outbound: cold email to a sales manager\n\nTarget: sales/events manager at an independent hotel whose website advertises weddings or meetings.\n\n> Subject: [Hotel name] group bookings\n>\n> Hi [Name],\n>\n> I was a hotel revenue manager for [X] years. The thing I hated most was tracking group blocks and attrition across email and spreadsheets.\n>\n> I built Tallyroom to fix that for independent hotels like [Hotel]. [Customer hotel] uses it for [weddings/conferences] and [result].\n>\n> Worth a 20-minute look?\n>\n> [Your name]\n\nFollow up twice (day 4, day 10) with one line each, then stop. If a customer reference is involved, ask the customer's permission before naming them.\n"}]}, "opus-3": {"cost_usd": 0.30204, "seconds": 157, "output_tokens": 10273, "files": [{"name": "email-templates.md", "text": "# Email Templates\n\nThese are starting points. Keep emails short and personalize the first line. Track which ones get replies, and drop the ones that don't.\n\n---\n\n## 1. First outreach (to sales or events manager)\n\n**Subject:** Group bookings at {{Hotel}}\n\nHi {{Name}},\n\nI was a hotel revenue manager for years. The part I hated most was running weddings and conferences through email threads and a spreadsheet.\n\nI built Tallyroom so independent hotels like {{Hotel}} can handle group inquiries, room blocks, and quotes in one place. {{Customer hotel, if nearby}} uses it now.\n\nWould a 20-minute call next week be worth it, to see whether it fits how you handle groups?\n\n{{Your name}}\nCEO, Tallyroom\n\n---\n\n## 2. Demo recap (same day)\n\n**Subject:** Tallyroom + {{Hotel}}: recap and next step\n\nHi {{Name}},\n\nThanks for your time today. Here's what I heard:\n- **Today:** {{their current process, in their words}}\n- **What hurts:** {{pain 1}}, {{pain 2}}\n- **What Tallyroom changes:** {{2\u20133 specific points}}\n\n**Next step:** {{GM call on DATE / proposal by DATE}}.\n\nI've attached a one-page summary you can forward to {{GM name}}. It covers what changes, the cost (USD 400/month), and how quickly you could be up and running.\n\n{{Your name}}\n\n---\n\n## 3. One-page summary for the GM (attachment or forward)\n\n**Tallyroom for {{Hotel}}**\n\n- **The problem:** {{N}} groups a year handled through email and spreadsheets \u2192 {{slow quotes / lost inquiries / room block errors}}\n- **What changes:** every group inquiry, quote, and room block in one place, so you can see the whole pipeline\n- **What it's worth:** ~{{N}} groups/year \u00d7 ~${{X}} average. Winning **one** extra group a year covers the ${{4,800}}/year cost {{several}} times over\n- **Cost:** USD 400/month\n- **Getting started:** live in {{N}} weeks, ready for {{season}}\n- **Contact:** {{Your name}}, {{phone}}\n\n---\n\n## 4. Nudge (no reply after 1 week)\n\n**Subject:** Re: Tallyroom + {{Hotel}}\n\nHi {{Name}},\n\nA quick thought after our call: {{a useful tip, or what a similar customer saw, e.g. \"one of our hotels cut quote turnaround from 2 days to the same day\"}}. *(Use only real customer results from your win/loss calls.)*\n\nDoes {{DATE}} still work to {{next step}}?\n\n{{Your name}}\n\n---\n\n## 5. \"Close the file\" (no reply after 2\u20133 weeks; also for the 8 silent deals)\n\n**Subject:** Should I close your file?\n\nHi {{Name}},\n\nI haven't heard back, so I'm guessing group bookings aren't a priority right now. That's completely fine.\n\nShould I close the file on my end? If timing is the issue, tell me when to check back and I'll do that.\n\nAnd if something in the demo didn't fit, one line on what it was would really help me.\n\n{{Your name}}\n\n---\n\n## 6. Q1 deals: budget question (send now, not in January)\n\n**Subject:** Tallyroom and your 2027 budget\n\nHi {{Name}},\n\nYou mentioned picking this back up in Q1. Quick question so I don't miss the window: does something like Tallyroom need to be in the 2027 budget? If so, who puts that together, and when?\n\nI'm happy to send {{GM name}} a short summary with the numbers so it's easy to include.\n\n{{Your name}}\n\n---\n\n## 7. Referral ask (to existing customers)\n\n**Subject:** Quick favor?\n\nHi {{Name}},\n\nGlad Tallyroom is working for {{Hotel}}. {{specific thing they told you}}\n\nDo you know 1\u20132 people at other independent hotels who are still running groups through email and spreadsheets? An intro would mean a lot. {{Optional: if they sign up, your next month is on us.}}\n\n{{Your name}}\n"}, {"name": "pipeline.csv", "text": "Hotel,City/Region,Rooms,Groups per year,Contact (user),Contact role,GM name,GM has seen demo (Y/N),Source,Stage,First contact date,Demo date,Last touch date,Next step,Next step date,Current tool,Main pain,Won/Lost reason,MRR (USD),Weeks to close,Notes\nCustomer 1 (fill in),,,,,,,,Network,Won,,,,Win/loss interview,2026-10-16,,,,400,,\nCustomer 2 (fill in),,,,,,,,Network,Won,,,,Win/loss interview,2026-10-16,,,,400,,\nCustomer 3 (fill in),,,,,,,,Network,Won,,,,Win/loss interview,2026-10-16,,,,400,,\nCustomer 4 (fill in),,,,,,,,Network,Won,,,,Win/loss interview,2026-10-16,,,,400,,\nCustomer 5 (fill in),,,,,,,,Network,Won,,,,Win/loss interview,2026-10-16,,,,400,,\nQ1 deal 1 (fill in),,,,,,,,Network,Nurture,,,,Call re: Q1 / 2027 budget,2026-10-23,,,,,,Said follow up in Q1 - find out why\nQ1 deal 2 (fill in),,,,,,,,Network,Nurture,,,,Call re: Q1 / 2027 budget,2026-10-23,,,,,,Said follow up in Q1 - find out why\nQ1 deal 3 (fill in),,,,,,,,Network,Nurture,,,,Call re: Q1 / 2027 budget,2026-10-23,,,,,,Said follow up in Q1 - find out why\nQ1 deal 4 (fill in),,,,,,,,Network,Nurture,,,,Call re: Q1 / 2027 budget,2026-10-23,,,,,,Said follow up in Q1 - find out why\nQ1 deal 5 (fill in),,,,,,,,Network,Nurture,,,,Call re: Q1 / 2027 budget,2026-10-23,,,,,,Said follow up in Q1 - find out why\nQ1 deal 6 (fill in),,,,,,,,Network,Nurture,,,,Call re: Q1 / 2027 budget,2026-10-23,,,,,,Said follow up in Q1 - find out why\nQ1 deal 7 (fill in),,,,,,,,Network,Nurture,,,,Call re: Q1 / 2027 budget,2026-10-23,,,,,,Said follow up in Q1 - find out why\nQ1 deal 8 (fill in),,,,,,,,Network,Nurture,,,,Call re: Q1 / 2027 budget,2026-10-23,,,,,,Said follow up in Q1 - find out why\nQ1 deal 9 (fill in),,,,,,,,Network,Nurture,,,,Call re: Q1 / 2027 budget,2026-10-23,,,,,,Said follow up in Q1 - find out why\nSilent deal 1 (fill in),,,,,,,,Network,Demo,,,,Send close-the-file email,2026-10-16,,,,,,Went silent after demo\nSilent deal 2 (fill in),,,,,,,,Network,Demo,,,,Send close-the-file email,2026-10-16,,,,,,Went silent after demo\nSilent deal 3 (fill in),,,,,,,,Network,Demo,,,,Send close-the-file email,2026-10-16,,,,,,Went silent after demo\nSilent deal 4 (fill in),,,,,,,,Network,Demo,,,,Send close-the-file email,2026-10-16,,,,,,Went silent after demo\nSilent deal 5 (fill in),,,,,,,,Network,Demo,,,,Send close-the-file email,2026-10-16,,,,,,Went silent after demo\nSilent deal 6 (fill in),,,,,,,,Network,Demo,,,,Send close-the-file email,2026-10-16,,,,,,Went silent after demo\nSilent deal 7 (fill in),,,,,,,,Network,Demo,,,,Send close-the-file email,2026-10-16,,,,,,Went silent after demo\nSilent deal 8 (fill in),,,,,,,,Network,Demo,,,,Send close-the-file email,2026-10-16,,,,,,Went silent after demo\n"}, {"name": "sales-playbook.md", "text": "# Tallyroom Sales Playbook (v1, Oct 2026)\n\nWritten for a solo founder doing the selling now. It's also the starting draft of what the first salesperson will get.\n\n**Goal:** 15 new hotels by early Feb 2027 (~17 weeks), and a process that someone else can run.\n\n---\n\n## 1. Where you are: the honest math\n\n| | Count | Rate |\n|---|---|---|\n| Demos (3 months, all from your network) | 22 | ~7/month |\n| Closed at USD 400/mo | 5 | 23% of demos |\n| \"Interested, follow up in Q1\" | 9 | \u2014 |\n| Went silent | 8 | \u2014 |\n| Average time to close | 7 weeks | \u2014 |\n\n**What this means for 15 more closes:**\n\n- **The 7-week cycle cuts the window short.** If a deal starts after about mid-December, it probably won't close by early February. So you really have **~10 weeks to start the deals** that count toward the goal.\n- **You have 17 open deals already.** If 3\u20135 of them close, you need **10\u201312 from new pipeline**.\n- **At a 23% demo-to-close rate, 10\u201312 closes takes ~45\u201350 new demos in 10 weeks. That's about 5 demos a week**, more than double your current pace.\n- **Your network probably can't supply that many.** You'll need new sources (section 7). You'll also need a better close rate, which comes from qualifying harder and getting the GM involved earlier (sections 3\u20135).\n\nCheck this every Monday. If you're under ~4 demos a week by early November, the goal is at risk. Fix the lead flow before you do anything else.\n\n**Revenue at stake:** 15 \u00d7 $400 = $6,000 more MRR. With the current 5, that's 20 hotels and $8,000 MRR ($96k ARR).\n\n---\n\n## 2. This week: find out why people bought (before you sell more)\n\nYou don't know why 5 bought and 17 didn't. That's the most valuable thing to learn right now, and the people who can tell you are all in your phone. Use `win-loss-interviews.md`.\n\n1. **Call all 5 customers** (20 min each). Ask why they bought, what nearly stopped them, who signed, and what changed after they started.\n2. **Call the 9 \"Q1\" deals.** Find out what Q1 actually means. Is it a budget cycle? A new GM? Just a polite no? (See the note below.)\n3. **Send the 8 silent deals a short \"close the file\" email** (template in `email-templates.md`). Silent deals reply to these more often than to \"just checking in\".\n4. **Log everything in `pipeline.csv`.** Use the 22 rows already there and fill in names and what you learn.\n\n> **Hypothesis to check on those calls:** many hotels set next year's budget in Q4. If \"follow up in Q1\" means \"it's not in this year's budget\", then waiting until January is the wrong move. Ask them now: *\"Is this something that needs to be in the 2027 budget? Who puts that together, and when?\"* That could turn several Q1 deals into December signatures or January start dates.\n\nAfter the 5 customer calls, write down 3\u20135 patterns: hotel size, group volume, who championed it, what triggered the purchase. That's your first real **ideal customer profile (ICP)**. Update section 3 with it.\n\n---\n\n## 3. Who to sell to (qualification)\n\nStarting ICP. Replace it with what the win/loss calls show you.\n\n- Independent hotel, **20\u2013150 rooms**\n- **Real group volume.** Regular weddings, conferences, or tour groups. Find out the number per year; this matters more than room count.\n- Group bookings are **currently run in email and spreadsheets**\n- **The person who handles groups feels the pain:** slow quotes, lost leads, mistakes in room blocks, no visibility\n- **The GM is reachable** and can approve ~$400/month without the owner or a head-office process (or you know who has to approve it)\n\n**Qualifying questions, for the first call or the first 10 minutes of the demo:**\n\n1. \"Roughly how many groups or events did you host in the last 12 months? What's a typical one worth?\"\n2. \"Walk me through what happens when a wedding inquiry comes in today.\"\n3. \"What went wrong the last time a group booking went badly?\"\n4. \"Who besides you would need to be involved in deciding on a tool like this?\"\n5. \"If this solved it, when would you want it running? Is there a season you're preparing for?\"\n\n**Disqualify or deprioritize** when a hotel has very few groups a year, there's no pain they can name, the GM is out of reach, or a chain or management company mandates its software. Spending less time on bad-fit demos is the cheapest way to raise your 23%.\n\n---\n\n## 4. Pipeline stages\n\nEach stage has an **exit criterion**, something that must be true before the deal moves forward. This is what makes the pipeline honest, and it's what lets you hand it to someone else later.\n\n| # | Stage | Exit criterion (to move forward) |\n|---|---|---|\n| 1 | **Lead** | Fits the ICP on paper (size, independent, does groups) |\n| 2 | **Discovery** | Pain confirmed, group volume known, GM identified by name |\n| 3 | **Demo** | Sales manager has seen it **and** GM has seen it or agreed to a GM call |\n| 4 | **Proposal** | Written proposal sent with price, start date, and onboarding plan; GM has received it |\n| 5 | **Decision** | Verbal yes, and paperwork or card sent |\n| \u2705 | **Won** | Signed / paying. Log *why they bought* |\n| \u274c | **Lost** | Clear no. Log *why* (one of: no budget, no pain, timing, chose other tool, no GM access, went dark) |\n| \ud83d\udca4 | **Nurture** | Real fit, but there's a concrete future reason (budget date, new GM, season). **Must have a dated next step** |\n\n**Pipeline rules:**\n\n- **Every open deal has a next step and a date.** No date means it's not a deal. Move it to Nurture with a date, or to Lost.\n- **\"Interested, follow up in Q1\" isn't a stage.** Either it's Nurture with a specific reason and date, or it's Lost.\n- **A deal with no reply in 3 weeks gets the \"close the file\" email.** If they still don't answer, mark it Lost: went dark.\n- **Write down every win and loss reason.** In 3 months that tells you more than anything in this document.\n\n---\n\n## 5. The two-buyer problem: sales manager uses it, GM signs\n\nThis is very likely where the 7 weeks go. You win over the sales manager, who then has to sell it to the GM without you there. Fix this on purpose:\n\n- **Ask for the GM early.** In discovery: *\"Usually the GM wants to see this before anything gets decided. Can we get 15 minutes with them after the demo, or have them join the last part?\"*\n- **Run two different conversations:**\n  - **Sales manager:** day-to-day workflow. Faster quotes, room blocks, nothing falling through the cracks, less spreadsheet work.\n  - **GM:** money and risk. Group revenue captured, faster response time to inquiries, visibility across all groups, no dependence on one person's inbox.\n- **Use your background.** You were a revenue manager, so give the GM a quick ROI calculation with their own numbers: *\"You do ~N groups a year at ~$X each. If faster quoting wins you one extra wedding a year, Tallyroom pays for itself several times over.\"* ($400/mo = $4,800/yr; plug in their real group value.)\n- **Hand the champion a one-page summary** to forward to the GM: their problem, what changes, price, start date. Don't let them describe it from memory.\n\n### Demo structure (30\u201340 min)\n1. **(5 min) Confirm what they told you in discovery.** \"You said X happens when a wedding inquiry comes in. Is that right?\"\n2. **(15\u201320 min) Show *their* workflow,** using an example from their kind of business (a wedding or conference like theirs). Skip the features they won't use.\n3. **(5 min) GM angle:** visibility, revenue, ROI calculation.\n4. **(5\u201310 min) Agree on the next step before you hang up:** GM call booked, or proposal date, or start date. **Never end a demo without a date in the calendar.**\n\n---\n\n## 6. Follow-up cadence\n\n| When | What |\n|---|---|\n| Same day as demo | Recap email + one-page summary for the GM + confirm the next step |\n| Day 3\u20135 | GM call, or answer to whatever's blocking (pricing, IT, onboarding) |\n| Week 2 | Proposal with start date. Tie it to their season (\"live before wedding inquiries pick up in January\") |\n| No reply, +1 week | Short nudge with something useful (a tip, a customer quote) |\n| No reply, +2 weeks | \"Close the file\" email |\n| Nurture | Contact on the agreed date; a light check-in every 4\u20136 weeks in between |\n\nAll templates are in `email-templates.md`.\n\n---\n\n## 7. Where the new demos come from\n\nYou need ~5 demos a week. Use several sources and track which one each deal came from (`Source` column) so you learn what works.\n\n1. **Referrals from the 5 customers.** Ask on the win/loss call: *\"Who else do you know who runs groups at an independent hotel?\"* Offer something for an intro that closes (e.g. a free month).\n2. **Your network, second circle.** Former colleagues, revenue managers you know who've moved to other hotels.\n3. **Direct outreach to independent hotels in 1\u20132 regions** where you already have customers, so you can say \"hotels near you use this.\" Write to the **sales/events manager** with a short, specific message, and bring in the GM after the first conversation.\n4. **Hotel associations, local tourism boards, and hospitality events.** One talk or webinar on \"running group sales without spreadsheets\" can fill several weeks of demos.\n5. **Wedding and event venues partners** (planners, DMCs). Lower priority; try later.\n\n**Weekly activity target** (adjust once you see your real conversion rates):\n- ~40 new outreach touches \u2192 ~10 conversations \u2192 ~5 demos \u2192 ~1\u20132 closes a week once the pipeline is full\n\n---\n\n## 8. Weekly routine (solo)\n\n- **Monday, 30 min: pipeline review** in `pipeline.csv`\n  - Every open deal: next step dated? Stage accurate according to the exit criteria?\n  - Any deal with no reply in 3+ weeks \u2192 send \"close the file\"\n  - Count this week's numbers (below)\n- **Block 2\u20133 fixed slots a week for outreach** and protect them from product work.\n- **After each call: 2 minutes to update the tracker.** If it isn't written down, the next salesperson won't have it.\n\n**The 5 numbers to track every week:**\n1. New conversations started\n2. Demos held (target ~5)\n3. Demos with the GM involved (% of total)\n4. Proposals sent\n5. Closes, and win rate by source\n\n---\n\n## 9. Getting ready to hand over to a first salesperson\n\nHire when you can say: *\"For hotels that look like X, if we get to a demo with the GM involved, we close about Y% in Z weeks.\"* That's when the process is repeatable enough to teach. Until then, a hire mostly inherits guesswork.\n\n**What you'll hand over (build it as you go):**\n- This playbook, updated with real win/loss patterns and a proven ICP\n- `pipeline.csv` with 4 months of real data: sources, stages, reasons, cycle times\n- The email templates that actually got replies\n- Recordings or notes from 3\u20135 good demos\n- A list of common objections and the answers that worked\n- The ROI calculation for GMs\n\n**What you probably keep, at least at first:** the GM conversations on bigger or tricky deals, and the customer relationships that drive referrals.\n\n**The first salesperson's likely role:** owning outreach, discovery, and demos with sales managers. Founder joins for GM calls until the hire is closing on their own.\n\n---\n\n## 10. Next 30 days at a glance\n\n| Week | Focus |\n|---|---|\n| **Week 1** (Oct 12) | Win/loss calls with the 5 customers. Fill in `pipeline.csv`. \"Close the file\" emails to the 8 silent deals |\n| **Week 2** (Oct 19) | Calls with the 9 Q1 deals about budget timing. Update the ICP. Ask the 5 customers for referrals |\n| **Week 3** (Oct 26) | Start outreach in 1\u20132 regions. Aim for 4\u20135 demos |\n| **Week 4** (Nov 2) | First Monday check against the numbers. Under ~4 demos a week? Fix lead flow first |\n"}, {"name": "win-loss-interviews.md", "text": "# Win/Loss Interview Guide\n\n**Purpose:** find out why 5 hotels bought and 17 didn't. Keep each call to 15\u201320 minutes. Write the answers into `pipeline.csv` (`Won/Lost reason`, `Main pain`, `Notes`) right after the call.\n\n**How to frame it:** \"I'm not selling anything on this call. I'm trying to learn how to do this better, and your honest answer helps me a lot.\"\n\n---\n\n## A. The 5 customers (why they bought)\n\n1. \"Before Tallyroom, what was going wrong with group bookings? Was there a specific moment that made you look for something?\"\n2. \"What made you decide to go ahead? What was the deciding factor?\"\n3. \"Who was involved in the decision? How did the GM get convinced?\" *(The answer here helps you shorten the 7 weeks.)*\n4. \"What nearly stopped you from buying?\"\n5. \"Did you look at anything else, or consider just staying with spreadsheets?\"\n6. \"What's different now? Can you put a number on it? (Time saved, groups won, mistakes avoided.)\"\n7. \"How would you describe Tallyroom to another hotel?\" *(Write down their exact words and reuse them in emails.)*\n8. \"Who else do you know who runs groups at an independent hotel?\" *(Referral ask.)*\n\n**Also note for each one:** rooms, groups per year, who championed it (sales manager or GM), how they first heard of you, and how many weeks it took to close.\n\n---\n\n## B. The 9 \"follow up in Q1\" deals\n\nDon't wait until Q1. Call now.\n\n1. \"Last time you mentioned coming back in Q1. What changes in Q1 for you?\"\n2. \"Does this need to go into your 2027 budget? Who builds that, and when is it finalized?\"\n3. \"Has the GM seen Tallyroom? What did they think?\"\n4. \"Is there a season you want to be ready for, like wedding inquiries or conference season?\"\n5. \"Honestly, is this a real priority for next year, or more of a nice-to-have?\"\n\n**Sort each one afterwards:**\n- **Real timing reason** (budget, new GM, season) \u2192 **Nurture**, with a dated next step. Try to get it into the 2027 budget now\n- **Could move sooner** \u2192 back to **Demo/Proposal** with a GM call booked\n- **Polite no** \u2192 **Lost**, and log the reason\n\n---\n\n## C. The 8 silent deals\n\nFirst send the \"close the file\" email from `email-templates.md`. If someone replies, ask:\n\n1. \"What made you decide it wasn't the right time, or the right fit?\"\n2. \"Was there anything in the demo that didn't match how you work?\"\n3. \"Was price a factor, or was it more about priorities and timing?\"\n4. \"Did the GM get involved at any point?\"\n\nAnyone who doesn't reply within 1 week \u2192 **Lost: went dark**.\n\n---\n\n## After all calls: patterns to look for\n\nFill this in once the calls are done.\n\n| Question | Won (5) | Not won (17) |\n|---|---|---|\n| Typical groups per year | | |\n| Room count range | | |\n| Who championed it | | |\n| Did the GM see a demo? | | |\n| Trigger event / pain named | | |\n| Lead source | | |\n| Weeks in pipeline | | |\n\n**Use the results to:** rewrite the ICP in `sales-playbook.md` section 3, and update the demo and GM talking points in section 5.\n"}]}, "haiku-1": {"cost_usd": 0.01075, "seconds": 66, "output_tokens": 13041, "files": [{"name": "pipeline.csv", "text": "hotel,rooms,city,gm_contact,sales_manager_contact,source,stage,status,first_demo_date,last_touch,next_step,next_step_date,loss_reason,notes\nBackfill-W1 (fill name),,,,,network,Won,won,,,Win interview: why they bought,,,\nBackfill-W2 (fill name),,,,,network,Won,won,,,Win interview: why they bought,,,\nBackfill-W3 (fill name),,,,,network,Won,won,,,Win interview: why they bought,,,\nBackfill-W4 (fill name),,,,,network,Won,won,,,Win interview: why they bought,,,\nBackfill-W5 (fill name),,,,,network,Won,won,,,Win interview: why they bought,,,\nBackfill-I1 (fill name),,,,,network,Demo,interested,,,Re-engage now (was Q1 follow-up),,,\nBackfill-I2 (fill name),,,,,network,Demo,interested,,,Re-engage now (was Q1 follow-up),,,\nBackfill-I3 (fill name),,,,,network,Demo,interested,,,Re-engage now (was Q1 follow-up),,,\nBackfill-I4 (fill name),,,,,network,Demo,interested,,,Re-engage now (was Q1 follow-up),,,\nBackfill-I5 (fill name),,,,,network,Demo,interested,,,Re-engage now (was Q1 follow-up),,,\nBackfill-I6 (fill name),,,,,network,Demo,interested,,,Re-engage now (was Q1 follow-up),,,\nBackfill-I7 (fill name),,,,,network,Demo,interested,,,Re-engage now (was Q1 follow-up),,,\nBackfill-I8 (fill name),,,,,network,Demo,interested,,,Re-engage now (was Q1 follow-up),,,\nBackfill-I9 (fill name),,,,,network,Demo,interested,,,Re-engage now (was Q1 follow-up),,,\nBackfill-S1 (fill name),,,,,network,Demo,stalled,,,One-line re-engage email; Lost if no reply in 14 days,,,\nBackfill-S2 (fill name),,,,,network,Demo,stalled,,,One-line re-engage email; Lost if no reply in 14 days,,,\nBackfill-S3 (fill name),,,,,network,Demo,stalled,,,One-line re-engage email; Lost if no reply in 14 days,,,\nBackfill-S4 (fill name),,,,,network,Demo,stalled,,,One-line re-engage email; Lost if no reply in 14 days,,,\nBackfill-S5 (fill name),,,,,network,Demo,stalled,,,One-line re-engage email; Lost if no reply in 14 days,,,\nBackfill-S6 (fill name),,,,,network,Demo,stalled,,,One-line re-engage email; Lost if no reply in 14 days,,,\nBackfill-S7 (fill name),,,,,network,Demo,stalled,,,One-line re-engage email; Lost if no reply in 14 days,,,\nBackfill-S8 (fill name),,,,,network,Demo,stalled,,,One-line re-engage email; Lost if no reply in 14 days,,,\n"}, {"name": "sales-process.md", "text": "# Tallyroom sales process\n\nOwner: CEO (sole seller). Review weekly. Source of truth for deals: `pipeline.csv` in this folder (move to Notion or a CRM later, but keep one place only).\n\n## Baseline\n\n- 22 demos in 3 months, all from the founder's network.\n- 5 won (23% demo-to-win) at USD 400/month each.\n- 9 \"interested, follow up in Q1\". 8 went silent.\n- Average cycle: 7 weeks. Decision maker: usually the GM. The sales manager is the daily user.\n- Unknown: why the 5 bought and the other 17 didn't. Fixing that is step one.\n\n## Goal math\n\nTarget: 15 new hotels in ~17 weeks (to about 2027-02-08).\n\n- 15 \u00d7 USD 400 = USD 6,000 new MRR (current: USD 2,000).\n- A deal that closes within the window must start by about week 10, given a 7-week average cycle.\n\nRough pipeline math (assumptions, to be replaced with real conversion data after week 4):\n\n| Source | Assumption | Expected wins |\n|---|---|---|\n| 9 \"interested\" deals, re-engaged now | 30% convert | ~3 |\n| 8 silent deals, one re-engage attempt | 10% revive | ~1 |\n| New demos started in the next 10 weeks | 23% (today's rate) | ~11.5 needed |\n\nAbout 50 new demos are needed in 10 weeks, roughly 5 per week. Today the rate is about 1.7 per week. Treat 15 as a stretch target and check the pace at each checkpoint below.\n\n## Ideal customer (hypothesis)\n\nTo confirm against the 5 wins before relying on it:\n\n- Independent hotel, 20\u2013150 rooms.\n- Regularly takes group bookings (weddings, conferences) handled by email and spreadsheets.\n- GM or owner has budget authority.\n- Has a group booking in the next few months that is painful to manage.\n\n## Stages\n\n| Stage | Enter when | Exit when | Next |\n|---|---|---|---|\n| Lead | Hotel identified, not yet talked to | Discovery call booked | Discovery |\n| Discovery | Call held with GM or sales manager | Qualified (see below) and demo booked with GM and sales manager | Demo |\n| Demo | Demo held | Trial agreed with a real upcoming group, or a decision date is set | Trial |\n| Trial | Trial started on a real booking | Signed and paid, or clear no | Won / Lost |\n| Won | Signed and paid | Onboarding handed over | \u2014 |\n| Lost | No, or no response after the follow-up cadence | \u2014 | Revisit in 90 days if reason is timing |\n\nNurture is not a stage. A deal that is \"not now\" stays in its stage with status `nurture` and a dated next step.\n\n## Qualification (discovery call)\n\nQualified when all of these are true. Otherwise, set status `nurture` with a date:\n\n1. **Pain:** names a specific group booking that went wrong or is slow (double bookings, lost deposits, spreadsheet errors).\n2. **Access:** the GM or owner is named and will attend the demo.\n3. **User:** the sales manager will attend.\n4. **Timing:** a real decision within 8 weeks, or a named reason it's later.\n5. **Price:** USD 400/month is acknowledged without a fight.\n\nRecord volume (group bookings per month) in the notes. Use it to find the threshold that wins share.\n\n## Demo rules\n\n- Invite the GM and the sales manager. Never demo to one person only.\n- Ask for one real upcoming group booking and demo with that scenario.\n- The last 5 minutes: ask \"What would you need to see to say yes?\", \"Who else decides?\", and book the next meeting with a date.\n- A demo does not end without a dated next step.\n\n## Follow-up rules\n\n- Send a recap within 24 hours: what they said they need, the next step, and its date.\n- Every open deal has a dated next step in `pipeline.csv`.\n- Cadence for a silent deal: touches on day 2, 5, 9, and 16. Vary the channel.\n- No reply after day 16: set stage to Lost with reason `no response`. Revisit in 90 days.\n- Stalled = no dated next step, or no activity for 14 days. Review every Monday.\n\n## Loss reasons (pick one)\n\n- `no response`\n- `timing` (revisit date set)\n- `no pain` (group volume too low or no current problem)\n- `no access` (never reached the decision maker)\n- `price`\n- `current system is good enough`\n- `chose another option`\n- `other` (write a sentence)\n\n## Win/loss learning (do this first)\n\n1. **5 wins, 30-minute calls:** what triggered the search, what they compared us to, who said yes and why, what almost stopped them, what they'd tell a peer.\n2. **9 \"interested\":** \"What would need to be true for you to say yes now?\" Re-engage now, not in Q1.\n3. **8 silent:** one short email: \"Quick question, was it timing, price, or not a fit? One word is fine.\" Offer a choice of reasons.\n4. After 10 calls, write a one-page \"why we win\" in this folder. Keep it to what the calls show.\n\n## Weekly routine (Monday, 60 minutes)\n\n1. Open `pipeline.csv`. Clear anything stalled or without a dated next step.\n2. Check this week's calendar: discovery calls, demos, trial starts.\n3. Record the numbers: discovery calls, demos held, trials, wins, losses, open deals by stage.\n4. Pick two lead-source actions for the week.\n\n## Lead sources (beyond the network)\n\n- Referrals: ask each of the 5 customers for 2 introductions to GMs at peer hotels.\n- Target list: build a list of independent hotels in one or two cities. Start with those that run event space.\n- Direct outreach: GMs and sales managers by email or LinkedIn.\n- Industry events where GMs gather.\n\nPick two to start. Measure which one produces qualified discovery calls.\n\n## 17-week plan\n\n**Weeks 1\u20132: foundations**\n- Load `pipeline.csv` with real hotel names (backfill has placeholders).\n- Run the 5 win calls and the 9 \"interested\" re-engagements.\n- Send the 8 silent re-engage emails.\n- Ask customers for referrals.\n\n**Weeks 3\u201310: build pipeline**\n- Run the Monday routine every week.\n- Start about 5 demos per week, adjusted to measured conversion.\n\n**Weeks 11\u201317: close**\n- Stop starting new demos unless they can close before the window ends.\n- Focus on trials and decision dates.\n\n**Checkpoints (proposed):**\n- End of week 4: 12+ open deals in Discovery or later, with dated next steps.\n- End of week 8: 6+ deals in Demo or Trial, each with a decision date.\n- If a checkpoint is missed, adjust lead sources, not the qualification bar.\n\n## Handover readiness (before the first salesperson)\n\nBefore hiring, have these in writing:\n\n- Ideal customer profile, validated against the wins.\n- Qualification questions and demo script.\n- Pricing and discount rules, including who approves exceptions.\n- Three months of clean tracker data: conversion per stage and average cycle length.\n- Loss reasons with counts.\n- Answers to the top objections.\n- Who does onboarding after a win, and how it is handed over.\n\n## Open questions\n\n- Does \"follow up in Q1\" mean January\u2013March 2027? If so, that is too late for the 4-month goal. Re-engage now.\n- What group-volume threshold separates wins from non-wins? (Answer after the win calls.)\n- Is there an annual plan or only monthly?\n- Is a paid or free trial the default, and how long?\n- Who handles onboarding after a win?\n"}]}, "haiku-2": {"cost_usd": 0.00911, "seconds": 57, "output_tokens": 11115, "files": [{"name": "pipeline.csv", "text": "hotel,city,rooms,contact_name,contact_role,source,stage,first_contact_date,demo_date,next_step,next_step_date,gm_involved,group_bookings_last_12m,current_tools,trigger,value_per_month_usd,outcome,loss_reason,last_touch_date,notes\n"}, {"name": "sales-process.md", "text": "# Tallyroom sales process\n\nVersion 1, 2026-10-08. Owner: CEO. Revisit after the win/loss interviews (section 8) and again at the end of January.\n\n## 1. Target and goal\n\n- **ICP:** independent hotels with 20-150 rooms that take group bookings (weddings, conferences) and run them on email and spreadsheets.\n- **Buyers:** the general manager decides. The sales manager uses it day to day. The GM must be in the room before a deal closes.\n- **Goal:** 15 new hotels by 2027-02-08 (four months).\n\n**The math (assumes each deal is USD 400/month; confirm this):**\n\n- Current demo-to-close rate: 5 of 22 = 23%.\n- 15 closes from new demos at 23% = about 65 demos, roughly 16 a month.\n- The 9 \"interested\" leads are the warm pool. If 1 in 3 closes, that's 3. The other 12 need about 52 new demos.\n- Average cycle is 7 weeks. A demo after mid-December probably won't close before the deadline. The pipeline has to be full by then.\n\n## 2. Stages\n\n| # | Stage | Enter when | Leave when |\n|---|---|---|---|\n| 1 | Lead | Known contact or inbound inquiry | Discovery call booked |\n| 2 | Discovery | Call done | Qualified (section 3) and demo booked with a date |\n| 3 | Demo | Demo done | Next step agreed with a date: proposal, trial group, or follow-up with GM |\n| 4 | Proposal | Pricing sent | Verbal yes from the GM, or a dated decision |\n| 5 | Won | Signed and paid | Onboarding starts |\n| 6 | Lost | Explicit no, or no reply after 4 follow-ups over 6 weeks | Loss reason recorded |\n| 7 | Nurture | \"Not now\" with a stated time | Dated re-contact |\n\n**Rule:** every open deal has a next step and a date. Drop the phrase \"follow up in Q1\" from the tracker. Replace it with a dated task.\n\n## 3. Qualify (discovery call, 30 min)\n\nA lead is qualified only when all four are true:\n\n- **Pain:** they describe a group booking that went wrong or took too long. Examples: a deposit lost, a room block double-booked, a proposal sent late.\n- **Volume:** they've handled 10+ group bookings in the last 12 months. Tune this threshold once the win/loss data is in.\n- **Access:** the GM is on the demo, or will be.\n- **Trigger:** something in the next 90 days makes this a priority. Examples: a big upcoming event, a booking-tool renewal, a new sales manager.\n\nDisqualify politely if the hotel is under about 20 rooms, takes almost no group business, or the GM won't engage after two attempts.\n\n## 4. Discovery questions\n\n1. Walk me through the last group booking you did, from first inquiry to signed contract.\n2. How many group bookings did you take last year? Weddings, conferences, other?\n3. Where does the information live today? Inbox, spreadsheet, PMS, paper?\n4. What went wrong the last time something slipped? What did it cost?\n5. Who sends proposals, and who approves pricing or contract changes?\n6. How many hours a week does group admin take?\n7. What's coming up in the next three months that would make this a priority?\n8. If this worked perfectly, what would change for you and for your GM?\n9. What have you tried before, and what happened?\n10. Who else would have to agree to change how you work?\n\nWrite the answers into the tracker. They become the demo script.\n\n## 5. Demo (45 min; sales manager and GM invited)\n\n- Open with the group booking they described in discovery, not a product tour.\n- Run that group end to end in their terms, using their room count and event type.\n- Show only the steps that address the pain named in discovery.\n- Before ending, ask: \"What would have to be true for you to use this on your next group?\"\n- Leave with a dated next step, the attendees for it, and what they'll bring: upcoming group details or their contract template.\n\n## 6. Proposal and close\n\n- Send the proposal within 24 hours of the demo, and only after you know who decides, the price range they accepted, and a start date.\n- Call within two days to review it. Ask: \"What's the one thing that would stop you signing this week?\"\n- Get a dated decision: \"We'll decide by [date].\" Record it in the tracker.\n- Don't discount until you've asked what they would give up in return.\n- Pricing: [fill in]\n\n## 7. Follow-up cadence\n\n- Demo or proposal not advancing: touch on day 2, 5, 10, and 17, then every 3-4 weeks until a yes or no.\n- Each touch must add something: a relevant case, a specific question, or a calendar invite. \"Just checking in\" is not a touch.\n- After 4 touches over about 6 weeks with no reply, mark Lost (no response) and set a nurture reminder for 3 months.\n- Nurture: re-contact on the date they gave, with a reason to talk.\n\n## 8. Win/loss interviews (do these first, in the next two weeks)\n\nContact all 22 demo leads in this order: the 8 silent, then the 9 interested, then the 5 won. Ask for 15 minutes. Say: \"I'm not selling anything. I want to understand why this didn't fit.\"\n\n**For the 5 who bought:**\n- What made you say yes?\n- What did you compare us against?\n- Who had to agree, and what slowed things down between demo and yes?\n\n**For the silent and interested:**\n- What was the main reason you didn't move forward: timing, price, fit, or the people?\n- What would have made it a yes?\n- What do you use now, and does it work?\n\nRecord each answer in `pipeline.csv` under `loss_reason`, using these codes:\n\n- `NO_TRIGGER`: no pressing reason to change now\n- `STATUS_QUO`: email and spreadsheets are good enough\n- `NO_BUDGET`\n- `NOT_DECIDER`: the GM wasn't involved\n- `FIT`: a missing feature or workflow\n- `PRICE`\n- `COMPETITOR`: they chose another option\n- `TIMING`: a stated date later than the goal window\n- `NO_RESPONSE`\n\nAfter 22 interviews, look for the top two or three reasons and change the discovery questions or qualification rules to filter out the losers earlier.\n\n## 9. Weekly routine\n\n- **Monday, 30 min, pipeline review.** Go through every open deal: stage, next step, date. Any deal without a dated next step gets one, or moves to nurture.\n- **Weekly activity targets.** Start with 4 discovery calls and 4 demos a week, then adjust. Log actual numbers for four weeks before changing the targets.\n- **Friday, 15 min.** Log closes and losses, and fill in each loss reason code.\n\n## 10. Handover to a first salesperson\n\n- The handover packet is this document, `pipeline.csv`, the discovery questions, the demo flow, and the win/loss findings.\n- Don't hire until the process has run for 8 weeks with the stage definitions unchanged. Changing the process during a hire's ramp costs more than it saves.\n- Before that, write down the parts that only you know: how you price, which objections you answer and how, and what the hotel's revenue numbers look like when a group goes well or badly.\n\n## Objections\n\n- **\"We're fine with email and spreadsheets.\"** Ask: \"What happened the last time a group nearly slipped? How many hours a week does this take?\"\n- **\"Not now.\"** Ask: \"What would need to change for now to be the right time?\" Put their answer and date in nurture.\n- **\"I need to check with my GM.\"** Ask: \"Can we get 20 minutes with them this week? I'll show them what you'd see day to day.\"\n- **\"Too expensive.\"** Ask: \"Compared to what? What does one badly run group cost you?\"\n- **\"We already use [PMS or tool].\"** Ask: \"What does it do for group bookings today, and where does it fall short?\"\n"}]}, "haiku-3": {"cost_usd": 0.00959, "seconds": 62, "output_tokens": 11863, "files": [{"name": "pipeline.csv", "text": "hotel,city,rooms,contact_name,contact_role,gm_involved,source,stage,first_contact,demo_date,next_step,next_step_date,monthly_usd,loss_reason,notes\n"}, {"name": "sales-process.md", "text": "# Tallyroom sales process\n\nOwner: CEO (solo seller). Tracker: `pipeline.csv` in this folder. Update it the same day as every touch.\n\n## Target check\n\n- Goal: 15 new hotels by about 2027-02-08 (four months out). At USD 400/month each, that is USD 6,000 MRR. Today it is 5 hotels, USD 2,000 MRR.\n- Current pace: 5 wins in about 3 months, so roughly 7 more in 4 months if nothing changes.\n- The 9 \"follow up in Q1\" leads: at the historical 23% win rate that is about 2 deals; at 40% it is about 4. Q1 mostly falls after the target date, so re-contact them now.\n- Expected result with no changes: about 9 wins by February, which leaves a gap of about 6.\n- Demo math: with a 7-week cycle, demos must happen by about 2026-12-18 to close by February. From now that is about 10 weeks.\n\n| Win rate (demo to won) | Demos needed for 15 wins | Demos per week, next 10 weeks |\n|---|---|---|\n| 23% (today, n=22) | ~65 | ~6.5 |\n| 35% | ~43 | ~4.3 |\n| 50% | ~30 | ~3 |\n\nToday's pace is about 1.7 demos per week. Either the win rate or the demo volume has to change a lot, or the February target should be reset to about 9 with 15 by spring. That call is yours; the tracker will show which lever is moving.\n\n## Stages\n\n| Stage | Enter when | Exit when | Target time |\n|---|---|---|---|\n| Lead | Named GM, owner, or group sales lead at an independent hotel (20-150 rooms) that takes group bookings | Discovery call booked | 1 week |\n| Discovery | 30-minute call with whoever handles group inquiries | All four qualification checks pass, or marked Lost | 2 weeks |\n| Demo | 45-minute demo with the GM and the group sales lead, built on a real group from discovery | They name a next step with a date | 1 week |\n| Proposal | GM says the price works in principle | Verbal yes with a signing or start date | 4 weeks |\n| Won | Signed and first event entered | - | - |\n| Lost | Explicit no, or six touches with no reply | Loss reason logged | - |\n| Nurture | \"Not now\" with a reason | Re-contact date set | - |\n\nThe stage targets add up to about 7 weeks, which matches your current average cycle.\n\n## Qualification (all four before the demo)\n\n- **Pain:** they lost or nearly lost a group in the last 6 months, or run groups in email and spreadsheets, and can name the cost (missed cutoffs, double bookings, lost inquiries).\n- **Power:** the GM or owner is in the demo or has agreed to attend. The sales manager alone is not enough.\n- **Timing:** they have group events in the next 6 months, or a trigger to change now.\n- **Budget:** ask directly, \"Is USD 400 a month something you can approve, or does someone else need to sign off?\"\n\nDisqualify early if they have only a few small groups a year, or if group sales is run by a central team outside the hotel.\n\n## Discovery questions\n\n1. Walk me through the last group booking you handled, from first inquiry to deposit. Where did it get stuck?\n2. Who touches a group during that process? GM, sales, events, F&B, accounting?\n3. How many group inquiries do you get a month, and how many do you lose? Why do you lose them?\n4. Where do room blocks, cutoff dates, and rooming lists live today? What happens when the person who owns that file is on leave?\n5. What happens when a client changes a block after signing?\n6. What did a missed group cost you last year, in rooms or in revenue?\n7. Who would have to say yes, and what do they care about most?\n8. Have you tried to fix this before? What happened?\n9. If nothing changes, what does the next six months look like?\n\nWrite answers in the `pain_summary` field. Use their words.\n\n## Demo (45 minutes)\n\n1. (5 min) Repeat back what they told you in discovery, in their words.\n2. (10 min) Run their last group through Tallyroom, from first inquiry to deposit.\n3. (10 min) Show cutoffs, room blocks, and rooming lists, focused on where their spreadsheet failed.\n4. (10 min) Hand over the keyboard. The GM or group sales lead runs one step themselves.\n5. (10 min) Close: \"What would have to be true for your next group to run through this?\" Then book the next step with a date.\n\nAdjust the steps to what the product actually does today.\n\n## Follow-up after a demo\n\n| When | Action |\n|---|---|\n| Day 1 | Recap email: the one thing they said mattered, plus two proposed dates for the next step |\n| Day 4 | Short call or message that references a specific item (their cutoff, their event) |\n| Day 10 | Send a value note built from their own numbers, such as the cost of one missed cutoff |\n| Day 21 | Ask directly: \"Is this a no, a not-now, or a yes with conditions?\" |\n| Day 45 | Move to Nurture with a reason to revisit (their event season, a change in staff) |\n\nAfter six touches with no reply, mark the deal Lost (went quiet) or move it to Nurture with a date.\n\n## Objections\n\n| They say | Reply direction |\n|---|---|\n| \"We're fine with spreadsheets.\" | \"Walk me through the last time a cutoff nearly slipped. What did it cost?\" |\n| \"Not in the budget right now.\" | \"Is it the amount or the timing? What would one event through this need to save or win to justify it?\" |\n| \"I need to check with the GM / owner.\" | \"Can we book 20 minutes with them this week? I'll show them the one-page version.\" |\n| \"Let's talk in Q1.\" | \"What changes in Q1? Could we get one upcoming event into the system now so you're not starting cold?\" |\n| \"How is this different from our PMS?\" | [Fill in: what your PMS does not do for groups, in one sentence] |\n\n## Win/loss capture\n\nDo this before anything else, since you have no record of why deals went the way they did.\n\n- **The 5 wins:** 30-minute call. Ask what was happening the month before they looked, what else they considered, what made them say yes, who pushed it internally, and what almost stopped it.\n- **The 8 silent:** one email. \"We're trying to learn why hotels don't move forward. Which fits best: timing, price, not a fit for groups, the team didn't see the value, or something we did wrong?\" Reply options keep it to one word.\n- **The 9 warm leads:** ask what would need to be true to decide within 30 days.\n\nLoss reason codes, used in `pipeline.csv`: `timing`, `price`, `no-pain`, `no-gm-buy-in`, `spreadsheet-is-fine`, `went-quiet`, `other`.\n\n## Weekly review (Friday, 30 minutes)\n\n- Count open deals by stage.\n- Fix every open deal with no dated next step.\n- Decide on every deal untouched for 10+ days: push, or move to Nurture.\n- Review demos held this week and what changed in the pitch.\n- Tally loss reasons. After 8 to 10 losses, look for a pattern.\n\nTrack demos per week, discovery-to-demo %, demo-to-won %, and median days to close. Compare against the 22 historical demos once you have entered them.\n\n## Handover to a first salesperson\n\nBefore hiring, the process needs to work without you in the room:\n\n1. Run at least 10 deals through this process yourself and fix any stage that does not hold up.\n2. Log stage conversion for at least two months.\n3. Write down the three messages and the demo flow that get the most next steps booked.\n4. Enter the 22 historical demos from Notion into `pipeline.csv`, with outcomes.\n\nA new salesperson should be able to run the process from this document alone.\n"}]}}, "verdicts": [{"vs": "haiku", "pair": "1", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "haiku", "why": "B provides a more actionable solo-selling routine and handover materials. A explicitly excludes the requested transfer plan. Both incompletely address target economics and prospect learning; B also contradicts its demo-volume schedule and follow-up deadlines."}, {"vs": "haiku", "pair": "1", "judge": "gpt", "order": "haiku shown first", "winner": "haiku", "why": "A provides a more actionable solo-sales plan and handover materials. Its pipeline forecasts are speculative and follow-up rules conflict. B acknowledges conversion uncertainty but delays outreach, lacks follow-up discipline, and explicitly excludes the requested transfer plan."}, {"vs": "haiku", "pair": "1", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "tie", "why": "Both provide a complete, actionable plan fitting the CEO's context, with accurate math and clear next steps."}, {"vs": "haiku", "pair": "1", "judge": "deepseek", "order": "haiku shown first", "winner": "haiku", "why": "A provides a complete, actionable plan with immediate steps, while B is overly analytical and lacks concrete actions."}, {"vs": "opus", "pair": "1", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "opus", "why": "B supplies usable tracking, differentiated follow-up, interviews, and a founder selling routine. Its growth assumptions remain speculative and handover evidence incomplete. A delays outreach, omits revenue arithmetic, and explicitly excludes the requested transfer plan."}, {"vs": "opus", "pair": "1", "judge": "gpt", "order": "opus shown first", "winner": "opus", "why": "A provides an executable solo-founder process, differentiated follow-up, buyer interviews, and handover materials. Its growth forecast relies on unvalidated assumptions. B omits MRR arithmetic, weakens prospect follow-up, delays outreach, and explicitly excludes the requested salesperson handover."}, {"vs": "opus", "pair": "1", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "tie", "why": "Both deliver a concrete, founder-appropriate plan, correct math, learning methods, tracking, and handover materials."}, {"vs": "opus", "pair": "1", "judge": "deepseek", "order": "opus shown first", "winner": "opus", "why": "A provides a concrete, actionable plan tailored to a solo CEO, while B is overly analytical and lacks immediate, practical steps."}, {"vs": "opus", "pair": "2", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "opus", "why": "B provides more usable outreach, follow-up rules and a solo-selling routine. Both miss the 20-hotel/$8,000 MRR endpoint and explicit Q1 verification. A understates required demos; B relies on unvalidated pipeline conversion assumptions."}, {"vs": "opus", "pair": "2", "judge": "gpt", "order": "opus shown first", "winner": "opus", "why": "A offers more actionable sourcing, follow-up scripts, buyer research, and a solo selling routine. Both omit total target MRR and explicit Q1 verification. A's pipeline conversion assumptions need validation; B rounds demo requirements down and lacks an unresponsive-deal parking rule."}, {"vs": "opus", "pair": "2", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "tie", "why": "Both provide excellent, tailored, actionable plans meeting all rubric criteria."}, {"vs": "opus", "pair": "2", "judge": "deepseek", "order": "opus shown first", "winner": "opus", "why": "A is more actionable, with a concrete weekly plan, immediate pipeline steps, and clear handover materials. B is overly academic and lacks specific actions."}, {"vs": "haiku", "pair": "2", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "haiku+skill", "why": "A better connects sourcing, learning, cycle timing and evidence to handover. Both omit MRR totals, round prematurely to 65 demos, and inadequately clarify Q1. B assumes unsupported warm-pipeline conversion and sets activity targets insufficient for its goal."}, {"vs": "haiku", "pair": "2", "judge": "gpt", "order": "haiku shown first", "winner": "haiku+skill", "why": "B offers stronger sourcing, customer-outcome learning, and evidence-based handover. Both omit MRR totals, round demo requirements down, inadequately investigate delayed prospects, and fail to clarify Q1 timing. B also lacks an explicit rule for parking unresponsive deals."}, {"vs": "haiku", "pair": "2", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "tie", "why": "Both provide a complete, tailored plan meeting all rubric criteria for a solo CEO, with accurate math and actionable steps."}, {"vs": "haiku", "pair": "2", "judge": "deepseek", "order": "haiku shown first", "winner": "tie", "why": "Both provide excellent, actionable, and tailored plans meeting all rubric criteria."}, {"vs": "haiku", "pair": "3", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "haiku+skill", "why": "A handles uncertainty and pipeline evidence better. B double-counts existing prospects in its forecast and incorrectly totals stage durations. Both lack a concrete weekly prospecting routine, complete tracking fields, and an evidence-based handover plan."}, {"vs": "haiku", "pair": "3", "judge": "gpt", "order": "haiku shown first", "winner": "haiku+skill", "why": "B better connects buyer interviews, measured outcomes, recorded findings, and sales adjustments. Both lack a sustainable acquisition routine and complete handover plan. A double-counts potential pipeline wins; B omits MRR totals and treats silent prospects as resolved."}, {"vs": "haiku", "pair": "3", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "tie", "why": "Both provide excellent, practical, and complete plans fitting the CEO's context and the rubric."}, {"vs": "haiku", "pair": "3", "judge": "deepseek", "order": "haiku shown first", "winner": "haiku", "why": "A provides a concrete, actionable plan tailored to a solo CEO, while B is overly analytical and defers key actions."}, {"vs": "opus", "pair": "3", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "opus", "why": "B supplies a usable solo routine, segmented follow-up, focused interviews, and concrete handover materials. Its conversion assumptions need stronger uncertainty treatment and ROI needs contribution margins. A omits revenue totals, underspecifies prospecting, and prematurely defers handover preparation."}, {"vs": "opus", "pair": "3", "judge": "gpt", "order": "opus shown first", "winner": "opus", "why": "A provides actionable solo selling, differentiated follow-up, interviews, and handover materials. Its growth math understates uncertainty. B incorrectly treats silent prospects as resolved, omits MRR totals, restricts prospecting prematurely, and defers the requested handover preparation."}, {"vs": "opus", "pair": "3", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "tie", "why": "Both provide a complete, tailored, and actionable plan meeting all rubric criteria."}, {"vs": "opus", "pair": "3", "judge": "deepseek", "order": "opus shown first", "winner": "opus", "why": "A provides a concrete, actionable plan with weekly routine, templates, and pipeline management. B is overly theoretical and defers key actions."}]}, {"id": "sal-3", "family": "founding-sales", "prompt": "Write me a founder-led sales playbook. Our situation is in context.md.", "input": "context.md", "skill": "kazanjy-founding-sales", "input_text": "Product: Patchnote, an API that turns product changelogs into in-app release notes and emails, for B2B SaaS companies.\nTraction: 300 free sign-ups (mostly developers), 6 paying teams (USD 99-499/month). Paying teams are product managers at 50-300 person SaaS companies.\nProblem: free users never upgrade; the paid ones came from me DMing PMs on LinkedIn.\nI'm a solo founder with 10 hours a week for sales. I hate cold calling.\nGoal: USD 20k MRR in 6 months (currently USD 1.4k).\n", "rubric": [{"id": "c1", "criterion": "Targets buyers supported by Patchnote\u2019s actual sales evidence.", "full_marks_when": "Prioritizes product managers at 50\u2013300-person B2B SaaS companies, defines observable prospect-selection and qualification signals, and treats developers as potential technical evaluators or introducers. Recognizes that founder LinkedIn DMs produced paying customers and proposes investigating free-user non-conversion without assuming its cause.", "zero_when": "Centers the playbook on converting all 300 free sign-ups, targets developers as the proven buyers, or invents a reason free users do not upgrade."}, {"id": "c2", "criterion": "Makes Patchnote\u2019s value clear enough to support a relevant sales conversation.", "full_marks_when": "Connects changelog-to-in-app-release-note-and-email functionality to plausible PM needs, such as keeping customers informed and reducing release-communication work. Supplies usable positioning and discovery questions that test these needs, and distinguishes hypotheses from established customer evidence.", "zero_when": "Provides generic SaaS messaging, misrepresents the product, or invents customer results, capabilities, integrations, or quantified benefits."}, {"id": "c3", "criterion": "Provides an executable prospecting approach and outreach assets suited to the founder.", "full_marks_when": "Builds on LinkedIn outreach with concrete prospect-finding instructions, personalization cues, ready-to-adapt initial messages, follow-ups, and a clear meeting request. Sets a bounded follow-up cadence and respects the founder\u2019s dislike of cold calling.", "zero_when": "Requires cold calling, substitutes a broad marketing strategy for direct sales, or merely recommends networking and outreach without actionable instructions or messages."}, {"id": "c4", "criterion": "Explains how to turn a prospect conversation into a paying, successfully activated team.", "full_marks_when": "Gives practical qualification questions, a discovery/demo structure, responses to likely objections, and explicit next steps for advancing or disqualifying a deal. Addresses both PM ownership and developer involvement in API implementation, with clear trial or evaluation success conditions, payment decisions, and onboarding steps.", "zero_when": "Stops at booking meetings, assumes a demo automatically closes a deal, or ignores the technical implementation required to use an API product."}, {"id": "c5", "criterion": "Fits the entire sales operation into a solo founder\u2019s 10-hour weekly budget.", "full_marks_when": "Allocates no more than 10 hours per week across prospect research, outreach, follow-ups, meetings, and sales-related onboarding or administration. Provides a repeatable weekly routine and six-month progression, with activity volumes compatible with the time budget and explicit priorities when demand exceeds capacity.", "zero_when": "Requires a sales team, exceeds the available time, or prescribes high activity volumes without accounting for research, meetings, follow-up, and implementation support."}, {"id": "c6", "criterion": "Uses correct revenue arithmetic and makes the growth plan measurable without presenting assumptions as facts.", "full_marks_when": "Correctly states that growing from USD 1,400 to USD 20,000 MRR requires USD 18,600 additional net MRR, averaging USD 3,100 per month over six months. Any customer-count scenario is consistent with its assumptions: retaining current revenue requires at least 38 additional teams at USD 499/month or 188 at USD 99/month; the current six teams average approximately USD 233.33/month. Separates observed results from assumed deal sizes, conversion rates, sales cycles, and churn, and supplies funnel tracking and checkpoints for adjusting the plan if progress falls short.", "zero_when": "Misstates the input figures, makes material arithmetic errors, confuses recurring revenue with bookings or cumulative revenue, or claims the goal is assured using unsupported funnel assumptions."}], "runs": {"haiku-skill-1": {"cost_usd": 0.02191, "seconds": 136, "output_tokens": 27913, "files": [{"name": "founder-sales-playbook.md", "text": "# Patchnote Founder-Led Sales Playbook\n\nBased on Peter Kazanjy's public method. Not affiliated with or endorsed by Peter Kazanjy.\n\nEvidence labels: **Observed** (from context.md), **Reported** (founder's account, not yet checked against records), **Hypothesized** (to test), **Unknown** (needed, not in hand).\n\nDrafted 2026-10-08. The plan covers the next 8 weeks with a review at week 8 (2026-12-03). The 6-month horizon ends 2027-04-08.\n\n---\n\n## 1. Situation and stage\n\n**Product:** Patchnote, an API that turns product changelogs into in-app release notes and emails for B2B SaaS. Price range USD 99 to 499 per month.\n\n**Buyer seen so far:** Product managers at 50 to 300 person SaaS companies.\n\n| Item | Status | Label |\n| --- | --- | --- |\n| 6 paying teams, USD 99 to 499 per month, total MRR USD 1.4k | Observed | Observed |\n| 300 free sign-ups, mostly developers | Observed | Observed |\n| Free users do not upgrade | Reported | Reported |\n| The 6 paying teams came from founder LinkedIn DMs to PMs | Reported | Reported |\n| Founder sales capacity: 10 hours per week | Observed | Observed |\n| DMs sent, reply rate, calls held, sales cycle length | Not recorded | Unknown |\n| Whether the 6 teams use the product, are live, or see an outcome | Not recorded | Unknown |\n| Who signed off on each purchase (PM alone or with an approver) | Not recorded | Unknown |\n| Pain the 6 described in their own words | Not recorded | Unknown |\n\n**Stage assessment** [S2]:\n\n- **Problem discovery:** Partly shown. Six buyers paid, so some pain exists for some PMs. No documented pain statements are on file.\n- **Value exchange:** Shown for six accounts (payment). Procurement and implementation details are unknown.\n- **Value creation:** Unknown. This is the earliest incomplete stage. No baseline or outcome exists for any of the six.\n- **Repeated exchange:** Not shown. Six sales came through one founder-run channel, and no funnel denominators were recorded.\n- **Seller transfer:** Not applicable yet.\n\n**Next proof:** Confirm that the founder-led channel repeats in a defined segment. Confirm the six teams get value in the same window, because repeat sales built on churning customers will not hold.\n\n**Free tier:** The 300 sign-ups are a list of prospects, not a conversion funnel that has been shown to work. Developers are likely the users, while PMs likely own the outcome and budget (Hypothesized). For this 8-week period, route free sign-ups at 50 to 300 person companies to the PM by hand. Do not build self-serve upgrade work yet.\n\n---\n\n## 2. The goal arithmetic\n\nThe goal is USD 20k MRR in 26 weeks. Current MRR is USD 1.4k across 6 teams, an average of about USD 233 per team.\n\n| Assumed price per team | Teams needed for USD 20k MRR | Net new teams from today's 6 |\n| --- | --- | --- |\n| USD 233 (current average) | 86 | 80 |\n| USD 400 | 50 | 44 |\n| USD 499 (top of current range) | 41 | 35 |\n| USD 800 (hypothetical higher tier) | 25 | 19 |\n\nThese figures exclude churn. Any churn adds to the requirement.\n\nFounder time over 26 weeks is about 260 hours. At USD 400 per team, that is about 6 founder hours per new paying team, including prospecting, calls, and onboarding. At the current average, it is about 3 hours per team.\n\n**Funnel check.** At USD 400 per team, the goal needs about 1.7 new paying teams per week. These conversion rates are illustrations, not measurements:\n\n| Stage | Assumed rate | Volume per week |\n| --- | --- | --- |\n| New paying teams | n/a | 1.7 |\n| Proposal to close | 40% | 4.3 proposals |\n| Call to proposal | 40% | 10.6 calls |\n| Reply to call | 50% | 21 replies |\n| DM to reply | 20% | 106 DMs |\n\nAt 106 personalized DMs per week, outreach alone would take most of the 10 hours before any calls or onboarding. Under these rates, the goal does not fit the time budget. The real rates from the next 8 weeks will show whether that is the binding constraint.\n\n**What moves the goal most:** Price and packaging, more than outreach volume. Doubling the average price halves the number of closes needed. A higher tier is a hypothesis that needs evidence from the existing 6 and from new prospects. Test it, but do not assume it.\n\n---\n\n## 3. The next 8 weeks\n\n**Experiment:** Founder-led outreach to a defined PM segment, with a measured funnel. In parallel, measure value for the existing 6.\n\n- **Owner:** Founder.\n- **Target segment:** PMs at 50 to 300 person B2B SaaS companies that publish a changelog or release notes. This is a hypothesis; week 1 refines it.\n- **Artifacts:** The outreach variants (Section 6), the discovery guide (Section 7), and the record sheet (Section 11).\n\n**Schedule:**\n\n- **Week 1 (by 2026-10-15):** Pull the data in Section 15. Interview all six existing teams for 30 minutes each. These are warm conversations, so no cold calls are needed. Identify the traits the six share.\n- **Weeks 2 to 8:** Work 20 qualified accounts per week. Log every touch. Hold discovery calls only with people who replied. Demo only after pain is acknowledged.\n- **Week 4 review (2026-11-05):** Check funnel rates by stage and variant. Revise the segment or message if a stage is weak.\n- **Week 8 review (2026-12-03):** Decide among three options: continue and re-base the goal, test a higher price tier, or add a channel with more leverage.\n\n**Exit condition for the week 8 review:**\n\n- Every touch has a recorded count, so each rate has a numerator and a denominator.\n- At least 6 new paying teams match the ICP. This matches the size of the existing base. The number is a package choice. Its purpose is to show the segment repeats, not that the first customers were lucky.\n- Every new customer has a baseline and an outcome measure recorded before go-live.\n\nIf the condition fails, find the stage where the funnel breaks (Section 13) before adding volume.\n\n---\n\n## 4. Ideal customer profile\n\n**Organization (verify each):**\n\n- B2B SaaS with a product that users log into, so an in-app surface exists.\n- 50 to 300 employees.\n- Ships releases regularly and writes changelog entries. A public changelog or release notes page is a clue, not proof.\n- Customers need to hear about changes (Hypothesized).\n\n**Exclusions:**\n\n- Consumer apps, or products with no in-app surface.\n- Teams whose changelog is internal only.\n- Companies already running an in-app announcement tool they are satisfied with. Do not invent a gap for them.\n- Companies outside the observed range of 50 to 300 employees.\n\n**People:**\n\n- **User:** A developer who integrates the API. This may be the free sign-up.\n- **Champion:** The product manager. Observed as the buyer so far.\n- **Operational decision maker:** Unknown. Confirm whether the PM can approve purchases alone.\n- **Budget authority:** Unknown. Likely the head of product or whoever approves SaaS spend. Confirm in discovery.\n\n**Current approach:** Unknown. Hypothesis: someone manually copies changelog text into in-app banners or customer emails, or skips it.\n\n**Timing:** Unknown. Possible timing signals to test: a higher release cadence, a major launch, or a new PM hire. These are hypotheses. Fit does not establish timing.\n\n---\n\n## 5. Master narrative\n\nDraft, to be confirmed in discovery:\n\n- **Current state:** After each release, someone writes a changelog entry. Someone else rewrites it for the app and for customer emails, often later or not at all.\n- **Problem:** Customers miss what shipped, so features go unused and the launch effort is wasted. Consequences to test: adoption, support questions, and time spent per release.\n- **Alternatives:** Manual copying, generic in-app announcement tools that still require writing, or doing nothing.\n- **Proposed change:** Patchnote generates in-app release notes and emails from the changelog the team already writes, via the API.\n- **Evidence:** Six paying teams (Observed). No outcome evidence yet. Do not claim a result until the baseline in Section 8 reports.\n- **Why now:** Unknown until discovery.\n- **Commercial proposal:** The current plans, with a written success plan and a baseline agreed before go-live.\n\n**One-sentence version (untested):** \"Patchnote turns the changelog your team already writes into in-app release notes and customer emails, so releases reach customers without a second round of writing.\"\n\n**Claims to avoid until measured:** percentage savings, \"50% less work,\" and any outcome figure. The early messaging described in context.md is an example of a claim that outran its evidence.\n\n---\n\n## 6. Outreach\n\n**Rules:**\n\n- One verified fact or one clearly stated hypothesis per message.\n- One question about the current process.\n- No product pitch in the first message.\n- Offer a conversation, not a demo.\n- One follow-up after 5 business days, then stop. This is a package convention.\n\n**Variant A: PM with a public changelog (verified fact)**\n\n> Hi [Name], [Company]'s changelog lists [N] releases since [month]. How do your customers hear about those today: email, in-app, or only the changelog page?\n\n**Variant B: PM without a verified fact (hypothesis)**\n\n> Hi [Name], I'm researching how product teams at 50 to 300 person SaaS companies tell customers about releases. Would you be open to a 15-minute conversation? I'll share what I learn.\n\n**Variant C: Developer who signed up for free (route to the PM)**\n\n> Hi [Name], thanks for trying Patchnote. Who on your product side owns what customers hear about releases? I'd like to speak with them.\n\nThese drafts are untested. Record which variant each prospect received.\n\n---\n\n## 7. Discovery call (20 minutes, only after a reply)\n\n**Purpose:** Learn the current process and its consequences. Decide whether a demo makes sense.\n\n1. **Opening (1 min):** \"You replied about [topic]. I'd like to understand how releases reach your customers today. If it doesn't fit, we can stop early.\"\n2. **Current process (5 min):** \"Walk me through your last release. What happened after the changelog was written?\"\n3. **Concrete incident (5 min):** \"Tell me about a release where customers didn't learn about a feature, or the announcement came late. What happened? How often does that happen?\"\n4. **Consequence and priority (5 min):** \"What did that cost in time, support questions, or adoption? Who noticed? Where does fixing it sit on your list this quarter?\"\n5. **Buying (3 min):** \"If this were worth fixing, who else would need to be involved? What would they need to see?\"\n6. **Close (1 min):** Ask for one specific next step, or record \"not now\" with a re-contact date.\n\n**Branches:**\n\n- **Current tool works:** Ask what falls short. If nothing does, thank them and close. Do not manufacture a gap.\n- **Manual workaround:** Ask who does it, how often, and what happens when it is skipped.\n- **Pain acknowledged, low priority:** Ask what would change the priority and whether a real event is expected. Set a re-contact date.\n- **No budget authority:** Ask how decisions get made and what the approver would need to see.\n- **Developer with no pain:** Ask who owns release communication, then route to that person.\n\nDo not show screens before summarizing the acknowledged problem and getting agreement [S3].\n\n---\n\n## 8. Demo and commitments\n\n**Demo (15 minutes):** Use one real or sample changelog entry from the prospect. Show it becoming an in-app note and an email. Show only capabilities that exist today.\n\n**Commitment ladder** [S2]:\n\n| Step | Ask | What it counts as |\n| --- | --- | --- |\n| Reply | Discovery call | Interest, not progress |\n| Discovery | Demo with the named approver | Problem acknowledged |\n| Demo | Evaluation with written success criteria, a baseline, and a decision date | Evaluation, not payment |\n| Evaluation | Paid plan | Value exchange (payment) |\n| Paid | Integration live | Implementation |\n| Live plus 30 days | Customer confirms the measured outcome | Value creation for that account |\n\nReport paid, live, and validated counts separately.\n\n**Evaluations:** No open-ended free pilots. Any evaluation needs written success criteria and a decision date. A free evaluation does not count as payment.\n\n**Baseline before go-live:** Releases per month, hours per release spent on notes and emails, the share of customers reached by announcements, and support questions about new features. Mark each figure as measured or estimated.\n\n---\n\n## 9. Customer success for the existing six (weeks 1 to 4)\n\nFor each of the six:\n\n- Confirm whether they are live and how they use the product.\n- Ask: \"If Patchnote stopped working, who would notice, and what would they do?\" This tests retention.\n- Establish a baseline, or reconstruct one from their own records, and agree a 60-day measurement plan.\n- Ask whether they would act as a reference if the result is positive.\n- Ask: \"What would make this plan worth more to you?\" This tests price.\n\nRecord for each: plan, start date, live status, usage, the pain they described, baseline, outcome, and reference permission.\n\n---\n\n## 10. Economics\n\n**Capacity value** = releases per month \u00d7 hours per release on notes and emails \u00d7 loaded hourly cost.\n\n**Illustration (assumed inputs, replace with discovery figures):** 4 releases per month \u00d7 2 hours \u00d7 USD 60 = USD 480 per month of released capacity.\n\n**Reading the illustration:** USD 480 is released capacity, not cash. At USD 400, the plan consumes about 83% of that estimate. At USD 499, it consumes about 104%, so the top plan needs a stronger case: more releases, measured adoption gains, or budget that is already set aside for this work.\n\n**Rules:**\n\n- Do not add capacity value and revenue effects that come from the same time.\n- Do not state a percentage saving until it is measured.\n- Record the baseline, period, and assumptions alongside every figure.\n\n---\n\n## 11. Records\n\nOne spreadsheet, one row per account, with these columns:\n\n- Account, company size, and fit evidence (with source)\n- Persona and role (user, champion, approver, budget)\n- Source: DM variant, sign-up, or referral\n- First touch date, reply date, call date, demo date, evaluation date, paid date, go-live date\n- Pain acknowledged (yes, no, or unclear), with a short quote\n- Current approach, objections, and next step with owner and date\n- Baseline, outcome measure, 30-day result, and validation status\n\nKeep denominators: touches, replies, calls, demos, proposals, paid, live, and validated, each with its period and segment.\n\n---\n\n## 12. Weekly cadence (10 hours)\n\n| Block | Hours | Activity |\n| --- | --- | --- |\n| Research and qualify | 2 | Build the next 20 accounts |\n| Outreach and follow-ups | 2 | Send DMs and follow-ups |\n| Discovery calls | 2 to 3 | Only with people who replied |\n| Customer success | 2 | Existing and new paying teams |\n| Records and review | 1 | Update the sheet and compute rates |\n\nThere are no cold calls. If replies are low, fix targeting or the first message. Adding call volume will not solve it.\n\n---\n\n## 13. Diagnosis at weeks 4 and 8\n\n| Symptom | Hypothesis | What to check |\n| --- | --- | --- |\n| Few replies to DMs | Targeting or message | Reply rate by variant and segment |\n| Replies but few calls | Promise or ask in the message | Call acceptance by variant |\n| Calls with no pain | Poor fit | Pain statements by segment |\n| Pain acknowledged, no progress | Priority, explanation, or authority | Next steps and named approvers |\n| Paid but little use or no outcome | Product, onboarding, or expectations | Usage, go-live dates, baselines |\n| Price objections | Value case or packaging | Capacity-value inputs against price |\n| Good outcomes, weak wins | Commercial explanation | How successful accounts are described to prospects |\n\nChange one thing at a time, and compare it against comparable conversations. A handful of calls cannot establish cause [S3].\n\n---\n\n## 14. Out of scope for now\n\n- **Hiring a VP of Sales or any seller:** Premature. No repeatable motion is documented, and a new hire would have to invent one [S2].\n- **Paid outreach or advertising:** Not tested, and not part of this plan.\n- **Self-serve upgrade automation:** Product work. Defer until the hand-routed free sign-ups show whether they are the right accounts.\n- **Outsourced appointment setting:** Possible after week 8, once the message and discovery are documented. The founder keeps discovery [S2].\n\n---\n\n## 15. Missing inputs (complete in week 1)\n\n1. How many DMs produced the six, with dates and reply counts.\n2. For each of the six: company size, persona, how they found Patchnote, plan, start date, usage, and whether they are still paying.\n3. Which of the 300 sign-ups work at 50 to 300 person companies, and how many are PMs or can be routed to one.\n4. Whether any paying team began as a free sign-up.\n5. Whether each purchase was approved by the PM alone.\n6. Any pain described by the six, in their words.\n7. Whether anything exists above USD 499.\n\n---\n\n## 16. Review result\n\n| Criterion | Status | Note |\n| --- | --- | --- |\n| Scope | Pass | 8-week sales experiment; hiring excluded with reason |\n| Maturity | Pass | Earliest incomplete proof named: value creation for the six |\n| Evidence integrity | Pass | Labels on each key fact |\n| Next experiment | Pass | Owner, segment, artifacts, dates, and exit condition |\n| Customer profile | Pass (hypothesis) | Not yet confirmed by interviews |\n| Timing | Open | No timing evidence; labelled unknown |\n| Narrative | Partial | Hypothesis until the six interviews |\n| Economics | Partial | Illustrative inputs; replace them |\n| Discovery | Pass | Branches follow answers |\n| Solution framing | Pass | Demo only after pain is acknowledged |\n| Commitment | Pass | Ladder separates evaluation from payment |\n| Record quality | Pass once the sheet is built | Fields defined |\n| Customer success | Open | No outcome data for any customer |\n| Diagnosis | Partial | No funnel data yet |\n| Metrics | Open | Six customers and no recorded denominators |\n| Transfer, seller fit, coaching, balanced scale | Not applicable | Premature |\n| Attribution | Pass | Method claims tied to [S2] and [S3] |\n\n**Status:** Usable as the 8-week experiment plan. The three open items are data-collection tasks for week 1 and will be closed or flagged at the week 4 review.\n\n**Immediate next action:** Founder pulls the Section 15 data and books the six existing-customer interviews by 2026-10-15.\n"}]}, "haiku-skill-2": {"cost_usd": 0.01548, "seconds": 83, "output_tokens": 16258, "files": [{"name": "sales-playbook.md", "text": "# Patchnote Founder-Led Sales Playbook\n\nDraft v1, 2026-10-08. Built from context.md. Evidence labels: **(observed)** from data you gave me, **(reported)** from your account, **(hypothesis)** to test, **(unknown)** not yet known.\n\n## 1. Where we are\n\n- **(observed)** 300 free sign-ups, mostly developers. Free users have not converted to paid.\n- **(observed)** 6 paying teams at USD 99\u2013499/month. MRR USD 1.4k, so the average is about USD 233 per team.\n- **(reported)** All 6 paid teams came from your LinkedIn DMs to product managers at 50\u2013300 person SaaS companies.\n- **(unknown)** How many DMs and calls produced the 6, how long each deal took, which tier each bought, churn, and whether any team has measured a benefit.\n\n**Stage:** Value exchange is partly shown. Six teams have paid. Two earlier questions remain open: does a paying team get a measured benefit, and can you find buyers through a channel other than your own DMs? The earliest unproven item is the first, so start there. Run a fixed outbound batch alongside it.\n\n**Free-funnel hypothesis:** Developers evaluate the API. Product managers own release communication and usually the budget. A developer can sign up and leave without a PM ever seeing the output. **(hypothesis)** Test this; do not assume it.\n\n## 2. The goal math\n\nYou need +USD 18.6k MRR. Ignoring churn:\n\n| Average price per team | Teams needed (total) | New teams per week over 26 weeks |\n| --- | --- | --- |\n| USD 233 (today's average) | ~86 | ~3.1 |\n| USD 300 | ~68 | ~2.4 |\n| USD 499 (top tier) | ~43 | ~1.4 |\n\nSix teams is your starting base. At 10 hours a week, even the top-tier path needs a large jump in pace from anything you have shown so far.\n\n**Implication:** The USD 20k in 6 months is unlikely on the current motion and time budget. Two levers have to move: which tier buyers choose, and how many qualified conversations you can run each week. Keep USD 20k as the aspiration. Set the 6-week milestone in section 10, and decide on the 6-month target at week 6 using real conversion data.\n\n## 3. Target customer (ICP v1, to be replaced in week 1)\n\n- **Organization:** SaaS company with 50\u2013300 employees, shipping product changes on a regular cadence, sending customer release notes or emails, and having someone who writes them by hand.\n- **Champion (primary DM target):** Product manager or head of product who owns release communication.\n- **Integrator (secondary):** Developer who set up the API. Often the route to the champion.\n- **Budget authority:** Unknown. Ask on the first call.\n- **Exclusions:** Releases are rare; release notes are not customer-facing; company is outside 50\u2013300 unless one of your 6 paid teams fits outside that range; the team already has a tool that handles this and they are satisfied.\n- **Timing signals (hypothesis):** A recent major launch, a PM or release-manager job posting, or a changelog that has gone quiet. These are reasons to open a conversation, not proof of need.\n\n## 4. Pipeline sources, in priority order\n\n1. **Your 6 paying teams.** Ask each for one introduction, but only after you have checked their value (section 8).\n2. **Free sign-ups at the company level.** Find companies where a free account has published changes or made API calls. Then find the PM at that company using the company, not the developer's personal contact details. **(hypothesis)** This is warmer than cold outreach.\n3. **LinkedIn DMs to PMs who match ICP v1.** This is the channel that produced all 6 **(reported)**.\n\nDo not cold call. Do not buy lead lists until you know your DM-to-paid conversion rate.\n\n## 5. Weekly time budget (10 hours)\n\n| Block | Hours | Output |\n| --- | --- | --- |\n| Research and DMs | 3 | 25 personalized DMs |\n| Discovery calls | 2.5 | 5\u20136 calls of 20\u201325 minutes each |\n| Follow-up, trial setup, CRM | 1.5 | Next steps set, trials started, records updated |\n| Customer value work | 3 | Outcome check-ins with paying teams, onboarding fixes |\n\nThe 3 hours of customer value work matter most. They produce the outcome evidence and referrals the rest of the motion depends on.\n\n## 6. Outreach\n\n**Rules:** Use one verified fact per message. Ask about their current process. Do not pitch in the first message. Keep it under five sentences. Send one follow-up after four business days, then stop.\n\n**Template A (verified release):**\n> Hi [Name], I saw [company] shipped [specific release] on [date]. After releases like that, are release notes and customer emails still written by hand? We turn a product changelog into in-app notes and emails. Would a 20-minute conversation about how you handle it be useful?\n\n**Template B (free-account activity; use only what you can verify at the company level):**\n> Hi [Name], a developer at [company] has been testing Patchnote [only if verified and appropriate]. Who on your team owns release communication? I'd like to learn how you handle it today.\n\n**If they say it's not relevant:** Ask one question: \"What do you use now?\" Record the answer and close the loop.\n\n## 7. Discovery call (20 minutes, only after they reply)\n\n**Goal:** Confirm or rule out the pain. If confirmed, ask for a paid start. Do not demo in the first 10 minutes.\n\n**Opener:** \"Walk me through your last release. From the moment it shipped to the moment customers heard about it, what happened?\"\n\n**Core questions:**\n- Who writes the release notes and emails? How long does it take per release?\n- What happened the last time one was late or missed?\n- What have you tried, and why did it not stick?\n- How often do you ship, and who signs off?\n- If this were solved, what would change for you or your team?\n- Who else needs to agree before a tool gets paid for?\n\n**Branches:**\n- **Current tool works:** Ask what is still unsatisfying. If nothing, thank them and close.\n- **Manual process:** Get the last incident and the hours it took.\n- **Pain acknowledged, low priority:** Ask what would change the priority. Set a re-engage date.\n- **No budget authority:** Ask how the decision gets made. Ask for 15 minutes with that person, or a short summary they can forward.\n\n**Close the discovery:** Summarize the problem in their words. Ask, \"Is that the problem you want fixed?\" Only then show one flow: their last release turned into an in-app note and an email preview. Stop there.\n\n**Commitment ask:** Choose one based on what you learned: \"Start on [tier] today\" or \"Start a paid 30-day trial with a success check on day 30.\" Confirm who signs, which tier, and when the team starts.\n\n**Objections:**\n- *\"We already do this in [tool].\"* \u2192 \"What breaks in that process?\"\n- *\"Our engineers can build it.\"* \u2192 \"How many engineer-hours per quarter would that take to maintain?\"\n- *\"Too expensive.\"* \u2192 Compare the price with hours per release times their hourly cost, labelled as an estimate.\n- *\"Not now.\"* \u2192 \"What would make it a priority?\" Record the answer and a date.\n\n## 8. Value and economics\n\n- **Baseline before the purchase:** Hours per release spent on notes and emails, releases per month, and how often customers are notified late or not at all.\n- **Estimate:** Hours per month times an hourly labor rate gives a capacity value. Call it capacity, not savings, unless headcount or spend actually changes.\n- **Validation at day 30:** Ask the customer for actual hours, whether notes shipped on time, and any feedback.\n- **Claims:** Publish an outcome only when the customer approves it in writing.\n\n## 9. Records and weekly review\n\n**Per account, minimum fields:** source, persona, first-touch date, touches, reply, call date, problem in their words, roles, tier offered, objection, next action, owner, due date, baseline, outcome.\n\n**Weekly metrics, with denominators:** DMs sent, replies (reply rate), calls held, paid starts, paid conversions, outcome check-ins completed.\n\n**Friday review (30 minutes):** Look at the numbers by segment. Decide what to change next week, and change one thing at a time.\n\n## 10. Six-week plan\n\n- **Week 1:** Interview all 6 paying teams for 30 minutes each. Capture their titles, company size, tier, how they found you, how release comms worked before, what outcome they see, and who approved the purchase. Output: ICP v2, value wording in their language, and a baseline. Also pull the 300 free sign-ups grouped by company.\n- **Weeks 1\u20136:** Send 25 DMs per week, split into two segments: A (ICP v1 from LinkedIn) and B (free-sign-up companies). Track reply rate by segment.\n- **Weeks 2\u20136:** Run discovery calls for every reply that qualifies.\n- **Week 4:** Compare segments by qualified calls per 25 DMs. Keep the better one and drop the other.\n- **Week 6:** Decide on the 6-month target and the next channel test.\n\n**Set the milestone:** Ask how many DMs produced the first 6 paid teams. Use that ratio to set a realistic number of new paid teams for weeks 1\u20136.\n\n**Decision rules (starting points, not validated thresholds; revise with your data):**\n- If the 6 teams show no measured benefit, pause outbound and fix onboarding and measurement first.\n- If calls happen but prospects do not name a problem after about 10 qualified calls, change the ICP, not the message.\n- If prospects name the problem but do not start paying after about 10 calls, test the commitment ask and tier, and confirm the budget authority.\n- If paid starts happen and the value holds, increase DM volume and consider outside help for research or drafting. Keep selling and discovery with you.\n\n**Hiring:** None for now. Consider a first seller only after you have repeat wins and written steps for the motion. A new hire now would inherit a motion nobody has proven.\n\n## 11. Open questions for you\n\n1. Who are the 6 paying teams: title, company size, tier, and how they found you?\n2. How many DMs and calls produced the first 6, and over what period?\n3. What tier does each team pay, and has anyone churned?\n4. What outcome has each team reported, and in what form?\n5. Do the 6 match ICP v1, or do they point to a different buyer?\n6. Who approved the purchase in each deal?\n7. Do any free sign-ups have a PM or release owner on the account?\n\n---\n\n*Method basis: Peter Kazanjy's public founding-sales method. Not affiliated with or endorsed by Peter Kazanjy. Numbers in section 2 are arithmetic from your figures; the decision thresholds in section 10 are starting points for this business, not validated benchmarks.*\n"}]}, "haiku-skill-3": {"cost_usd": 0.01549, "seconds": 87, "output_tokens": 17217, "files": [{"name": "founder-sales-playbook.md", "text": "# Patchnote Founder-Led Sales Playbook\n\nBased on Peter Kazanjy's public founder-sales method. Not affiliated with or endorsed by him. The stage logic comes from his method; the templates, economics, and weekly rhythm are working conventions for this business.\n\nOwner: founder. Written: 2026-10-08. Review: every Friday, full review at weeks 4, 8, 13, and 26.\n\n## 1. Where we are\n\n| Stage | Status | Evidence |\n| --- | --- | --- |\n| Problem discovery | Partly done | Observed: PMs responded to founder LinkedIn DMs and six of them paid. Unknown: why the other sign-ups did not engage. |\n| Value creation | Unproven | No measured customer outcome in our records. Unknown: hours saved, notes shipped, or any before/after data. |\n| Value exchange | Achieved, small scale | Observed: 6 paying teams at USD 99-499 per month. |\n| Repeated exchange | Not shown | Observed: all six came through one founder-led channel. No other channel has produced a paid team yet. |\n| Seller transfer | Premature | One founder, 10 hours per week. Nothing to transfer yet. |\n\n**Earliest missing proof:** measured value for the six paying teams. A paid deal does not prove delivery, and we need the outcome data for our own outreach anyway.\n\n**Biggest commercial gap:** repeatable acquisition of the right buyer through a channel that does not depend on the founder's personal DMs.\n\n## 2. The math\n\nCurrent MRR is USD 1.4k across 6 teams, an average of about USD 233 per team. The target is USD 20k MRR in about 26 weeks. Churn is ignored here, so these are optimistic.\n\n| Average price per team | Teams needed for USD 20k | Net new teams per week |\n| --- | --- | --- |\n| USD 233 (current average) | about 86 (+80) | about 3.1 |\n| USD 400 (if we move upmarket) | 50 (+44) | about 1.7 |\n\nQualified conversations needed per week, depending on the conversation-to-paid rate. The rates are unknown and are hypotheses:\n\n| Net new teams per week | at 5% | at 10% | at 20% |\n| --- | --- | --- | --- |\n| 1.7 | 34 | 17 | 9 |\n| 3.1 | 62 | 31 | 16 |\n\nAt 10 hours per week, 30 live conversations is not feasible. Reaching USD 20k in six months on founder time alone requires at least one of these: a conversion rate well above 10%, a much higher average price, a self-serve upgrade path that carries part of the load, or a later target or more hours. The first task is to measure which one is real. Do not commit to the USD 20k date until week 4.\n\n## 3. Ideal customer profile (hypothesis)\n\n**Organization:**\n- B2B SaaS company with 50-300 employees.\n- Ships product changes on a regular cadence (weekly or biweekly).\n- Writes release communication by hand: changelog, in-app announcements, customer emails.\n\n**People:**\n- **Buyer and champion:** the product manager, or whoever writes release communication.\n- **Budget authority:** unknown. Establish it in discovery. It may be the head of product, operations, or marketing.\n- **Developers** sign up for the free tier but rarely own the buying decision. Treat them as referrers unless they are also the release owner.\n\n**Observable signals (clues, not proof):**\n- Public changelog or \"what's new\" page with frequent posts.\n- In-app announcement or release-note widget.\n- Job posting for a product manager or product-marketing role.\n- Recent product launch.\n- Free sign-up from a PM-titled user at a company in range.\n\n**Exclusions:**\n- No regular releases.\n- No one owns release communication.\n- Clearly outside 50-300 employees. Record the reason if we make an exception.\n- Already has a release-notes process that works. Ask before assuming.\n\n## 4. Value argument\n\n**Current state:** after each release, the PM or a developer writes notes, drafts emails, and chases engineers for what shipped. Notes lag or get skipped.\n\n**Consequences (to verify in discovery):** customers miss features, adoption of new features is slow, support gets questions about things we already shipped.\n\n**Alternatives:** manual writing, a generic newsletter tool, a template, or doing nothing.\n\n**Proposed change:** Patchnote turns the changelog into in-app notes and emails automatically.\n\n**Illustrative economics (all inputs are assumptions until the prospect confirms them):**\n\n| Input | Assumed value | Replace with |\n| --- | --- | --- |\n| Releases per month | 4 | Prospect's own count |\n| Hours per release on notes, emails, and coordination | 5 | Prospect's estimate from their last three releases |\n| Loaded hourly rate | USD 50 | Prospect's own rate or a role benchmark |\n| Capacity value per month | USD 1,000 | Recalculated from the above |\n\nAt USD 399 per month, that is roughly 2.5 times the fee in released capacity. It is **not** cash savings. Released time becomes cash only if spend or headcount changes. Say \"capacity\" in every conversation and do not present the figure as a saving.\n\n## 5. Weekly operating rhythm (10 hours)\n\n| Block | Hours | What it covers |\n| --- | --- | --- |\n| Free-user triage | 1.0 | Segment sign-ups by role, company size, and activation. Flag ICP matches. |\n| Written outreach | 3.0 | Personalized messages to verified-signal accounts. Planning target: 15-20 per week. |\n| Discovery calls | 2.5 | Up to five 30-minute calls with prospects who replied with interest. |\n| Demos and pilot setup | 1.5 | Demo of their workflow only. Set up accounts for those who commit. |\n| Customer value check-ins | 1.0 | Baseline and day-30 measures with paying teams. |\n| Logging and review | 1.0 | Update the tracker and review the week's numbers on Friday. |\n\nThere are no phone cold calls. Outreach is written, personalized, and based on something verified. This matches what has already worked.\n\n## 6. Outreach templates\n\n**Rules:** one verified fact, one current-state question, one small ask. No pitch in the first message. Do not send anything with personalization you have not checked.\n\n**A. ICP account from a public signal:**\n\n> Subject: Release notes after [feature] launch\n>\n> Hi [Name], I saw [specific post or launch, with link]. How does your team currently get release notes into the app and to customers? I'm building Patchnote, which turns a changelog into in-app notes and emails. Would 15 minutes on how you do it today be useful? I'm happy to send the three questions in writing first.\n\n**B. Free sign-up on a developer account:**\n\n> Hi [Name], thanks for trying Patchnote with [company]. Who on your team owns customer-facing release communication? If it's a product manager, would you introduce me, or would they be open to 15 minutes?\n\n**C. Follow-up after no reply:** one follow-up, five to seven days later, that restates the question in one line. Then stop.\n\n## 7. Discovery call\n\n**Objective:** learn how release communication works today, what the last failure cost, and who decides. This is not a demo.\n\n**Opening question:** \"Walk me through your last release. Who wrote the notes, and where did they end up?\"\n\n**Branches (follow the answers; do not read them as a script):**\n\n- **Manual process, pain acknowledged:** ask for the most recent incident: a missed announcement, a support ticket, delayed notes. Ask how often it happens and how many hours it takes.\n- **Current tool works:** ask what is still unsatisfying. If there is nothing meaningful, stop and record \"not a fit now\" with the reason. Do not invent a gap.\n- **Pain acknowledged, low priority:** ask what would make it a priority, and whether a launch or hire is coming.\n- **Champion lacks budget authority:** ask how tools are approved and what the approver would need to see.\n- **Developer owns it, PM not involved:** ask who should join the next conversation.\n\n**Problem summary to confirm before any demo:** \"So each release costs your team about [X] hours, and [incident] happened last quarter. Is that right?\"\n\n**Demo:** show only the workflow they described. Ask them to send one real changelog beforehand and run that release through the product.\n\n**Commitment ask:**\n- A paid start on the tier that fits their size, with a 30-day success measure agreed before they start.\n- Baseline captured now: hours per release and releases shipped in the last three months.\n- Who approves payment, and what the procurement steps are.\n\n**If there is no commitment:** record the reason and the condition for revisiting (next major release, a hire, a budget cycle). Follow up once.\n\n## 8. Customer value\n\nApply to all six current paying teams first, then to every new one.\n\n- **Week 1:** capture the baseline. Hours per release, releases in the last three months, notes and emails written by hand.\n- **Day 30:** repeat the same measures. Add how many releases went through Patchnote and a rating of how much the team relies on it.\n- Record everything as self-reported unless measured from the product.\n- Ask for a reference or case study only after a measured result, and only with permission.\n\n## 9. Six-month plan\n\n**Phase 1, weeks 1-3: diagnose**\n- Reconstruct what produced the six: DMs sent, replies, calls, conversions, dates. Owner: founder.\n- Segment the 300 sign-ups by role, company size, and activation. Define activation (for example, at least one successful API call).\n- Interview six paying teams and six activated free users who did not convert. Twenty to thirty minutes each.\n- **Exit:** a conversion table by step, even if small; a list of activated ICP-matching free users; a written hypothesis for why free users do not convert.\n\n**Phase 2, weeks 4-8: test a second channel**\n- Run written outreach (templates A and B) at the planned volume.\n- Ask free users for introductions to their PM.\n- **Review at week 8.** Planning decision rules, to be revised with real data:\n  - Reply rate below about 5% after roughly 100 messages: change the ICP or the message before scaling.\n  - Replies but few calls: fix the ask.\n  - Calls but no paid teams: inspect discovery, the demo, and pricing.\n\n**Phase 3, weeks 9-16: scale what works and test packaging**\n- Double volume on the best-performing segment and template.\n- Test a paid 30-day pilot offer with an agreed measure for ICP-size accounts.\n- Change one price or packaging variable at a time.\n- **Checkpoint at week 13:** a linear path to USD 20k puts MRR near USD 10k. Below that, revisit the target, the hours, or the price.\n\n**Phase 4, weeks 17-26: decide on help**\n- Only if a non-DM channel has produced repeatable paid conversions: consider scoped help, such as outreach writing or appointment setting. The founder keeps discovery and the demo.\n- Do not hire a sales leader or account executives in this window. The method is explicit that transfer comes after the motion is documented and repeatable.\n\n## 10. Do not\n\n- Do not call anyone cold by phone.\n- Do not widen the ICP to \"all SaaS\" to get more volume.\n- Do not count beta, friendly, or free users as proof of paid demand.\n- Do not quote outcome figures (hours saved, percentage gains) without measured customer data.\n- Do not hire a VP of Sales to find the motion.\n- Do not build features for prospects you have not interviewed.\n\n## 11. Tracker\n\nOne row per account. Columns:\n\nDate, account, persona, source signal, message sent, reply (Y/N), call date, pain acknowledged (yes/partial/no), buying role confirmed, next step, owner, due date, proposed amount, status, baseline captured, day-30 result.\n\nWeekly totals: messages sent, replies, calls, demos, paid teams, MRR, hours spent.\n\n## 12. Inputs needed from you\n\n1. Number of DMs, replies, calls, and paid conversions behind the six, and over what period.\n2. How many of the 300 sign-ups have made an API call, and how many have a PM or other non-developer title.\n3. Company size and plan for each of the six, and any churn so far.\n4. Any measured outcome from any paying team.\n5. Whether the 10 hours per week includes product and support work.\n\nUntil these are answered, the conversion rates, the channel results, and the 20k timeline are hypotheses.\n"}]}, "opus-1": {"cost_usd": 0.31508, "seconds": 144, "output_tokens": 11246, "files": [{"name": "founder-led-sales-playbook.md", "text": "# Patchnote: Founder-Led Sales Playbook\n\n**Goal:** $1.4k \u2192 $20k MRR by April 2027 (6 months)\n**Budget:** 10 hours/week, one founder, no cold calling\n**Written:** 2026-10-08\n\n---\n\n## 0. The short version\n\n1. **Your free funnel isn't broken, it's pointed at the wrong person.** Developers sign up, but PMs pay. Stop waiting for the 300 free users to upgrade. Use them as a **lead list of companies** and sell to the PM at those companies.\n2. **What already works is LinkedIn DMs to PMs at 50\u2013300 person SaaS companies.** That's how you got all 6 paying teams. Make it a weekly system instead of something you do when you have time.\n3. **You can't reach $20k at current prices.** At ~$233 average per team, you'd need ~80 new customers in 6 months, which one person can't close in 10 hours a week. **New customers need to average ~$600/month.** That means $499 is the default quote and there's a ~$999 tier for bigger teams.\n4. **You never have to make a cold call.** Every call in this playbook is one the prospect agreed to.\n5. **Check the numbers at week 8.** If leading indicators are behind (section 9), change the plan then. Don't wait until month 6 to find out.\n\n---\n\n## 1. The math (read this before anything else)\n\nYou need **+$18.6k new MRR**. How many new customers that takes depends on price:\n\n| Avg new MRR per customer | New customers needed | Per month | Realistic at 10 h/week? |\n|---|---|---|---|\n| $233 (today's average) | ~80 | ~13 | No |\n| $400 | ~47 | ~8 | Very hard |\n| **$600** | **~31** | **~5** | **Stretch but possible** |\n| $800 | ~23 | ~4 | Yes, if the higher tier sells |\n\n**Working plan: ~30 new customers at ~$600 average, plus some expansion from your existing 6.**\n\nMonthly targets (ramping, since the pipeline takes time to fill):\n\n| Month | New customers | Cumulative MRR (approx.) |\n|---|---|---|\n| Oct\u2013Nov (M1) | 2 | $2.6k |\n| M2 | 4 | $5.0k |\n| M3 | 5 | $8.0k |\n| M4 | 6 | $11.6k |\n| M5 | 6 | $15.2k |\n| M6 | 7 | $19.4k + expansion \u2248 $20k |\n\n**Being honest:** this is a stretch goal. If price increases don't hold or churn shows up, the likely outcome is $10\u201314k MRR. That's still 7\u201310x where you are now, and a real business. The plan below gives you the best shot at $20k and tells you early if you're off track.\n\n### Funnel assumptions (replace these with your real numbers in week 1)\n\n| Stage | Cold LinkedIn | Warm (free-user company, referral, inbound) |\n|---|---|---|\n| Touch \u2192 reply | 15% | 35% |\n| Reply \u2192 call booked | 40% | 60% |\n| Call \u2192 pilot | 50% | 60% |\n| Pilot \u2192 paid | 50% | 60% |\n| **Touch \u2192 customer** | **~1.5%** | **~7.5%** |\n\n**First task:** go back through the LinkedIn threads that produced your 6 customers. Count roughly how many PMs you messaged, how many replied, how many took a call, and how many paid. Your real rates are better than these guesses. If your cold rate is above ~2%, the plan gets easier.\n\nWhat these rates imply per week, on average:\n- **~40 cold touches** \u2192 ~0.6 customers\n- **~10 warm touches** \u2192 ~0.75 customers\n- Total \u2248 1.3\u20131.4 customers/week \u2248 30+ over 6 months\n\nThat's why warm channels (sections 3B\u20133D) matter so much. One warm touch is worth about five cold ones.\n\n---\n\n## 2. Who you sell to\n\n### Ideal customer profile\n- **Company:** B2B SaaS, 50\u2013300 employees, shipping often (weekly or biweekly releases), with a public changelog or a \"What's new\" page that is obviously stale or hand-written\n- **Buyer and user:** Product Manager, Senior PM, Head of Product, or Product Ops. At the top of this range, sometimes Product Marketing.\n- **Technical champion:** the developer who wires up the API. Often already one of your free users.\n- **Trigger events** (reach out when you see one):\n  - They just posted a launch or big release on LinkedIn\n  - Their changelog hasn't been updated in more than 4 weeks while their product clearly moves faster\n  - They're hiring a PM, Product Marketing Manager, or Product Ops person\n  - They recently raised a round (more releases and more pressure to show momentum)\n  - They're running a manual in-app announcement tool, or sending release-note emails by hand\n\n### Who to skip\n- Companies under ~30 people: price-sensitive, and the founder does release notes themselves\n- Companies over ~1,000 people: procurement and security reviews will eat your 10 hours\n- Developers as the buyer: they're great champions, but they don't own the budget or the problem\n\n### The pain you're selling (in their words, not yours)\n- \"Customers don't know what we shipped, so they think we're slow.\"\n- \"Writing release notes takes me half a day every sprint.\"\n- \"Support keeps getting asked about features that already exist.\"\n- \"Adoption of new features is low.\"\n\n**Check this:** ask your 6 customers which of these is closest to why they bought (section 6). Then use *their exact words* in every message.\n\n---\n\n## 3. Channels, ranked by return on your hours\n\n### A. LinkedIn DMs to PMs (core engine, ~3 h/week)\nThis is your proven channel. Turn it into a routine:\n- Build a list with LinkedIn search (or Sales Navigator, about $100/month, which is worth it at this stage). Filter: title contains Product Manager / Head of Product / Product Ops; company size 51\u2013200 and 201\u2013500; industry Software / SaaS.\n- **Personalize with one line, not a paragraph.** Mention a recent release, their changelog, or a post they wrote. Budget ~3 minutes per prospect.\n- **About 40 new prospects a week.** Batch this into two sessions.\n- Stay well under LinkedIn's weekly connection-request limit (roughly 100\u2013200/week, depending on account).\n\n### B. Free users \u2192 their PM (highest leverage, ~1 h/week)\nYou have 300 developers. Some of them work at companies that match your ICP.\n1. Export sign-ups and pull company from the email domain. Ignore gmail and other personal addresses.\n2. Filter to companies with 50\u2013300 employees.\n3. For each match:\n   - **Option 1 (best):** email the developer and offer to help them get it live. Ask: *\"Who on your side owns release notes / product announcements? Happy to show them what it looks like with your real changelog.\"*\n   - **Option 2:** DM the PM at that company directly, but **don't name the developer** or share their usage. Say *\"a few folks on your engineering team have tried Patchnote.\"* That's honest and doesn't expose anyone.\n4. **Product change worth making (small build, big payoff):** add an \"Invite your PM / share a preview\" button in the free product that sends a rendered preview of *their* release notes to a colleague. That way the product generates PM leads for you every week.\n\nExpect these to convert several times better than cold outreach, because the integration is already partly done.\n\n### C. Existing customers (~0.5 h/week, mostly in month 1)\nYour 6 customers are your best source of referrals, a case study, and higher prices.\n- Customer interviews and a referral ask (scripts in section 6)\n- Expansion: are any of them on $99 who should be on a higher tier?\n- One written case study with a number in it, like \"cut release-notes time from 4 hours to 20 minutes\" or \"feature adoption up X%\"\n\n### D. Light LinkedIn content (~1 h/week)\nYou hate cold calling, but posting once a week warms up everyone you're DMing. Prospects check your profile before they reply.\n- One post a week. Rotate: (1) a teardown of a great or bad changelog (no dunking on anyone by name), (2) a customer result, (3) a lesson about release communication, (4) a behind-the-scenes Patchnote build.\n- Your profile headline should say who you help: *\"Helping B2B SaaS PMs turn changelogs into release notes customers actually read | Founder, Patchnote\"*\n\n### Skip for now\n- Cold calling (you'd hate it, and it's poor value for this ICP)\n- Cold email at volume (deliverability setup isn't worth 10 hours a week yet)\n- Paid ads, conferences, SEO (too slow or expensive for a 6-month goal)\n- Partnerships and integrations marketplaces: revisit in month 4 if a clear partner shows up (for example, the changelog or issue tracker your customers already use)\n\n---\n\n## 4. The 10-hour week\n\n| Block | Time | What |\n|---|---|---|\n| **Mon: prospecting** | 1.5 h | Build the week's list: 40 cold + ~10 warm (free-user companies, referrals). Write the personalized first lines. |\n| **Mon/Tue: send** | 1 h | Send connection requests and first DMs. Send free-user emails. |\n| **Tue\u2013Thu: calls** | 3 h | Up to 6 \u00d7 30-minute calls (discovery+demo or pilot check-ins). Keep them on 2 days so your build time stays protected. |\n| **Wed: follow-ups** | 1.5 h | Follow-up messages, pilot nudges, proposals, invoices. |\n| **Thu: content** | 1 h | Write and post one LinkedIn post. Reply to comments; commenters become warm leads. |\n| **Fri: review** | 1 h | Update the pipeline tracker, check numbers against section 9, note one thing to change next week. |\n| **Buffer** | 1 h | Calls run over, the case study, the free-user export, etc. |\n\n**Rules:**\n- Calls go only into pre-set slots via your booking link (Cal.com or Calendly). No back-and-forth over times.\n- If you're short on time one week, **cut content first, never follow-ups.** Most deals are won in follow-up.\n\n---\n\n## 5. Scripts\n\nRewrite these in your own voice. They should sound like a founder, not an SDR.\n\n### 5.1 Connection request (\u2264 200 characters)\n> Hi {Name}, saw {Company} shipped {feature}. Looks great. I'm building a tool for PMs who'd rather not hand-write release notes. Would love to connect.\n\n### 5.2 First DM (after they accept)\n> Thanks for connecting, {Name}.\n>\n> Quick question. I noticed {specific: your changelog's last entry is from August / you announced X on LinkedIn but I couldn't find it in-app}.\n>\n> I built Patchnote. It turns your changelog (or merged PRs) into in-app release notes and customer emails automatically. PMs at {customer-type, e.g. \"a few 100\u2013200 person SaaS teams\"} use it so customers actually notice what ships.\n>\n> Is release communication something you own, or is it someone else's job at {Company}?\n\nWhy it works: it's specific, short, and **ends with an easy question that isn't a meeting request.** Ask for a call in the second message.\n\n### 5.3 If they reply with interest\n> Makes sense. Happy to show you what your last 3 releases would look like as in-app notes + an email, using your real changelog. 20 minutes, no slides. Here's my calendar: {link}. Or if it's easier, I can just send a Loom.\n\n**The Loom option matters.** For people who don't want a call, a 2-minute video built on *their* changelog is a strong, low-pressure demo. Make one per warm prospect.\n\n### 5.4 Follow-ups (when there's no reply)\n- **Day 4:** *\"Bumping this in case it got buried. I mocked up what {Company}'s last release would look like as an in-app note: {screenshot/Loom}. Worth a look?\"*\n- **Day 10:** *\"Last one from me. If release notes aren't a priority right now, totally fine. Is there someone else on the product team I should talk to?\"*\n- After that, stop. Re-engage only if a new trigger event appears.\n\n### 5.5 Free user \u2192 offer of help (email to the developer)\n> Subject: Your Patchnote setup\n>\n> Hi {Name}, I'm the founder of Patchnote. Saw you signed up. Thanks! Did you get it working with your changelog?\n>\n> If it'd help, I can set up the in-app widget and email template with you in 15 minutes. And if someone on your product team owns release notes, I'm happy to show them a preview built on your real changelog so you don't have to sell it internally.\n\n### 5.6 Free-user company \u2192 PM (LinkedIn DM)\n> Hi {Name}, a few folks on {Company}'s engineering team have been trying Patchnote, which turns changelogs into in-app release notes and customer emails. Usually the PM ends up owning this, so I wanted to check in with you directly. Would a preview built on your actual changelog be useful?\n\n---\n\n## 6. Existing customers (do this in weeks 1\u20133)\n\n### 6.1 Customer interview (20 min each, all 6)\n1. What were you doing for release notes before Patchnote?\n2. What made you look for something? What was the trigger?\n3. What almost stopped you from buying?\n4. What's changed since? Any number you can point to: time saved, feature adoption, fewer support tickets?\n5. If Patchnote went away tomorrow, what would you miss most?\n6. Who else do you know who has this problem?\n\n**Use the answers to:** rewrite your DM copy, pick the \"hero\" pain, decide what goes in the higher tier, and find a case-study candidate.\n\n### 6.2 Referral ask (at the end of the interview, or right after a win)\n> This has been really useful, thanks. One ask: do you know 1\u20132 other PMs who struggle with release notes the way you did? I'm happy to write the intro note so it takes you 30 seconds.\n\nThen send them a ready-to-forward blurb. A referral that takes them no effort is one they'll actually make.\n\n### 6.3 Case study\nPick the happiest customer with a number. Draft it yourself (half a page: before \u2192 after \u2192 result \u2192 quote) and ask them to approve it. Use it in DMs, in proposals, and in a LinkedIn post.\n\n### 6.4 Expansion\nIf customers on the $99 plan are getting real value, move them to the new pricing at renewal or when they hit a limit. Treat existing customers fairly (grandfather for a period if you need to), but don't leave them on a price that undercuts what you're selling to new teams.\n\n---\n\n## 7. Discovery + demo call (30 min)\n\n| Minutes | What |\n|---|---|\n| 0\u20133 | Set the agenda: *\"I'll ask a few questions about how you handle release notes today, then show you yours running in Patchnote. If it's not a fit, I'll tell you.\"* |\n| 3\u201312 | **Discovery.** How often do you ship? Who writes release notes today, and how long does it take? Where do customers find out about new features? What happens when they don't? Who else would care about this (support, CS, marketing)? Have you tried other tools? |\n| 12\u201322 | **Demo on their data.** Prepare it before the call from their public changelog. Show the in-app note and the email. Keep it to what they said hurts. |\n| 22\u201327 | **Next step.** *\"Want to try it on your next release? I'll set it up with your engineer. It takes about 15 minutes, and we can look at the results together after you ship.\"* |\n| 27\u201330 | Book the pilot kickoff and the review meeting **before you hang up.** Ask who else needs to be involved in the decision. |\n\n**Qualifying checklist** (all yes = real deal):\n- [ ] Releases at least every 2 weeks\n- [ ] Release communication is a real pain the PM owns\n- [ ] Has budget authority (or knows who does) at $500/month\n- [ ] An engineer can spend 15\u201330 minutes on setup\n- [ ] Agreed to a specific pilot start date\n\n---\n\n## 8. Pilot \u2192 close\n\n### Pilot structure\n- **Length:** 14 days, or one release cycle, whichever is shorter\n- **Success criteria, agreed up front:** e.g. \"next release goes out as an in-app note + email with less than 30 minutes of PM time,\" or \"N customers open/click the announcement\"\n- **Day 0:** 15-minute setup call with their engineer (you drive)\n- **After first release:** send a short summary: what went out, opens/clicks if you track them, time saved\n- **Day 14:** 15-minute review \u2192 close\n\n### Close\n> Based on what we saw ({result}), the {Tier} plan at ${price}/month covers {their usage}. I can send the invoice today so you don't lose the setup. Monthly, or annual with 2 months free?\n\n**Offer annual.** It doesn't change MRR, but it improves cash flow and retention. Some PMs prefer one annual approval to a recurring expense.\n\n### Pricing (decide by end of week 2)\nYou don't need a perfect price page, but you do need these:\n- **Stop quoting $99 to your ICP.** Keep a cheap tier for small teams and self-serve if you like, but it's not the plan you sell on calls.\n- **$499/month: the standard team plan.** Quote this by default.\n- **~$999/month: a higher tier** built around what bigger teams (150\u2013300 people) value. Base it on your customer interviews. Typical candidates are multiple products or workspaces, custom branding, audience segmentation, more seats or approvals, and priority support. Only include what you can actually deliver.\n- Hold the price. If you discount, trade for something: annual prepay, a case study, a referral.\n\n### Objections\n\n| They say | You say |\n|---|---|\n| \"We just write them by hand / in Notion.\" | \"How long does each one take you? Over a year that's {X} hours of PM time. Patchnote is less than one day of that per month.\" |\n| \"Too expensive.\" | \"Compared to what? If it saves you half a day per sprint, that's ~{N} hours a month. Want to start with a pilot so you can judge from real results?\" |\n| \"Engineering doesn't have time.\" | \"Setup takes about 15 minutes, and I'll do it with them on a call.\" |\n| \"We already use {in-app tool}.\" | \"We plug into that. Patchnote writes the content; your tool can still deliver it.\" *(Only say this if it's true. Check integrations first.)* |\n| \"Not now.\" | \"Totally fair. When's your next big release? I'll check in a week before.\" Then put a dated reminder in the tracker. |\n| \"Need to check with my boss.\" | \"Makes sense. Want me to put together a one-pager with the pilot results you can forward? Or I'm happy to join a 15-minute call with them.\" |\n\n---\n\n## 9. Pipeline tracking and weekly numbers\n\nUse the included `pipeline-tracker.csv` (open in Google Sheets). A CRM is overkill for now.\n\n**Stages:** Prospect \u2192 Contacted \u2192 Replied \u2192 Call booked \u2192 Pilot \u2192 Proposal \u2192 Won / Lost / Nurture\n\n**Check every Friday:**\n\n| Metric | Weekly target | Red flag (2 weeks in a row) |\n|---|---|---|\n| New prospects contacted (cold + warm) | 50 | < 35: protect your Monday block |\n| Replies | 8+ | < 5: rewrite the first DM; test a new opener |\n| Calls held | 3\u20135 | < 2: push Looms and the calendar link harder |\n| Pilots started | 2 | < 1: is the call ending with a booked date? |\n| New customers (rolling 4-week avg) | 1\u20131.5 | < 0.75: see decision points below |\n| Avg new MRR per customer | \u2265 $500 | < $400: you're discounting or quoting the wrong tier |\n\n### Decision points\n- **End of week 4:** you should have real funnel rates. Update the table in section 1 and recompute what's needed.\n- **End of week 8 (early December):** target \u2265 $5k MRR, plus 8+ active pilots or proposals. If you're well below:\n  - Cold reply rate low \u2192 messaging problem. Use customer interviews to rework it.\n  - Calls happen but don't convert \u2192 ICP or product problem. Narrow the ICP to whatever your best 6 customers have in common.\n  - Everything converts but volume is low \u2192 you need more hours or help. Consider 5 more hours a week, or a part-time contractor to build lists (not to sell).\n- **End of month 4:** if you're below $10k MRR, reset the goal to something like $14k and focus on retention and a strong case study, rather than burning out chasing the number.\n\n---\n\n## 10. Week 1 checklist\n\n- [ ] Count your historical LinkedIn funnel for the 6 customers (section 1)\n- [ ] Book interviews with all 6 customers (section 6.1)\n- [ ] Export the 300 free sign-ups; tag ICP-fit companies (section 3B)\n- [ ] Update your LinkedIn headline and the \"About\" section\n- [ ] Set up the booking link with 6 call slots, Tue\u2013Thu\n- [ ] Copy `pipeline-tracker.csv` into Google Sheets\n- [ ] Build the first list of 40 PMs; send the first batch\n- [ ] Draft the new pricing ($499 standard, ~$999 higher tier) to finish after interviews\n\n## 11. Six-month roadmap\n\n| Period | Focus |\n|---|---|\n| **Weeks 1\u20133** | Customer interviews, pricing, free-user mining, start the weekly outbound routine |\n| **Weeks 4\u20138** | Full weekly rhythm; first case study published; \"invite your PM\" feature shipped; first funnel review |\n| **Months 3\u20134** | Scale what converts best; drop channels that don't; referral ask after every win; push larger deals to the ~$999 tier |\n| **Months 5\u20136** | Keep volume steady; prioritize closing open pilots; expansion conversations with early customers; decide whether to hire help or raise prices again |\n\n---\n\n### Assumptions to verify\n- Funnel conversion rates in section 1 are generic estimates. Your real LinkedIn history is better data.\n- The higher-tier features are suggestions. Build the tier from what customers say they value and what Patchnote actually supports.\n- The integration objection answer depends on what Patchnote really integrates with.\n- LinkedIn connection limits vary by account and change over time. Check yours.\n"}, {"name": "pipeline-tracker.csv", "text": "Company,Employees,Contact name,Title,LinkedIn URL,Source (cold/free-user/referral/inbound),Trigger / personalization,Stage (Prospect/Contacted/Replied/Call booked/Pilot/Proposal/Won/Lost/Nurture),Date first contacted,Last touch,Next step,Next step date,Pilot start,Pilot success criteria,Quoted tier,Quoted MRR (USD),Won date,Lost reason,Notes\nExample Co,140,Jane Doe,Senior PM,https://linkedin.com/in/example,cold,\"Shipped bulk export last week; changelog last updated in Aug\",Contacted,2026-10-12,2026-10-12,Day-4 follow-up with Loom,2026-10-16,,,,,,,\n"}]}, "opus-2": {"cost_usd": 0.24059, "seconds": 95, "output_tokens": 8261, "files": [{"name": "founder-led-sales-playbook.md", "text": "# Patchnote: Founder-Led Sales Playbook\n\n**Goal:** USD 1.4k \u2192 USD 20k MRR in 6 months\n**Constraints:** solo founder, 10 hours/week for sales, no cold calling\n**Date:** October 2026 (target: April 2027)\n\n---\n\n## 0. The short version\n\n1. **Your buyer is the PM, not the developer.** All 6 paying teams came from PMs at 50\u2013300 person SaaS companies. Treat the 300 free developer sign-ups as leads into their companies, not as people you need to upgrade.\n2. **LinkedIn DMs to PMs already work.** Do more of that, on a fixed weekly schedule, and track the numbers.\n3. **The goal depends on price more than on volume.** At today's average of ~USD 233/month you'd need ~80 new customers. At ~USD 600/month you'd need ~31. With 10 hours a week, you can only reach USD 20k by raising the average deal size.\n4. **No cold calls are needed.** The whole system runs on written outreach, warm intros, and booked demos, and you only get on a call after the prospect has agreed to one.\n\n---\n\n## 1. The math: what USD 20k MRR actually requires\n\nCurrent: 6 teams, USD 1.4k MRR \u2192 **average ~USD 233/month**.\nGap: **USD 18.6k MRR**.\n\n| Avg new deal (MRR) | New customers needed | Per month (6 mo) | Per month if month 1 is pipeline-building (5 mo) |\n|---|---|---|---|\n| USD 233 (today) | ~80 | ~13 | ~16 |\n| USD 400 | ~47 | ~8 | ~9\u201310 |\n| USD 600 | ~31 | ~5 | ~6 |\n| USD 800 | ~23 | ~4 | ~5 |\n\nThese numbers assume no churn. Plan for losing a customer here and there and aim ~10% above.\n\n**What 10 hours/week can realistically produce.** These are planning assumptions. Swap in your real numbers once you have 4 weeks of tracking (section 9):\n\n- Personalized DMs/emails to PMs: ~60/week (\u22483 hrs)\n- Reply rate ~15% \u2192 ~9 replies/week\n- Reply \u2192 booked call ~50% \u2192 ~4 calls/week\n- Call \u2192 paid ~25% \u2192 **~1 new customer/week, ~4\u20135/month**\n\n**Conclusion:** At 4\u20135 closes/month, you only reach USD 20k if the average new deal is around **USD 600+/month**, and if you also expand some existing accounts. If you keep pricing as it is, this playbook gets you to roughly **USD 6\u20137k MRR**. That would be a real improvement, but it falls well short of the goal. So pricing (section 6) is on the critical path, not a side task.\n\n> Check these against what you already know: how many PMs did you DM to land the 6 current customers? If you have that number, use your real conversion rates instead of mine.\n\n---\n\n## 2. Ideal Customer Profile (ICP)\n\nWrite this down and only sell to people who match it. Every hour spent on people outside the ICP is an hour you don't get back.\n\n**Company**\n- B2B SaaS, 50\u2013300 employees (matches all 6 current customers)\n- Ships often (weekly or biweekly releases)\n- Already has a public changelog, a \"What's new\" page, release-note emails, or uses a tool like Beamer/Headway/Announcekit (so they already feel the pain and have a budget line)\n- Has a product-led or self-serve motion where in-app announcements drive adoption\n\n**Buyer**\n- Product Manager, Senior PM, Head of Product, or Product Ops\n- Secondary: Product Marketing Manager (often owns release communications)\n- Developers are **users and champions**, not buyers\n\n**Pain you solve (in their words, so check these against your existing customers)**\n- \"Writing release notes every sprint is a chore and always late.\"\n- \"Customers don't know about features we shipped.\"\n- \"Engineering changelogs are unreadable for customers.\"\n\n**Disqualify fast**\n- Companies under ~30 people (no PM, no budget)\n- Teams shipping quarterly (not enough pain)\n- Developers using it for side projects\n\n**Action this week:** Spend 30 minutes re-interviewing (or re-reading notes from) your 6 paying customers. Ask: *What triggered you to look? What were you doing before? What would you lose if Patchnote disappeared?* Use their exact words in your outreach.\n\n---\n\n## 3. Lead sources, ranked by warmth\n\nYou don't cold call. Work these sources in order, warmest first.\n\n### Source A: Your 6 customers (warmest)\n- **Referrals:** Ask each one: \"Do you know 2\u20133 PMs at other SaaS companies who'd find this useful? Happy to give you [a free month / a discount] for each one who signs up.\"\n- **Case study:** Get 1\u20132 short quotes or a mini case study with a concrete number (e.g., \"release notes now ship the same day as the release\" or \"feature adoption up X%\"). You'll reuse it in every message.\n- **Expansion:** Who else at their company could use it, such as another product line or the PMM team? Can they move to a higher tier?\n\n### Source B: The 300 free sign-ups (account-based leads)\nThis is your biggest unused asset. Developers signed up, which means **their company has the problem**. Don't try to upgrade the developer. Go around them, politely.\n\n1. Export the 300 sign-ups. Enrich by email domain: company name, headcount, is it B2B SaaS?\n2. Filter to ICP companies (50\u2013300 people, B2B SaaS). Expect maybe 30\u201380 accounts.\n3. Rank by activity: API calls made, changelogs processed, last active date.\n4. For each ICP account:\n   - Email the developer first: \"Saw you're trying Patchnote at [Company]. Who owns release notes / product comms on your side? Happy to set up a team workspace for them.\"\n   - Find the PM on LinkedIn and DM them, mentioning that someone on their engineering team is already using it.\n5. Personal email domains (gmail etc.) are low priority. Skip them unless the usage is heavy.\n\n### Source C: Targeted LinkedIn DMs (the channel that already works)\nBuild a list of ~300 PMs at ICP companies:\n- LinkedIn search (Sales Navigator is worth the cost if it saves time): title = Product Manager / Head of Product, industry = software, company size 51\u2013200 and 201\u2013500\n- Prioritize companies that **visibly have a changelog** (search for \"[company] changelog\" or look at their site footer). This is your strongest qualifier and a natural opener.\n\n### Source D: Low-effort inbound (runs alongside, don't overinvest)\n- Post on LinkedIn 1\u20132\u00d7/week about release notes, changelogs, and feature adoption. Use before/after examples (\"this engineering changelog \u2192 this customer-facing note\").\n- A free tool can pull in PMs instead of developers: e.g., \"Paste your changelog, get a customer-ready release note.\" Put a \"send to my PM\" / team invite CTA inside it.\n- Change sign-up onboarding to ask \"What's your role?\" and \"Who on your team owns release notes?\" so you can find PM leads automatically.\n\n---\n\n## 4. Outreach scripts\n\nRules: keep it short, specific to them, and end with one clear question. Never paste a pitch deck into a DM.\n\n### LinkedIn connection note (\u2264300 chars)\n> Hi [Name], I saw [Company] ships a public changelog. I'm building a tool that turns engineering changelogs into customer-ready release notes and in-app announcements. Would love to connect.\n\n### DM after they accept: changelog angle\n> Thanks for connecting, [Name]. I looked at [Company]'s changelog. [One genuine observation: e.g., \"you ship a lot, but the entries read like commit messages\" / \"last update was 6 weeks ago even though you're clearly shipping\"].\n>\n> Patchnote turns that raw changelog into customer-facing release notes + in-app announcements + an email, automatically. [Customer type, e.g., \"a 120-person HR SaaS\"] uses it to [concrete outcome].\n>\n> Worth a 20-min look? Happy to run it on your actual changelog so you can see real output.\n\n### DM: warm angle (their developer already signed up)\n> Hi [Name], quick one: [Dev name / \"someone on your engineering team\"] has been trying Patchnote, which turns changelogs into customer-facing release notes and in-app updates. Usually the PM ends up owning that, so I wanted to check: is release communication something on your plate? Happy to show what it'd look like on [Company]'s real changelog.\n\n### Referral intro (for customers to forward)\n> Hey [Name], we've been using Patchnote to turn our changelog into release notes and in-app announcements. Saves [X hours/sprint]. The founder, [You], is great to talk to. Want an intro?\n\n### Follow-up sequence (no reply)\n- **Day 4:** \"Bumping this. I can send you a sample release note generated from your last changelog entry, no call needed. Want it?\"\n- **Day 10:** Send the sample anyway (screenshot or link). \"Made this from your [date] changelog. Is it useful or off-base?\"\n- **Day 20:** \"Last note from me. If release notes aren't a priority right now, no worries. Should I check back next quarter?\"\n- Then stop. Mark as \"nurture\" and revisit in 90 days.\n\n**The \"sample from their own changelog\" is your best weapon.** It replaces a cold call: you show the product working on their data before you ask for any time. Build a 10-minute internal workflow to produce these.\n\n---\n\n## 5. The sales call (20\u201330 min, booked, never cold)\n\n**Before the call (5 min):** look at their changelog, their release cadence, and any existing tool they use. Prepare one sample output.\n\n**Agenda**\n1. **(2 min) Frame:** \"I'd like to understand how you handle release comms today, show you Patchnote on your own changelog, and if it fits, talk next steps. Sound good?\"\n2. **(8 min) Discovery:**\n   - How do release notes get written today? Who writes them? How long does it take?\n   - How do customers find out about new features?\n   - What happens when they don't? (support tickets, low adoption, churn risk)\n   - What have you tried? (tools, manual, nothing)\n   - Who else cares about this? (PMM, CS, Head of Product)\n3. **(8 min) Demo on their data.** Show only the 2\u20133 things tied to the pain they just described.\n4. **(5 min) Next step:**\n   - If it's a strong fit: \"Want to set it up this week? I'll do the onboarding with you personally.\"\n   - If someone else needs to approve: \"Who else needs to see this? Want me to join a 15-min call with them?\"\n   - Always leave with a **date on the calendar**, not \"I'll think about it.\"\n\n**Qualify out politely** if they release quarterly, have no PM owner, or have no budget. Point them to the free tier and move on.\n\n**Founder advantage:** you personally onboard every new customer in the first 6 months. This improves activation, gives you product feedback, and builds referral relationships.\n\n---\n\n## 6. Pricing and closing (this decides whether you hit the goal)\n\nAt ~USD 233 average, 10 hours/week cannot get to USD 20k (see section 1). Steps:\n\n1. **Check the current tiers.** Which tier do your best-fit customers (50\u2013300 person companies) land on? If most are at USD 99, the entry tier is probably too cheap for this buyer.\n2. **Design tiers around the PM buyer**, for example:\n   - **Starter (~USD 149):** single product, release notes + email\n   - **Team (~USD 499):** in-app widget, multiple products/audiences, segmentation, analytics\n   - **Business (~USD 999+):** multiple workspaces, SSO, custom branding, priority support\n   Put the features that 50\u2013300 person companies actually need (in-app + segmentation + analytics) in the USD 499 tier.\n3. **Offer annual plans** (e.g., 2 months free). This doesn't change MRR, but it helps cash flow and reduces churn.\n4. **Test the new price on new prospects first.** Quote USD 499 to the next 10 qualified prospects. If nearly everyone says yes without pushback, the price is still too low.\n5. **Grandfather existing customers** but offer them the upgrade path with new features.\n6. **Developers stay free.** Keep the free tier as a developer channel, but gate team and PM features (in-app widget, email sends, multiple seats, analytics) behind paid plans.\n\n### Common objections\n\n| Objection | Response |\n|---|---|\n| \"We just write them manually.\" | \"How long does that take per release? [Answer]. Over a year that's [X] hours of PM time. Patchnote costs less than one afternoon of that per month.\" |\n| \"We already use Beamer/Headway.\" | \"Those display announcements, but someone still has to write them. Patchnote writes them from your changelog and can publish to the tools you already have / replace them.\" (Verify what integrations you actually support before saying this.) |\n| \"Need to check with engineering.\" | \"Great. Their buy-in matters. Want me to send a 2-line technical summary, or join a 15-min call with them?\" |\n| \"Too expensive.\" | \"Compared with what? What does it cost you today in PM time or in features customers never discover?\" If they're truly outside the ICP, offer Starter or the free tier. |\n| \"Not now.\" | \"Makes sense. What would need to change for this to be a priority? I'll check back in [timeframe].\" |\n\n---\n\n## 7. The weekly 10-hour system\n\nBlock this time in your calendar and protect it like customer meetings.\n\n| Block | Time | Activity |\n|---|---|---|\n| Monday | 1.5 h | Build list: 60 new ICP PMs + check 5\u201310 free sign-up accounts |\n| Monday | 1 h | Send connection requests + warm-angle emails to sign-up accounts |\n| Tuesday | 1.5 h | DMs to newly accepted connections + generate 3\u20135 \"your changelog\" samples |\n| Wed / Thu | 3 h | Sales calls (aim for 4) + same-day follow-up |\n| Thursday | 1 h | Follow-up sequence (day 4 / 10 / 20), referral asks |\n| Friday | 1 h | Onboard new customers / check in with existing ones (expansion, referrals) |\n| Friday | 1 h | Update tracker, review numbers, write 1 LinkedIn post |\n\n**Batch everything.** Context switching is the main thing that eats founder sales time.\n\n---\n\n## 8. 6-month plan\n\n| Month | Focus | Target end-of-month MRR |\n|---|---|---|\n| **1 (Oct)** | Set up: ICP doc, customer interviews, enrich 300 sign-ups, new pricing, tracker, first 200 DMs, 1 case study | ~USD 2k |\n| **2 (Nov)** | Full weekly cadence. Test new pricing on every new deal. Referral asks to all 6 customers | ~USD 4k |\n| **3 (Dec)** | Fix the weakest funnel step based on data. December is slow, so use it for content + the free tool | ~USD 6.5k |\n| **4 (Jan)** | Budget season: new-year budgets open. Push hard on pipeline | ~USD 10k |\n| **5 (Feb)** | Second case study. Expansion upgrades for early customers | ~USD 14.5k |\n| **6 (Mar\u2013Apr)** | Close the pipeline. Decide: hire, raise hours, or add a channel | ~USD 20k |\n\nThis curve assumes the average new deal moves to ~USD 500\u2013600 by month 2 and that closes ramp from ~3/month to ~6\u20137/month as referrals and inbound add to outbound.\n\n### Checkpoints (be honest at these)\n- **End of month 2:** Is the call \u2192 paid rate \u2265 20%? Is the average new deal \u2265 USD 400? If not, the problem is pitch/fit/price, not volume. Fix that before sending more DMs.\n- **End of month 3:** Below ~USD 5k MRR? Then USD 20k by April isn't realistic at 10 hours/week. Your options:\n  - (a) raise sales time to 15\u201320 hours\n  - (b) move further upmarket (larger deals)\n  - (c) reset the target to a pace you can sustain, e.g., USD 10k by April\n\n---\n\n## 9. What to track (one spreadsheet is enough)\n\n**Pipeline columns:** Name \u00b7 Company \u00b7 Headcount \u00b7 Source (referral / sign-up / LinkedIn / inbound) \u00b7 Stage \u00b7 Date of last touch \u00b7 Next step + date \u00b7 Deal size \u00b7 Notes\n\n**Stages:** Targeted \u2192 Contacted \u2192 Replied \u2192 Call booked \u2192 Call held \u2192 Trial/setup \u2192 Won / Lost / Nurture\n\n**Weekly numbers (review every Friday):**\n- DMs/emails sent\n- Reply rate\n- Calls booked / held\n- Call \u2192 won rate\n- Average new MRR per deal\n- MRR added this week; total MRR\n- **Conversion by source.** After 4\u20136 weeks, put more time into the source that converts best.\n\n---\n\n## 10. Stop doing\n\n- **Trying to upgrade free developers directly.** They aren't the buyer. Use them as an entry point to their company's PM.\n- **Unstructured outreach when you happen to have time.** Use fixed blocks with weekly targets.\n- **Long trials without a check-in date.** Every trial gets a scheduled review call.\n- **Selling to companies outside the ICP** because they replied. Point them to self-serve.\n\n---\n\n## First 7 days: checklist\n\n- [ ] Interview or re-read notes from all 6 paying customers; write down their exact pain phrases\n- [ ] Ask each of the 6 for a referral and/or a quote\n- [ ] Export and enrich the 300 sign-ups; filter to ICP accounts\n- [ ] Decide on a new pricing structure (at least a ~USD 499 Team tier)\n- [ ] Create the pipeline spreadsheet\n- [ ] Build a list of 60 ICP PMs with visible changelogs\n- [ ] Send the first 60 connection requests + warm emails to the top 10 sign-up accounts\n- [ ] Make the \"sample release note from your changelog\" workflow repeatable in under 10 minutes\n- [ ] Block the 10 weekly hours in your calendar\n"}]}, "opus-3": {"cost_usd": 0.2838, "seconds": 170, "output_tokens": 9970, "files": [{"name": "founder-led-sales-playbook.md", "text": "# Patchnote: Founder-Led Sales Playbook\n\n**Goal:** USD 1.4k \u2192 USD 20k MRR in 6 months\n**Constraints:** solo founder, 10 hours/week for sales, no cold calling\n**What's working:** LinkedIn DMs to PMs at 50\u2013300 person B2B SaaS companies (all 6 paying teams came from this)\n**What's not working:** 300 free sign-ups, mostly developers, and none have upgraded\n\n---\n\n## 0. The short version\n\n1. **Developers sign up but PMs pay.** Your free funnel brings in the person who uses the product, not the person with the budget. Don't try to make developers upgrade. Use their sign-ups as a way into their company and sell to the PM there.\n2. **Do more of what already works.** LinkedIn DMs to PMs got you all 6 customers. Make that a weekly routine with written messages and a tracking sheet. No cold calling needed.\n3. **The goal depends on price as much as on number of deals.** At your current average (~USD 233/month), you'd need ~80 new customers in 6 months. That's not possible in 10 hours a week. At ~USD 420/month average it's ~45, which is a stretch but possible. Raising your average deal size is part of the plan, not an afterthought.\n4. **Spend 10 hours a week, every week, on a fixed schedule** (Section 6). Check the numbers every Friday. Re-plan honestly at the end of Week 4 and Week 12.\n\n---\n\n## 1. The math (read this first)\n\n| | Value |\n|---|---|\n| Current MRR | USD 1,400 (6 teams, avg \u2248 USD 233) |\n| Target MRR | USD 20,000 |\n| Gap | \u2248 USD 18,600 |\n| Customers needed at USD 233 avg | ~80 new |\n| Customers needed at USD 420 avg | ~45 new |\n| Customers needed at USD 600 avg | ~31 new |\n| Sales hours available (10 h \u00d7 26 wks) | 260 h |\n\n**What this means:** 45 customers in 260 hours means each closed deal can cost you about 5\u20136 hours of total sales time, including prospecting, calls, and follow-up. That's only possible if:\n\n- you qualify hard, so you don't spend demo time on teams that won't buy,\n- the average deal is around USD 400+ (Section 5),\n- some deals come from cheaper sources: referrals, expansion of current customers, and companies whose developers already signed up (Section 3).\n\n**Monthly new-customer targets (ramping up):**\n\n| Month | New customers | Cumulative MRR target (approx.) |\n|---|---|---|\n| 1 | 3 | USD 2.8k |\n| 2 | 5 | USD 4.9k |\n| 3 | 7 | USD 7.8k |\n| 4 | 9 | USD 11.6k |\n| 5 | 10 | USD 15.8k |\n| 6 | 11 | USD 20k |\n\nThese assume ~USD 420 average for new deals and roughly no churn. Change the targets once you have real conversion data from Month 1.\n\n**Be honest with yourself:** if you stick strictly to 10 hours a week and your average deal stays at ~USD 233, the realistic outcome is more like USD 6\u201310k MRR. That's still a big result. Reaching USD 20k most likely needs *both* higher prices *and* the efficient channels below. Check where you stand at the Week 12 checkpoint (Section 8).\n\n---\n\n## 2. Ideal customer profile (ICP)\n\nBase this on your 6 paying teams, not on guesses. **Before Week 1, spend 1 hour writing down for each paying customer:** company size, what they sell, the buyer's title, what they used before Patchnote, why they bought, and how long it took from first DM to payment. Update the ICP below with what you find.\n\n**Starting ICP:**\n\n- **Company:** B2B SaaS, 50\u2013300 employees, shipping product updates at least every two weeks\n- **Buyer:** Product Manager, Senior PM, Head of Product, or Group PM. At the smaller end, possibly a Product Marketing Manager\n- **Signs of pain (things you can see from outside):**\n  - a public changelog that's out of date, or missing entirely\n  - release notes written by hand in Notion, Google Docs, or a CMS\n  - job posts that mention \"release notes,\" \"product communication,\" or \"feature adoption\"\n  - PMs posting on LinkedIn about feature adoption or \"nobody knows we shipped this\"\n  - an existing tool like Beamer, Headway, LaunchNotes, or Canny changelog (they already pay for this category, so you're offering a switch or an upgrade)\n- **Bad fits, don't chase:** companies under ~30 people (founder writes the notes, low budget), enterprises (slow procurement), agencies, and developer-tool hobby projects\n\n**One-sentence pitch (test and change it):**\n> \"Patchnote turns the changelog your engineers already write into polished in-app release notes and customer emails, so PMs stop spending hours every release rewriting updates and customers actually notice what you shipped.\"\n\n---\n\n## 3. Lead sources, in priority order\n\n### Source A: Your existing 6 customers (cheapest, do it first)\n\n**Time:** ~3 hours total in Weeks 1\u20132, then 30 minutes a month.\n\n1. **Customer check-in call (20 min each).** Ask:\n   - \"What would you be doing if Patchnote didn't exist?\"\n   - \"What made you say yes?\" (Use their exact words in your messages.)\n   - \"What almost stopped you?\" (That's your most common objection.)\n   - \"Who else do you know, a PM friend or a former colleague, who has this same problem?\"\n2. **Ask for intros directly:** \"Would you be willing to intro me to 2 PMs you know? I'll write the intro note so it takes you 30 seconds.\" Then send them a draft they can forward.\n3. **Ask for a quote or short case study** you can use in DMs (\"[Company] cut release-notes time from X hours to Y\").\n4. **Look for expansion:** is anyone on a low tier who should be on a higher one? (More seats, more products, email volume, removing your branding.) Moving 2 customers from USD 99 to USD 299 is +USD 400 MRR for one conversation.\n\n**Referral offer (optional):** 1 free month for both sides when a referral pays.\n\n### Source B: Your 300 free sign-ups (an untapped account list)\n\nThese developers aren't buyers, but **each one points to a company that's already looking at your product.** That's a warmer lead than any cold DM.\n\n**Time:** ~4 hours one-time in Week 1, then 30 minutes a week for new sign-ups.\n\n1. **Export the sign-ups and look up the company** for each one (email domain \u2192 company). Throw out personal emails (gmail, etc.) unless they can be looked up easily.\n2. **Score each company against the ICP:** B2B SaaS, 50\u2013300 people. Expect maybe 20\u201330% to fit. That's roughly 60\u201390 target accounts.\n3. **Check usage:** did they connect a repo or changelog, send test notes, or come back more than once? Active + ICP fit = top priority.\n4. **Two ways in, use both:**\n   - **Through the developer (in-app message or email):** \"Saw you set up Patchnote for [Company]. Who on your side owns release notes or product updates? Happy to set them up with a free trial of the full version, so you're not stuck rewriting notes for them.\"\n   - **Directly to the PM (LinkedIn DM):** \"Someone on your engineering team has been trying Patchnote for turning changelogs into release notes. Wanted to reach out to you since PMs usually own how updates go out to customers. Worth a 20-minute look at what it'd do for [Company]?\"\n     *(Keep it vague. Don't name the developer, and don't share their usage details without their permission.)*\n5. **Change the product so this happens on its own (small build tasks, high payoff):**\n   - Add an \"Invite your PM / product owner\" step in onboarding.\n   - Put PM-focused features behind the paid plan: email sends to customers, in-app widget without Patchnote branding, analytics (who opened, who clicked), approval workflow, multiple seats. Developers can test for free. PMs need to pay to use it in production.\n   - Add a 14-day trial of the full version that starts when someone invites a teammate.\n\n### Source C: LinkedIn outreach to PMs (your proven channel, made repeatable)\n\n**Time:** ~4 hours a week, the core of the plan.\n\n**Weekly volume:** 50\u201375 connection requests or InMails + follow-ups. This can be done in about 3 sessions of 60\u201380 minutes. Don't send hundreds. Personalized, small batches work better and keep your account safe.\n\n**Building the list (sessions of 30\u201345 min):**\n- LinkedIn search (Sales Navigator is worth the ~USD 100/month at this volume): title = Product Manager / Head of Product / Senior PM; company size 51\u2013200 and 201\u2013500; industry = Software / SaaS; country = where your current customers are.\n- Before messaging, spend 2 minutes on each person: check their company's changelog page, recent product launches, and the PM's recent posts. Write one line of personalization.\n\n**Message sequence (copy and adjust; keep each one short):**\n\n**Connection request (no pitch, \u2264300 characters):**\n> \"Hi [Name], I follow how B2B SaaS teams share what they ship. Saw [Company] just launched [feature/update]. Would love to connect.\"\n\n**Message 1, after they accept (send within 24 hours):**\n> \"Thanks for connecting, [Name]. Quick question: when [Company] ships something, who turns the engineering changelog into the customer-facing note? I've been talking to PMs at 50\u2013300 person SaaS teams, and it's usually the PM, late on release day. I built Patchnote to automate that (changelog \u2192 in-app notes + email). Is it a pain for you, or have you got it handled?\"\n\n*(It ends with a real question that's easy to answer either way. \"We've got it handled\" is still useful to learn.)*\n\n**Message 2, 4\u20135 days later if no reply:**\n> \"Here's a 60-second example of what Patchnote did with [a real public changelog entry from their company]: [link to a short Loom or a before/after image]. Happy to set one up for your next release if useful.\"\n\n*(This is your strongest message. A before/after made from their own changelog takes 10 minutes and stands out. Use it for the best-fit accounts.)*\n\n**Message 3, 7 days later (breakup message):**\n> \"Last note from me. If release notes become a priority later, I'm here. Either way, I'd love to hear how you handle it today. I'm collecting how PMs approach this.\"\n\n**When they reply with interest:**\n> \"Great. Easiest is a 20-minute call where I run your last release through Patchnote live. Here's my calendar: [link]. Or if you'd rather look on your own first, here's a trial link: [link].\"\n\n**Getting replies without calling:** everything here is written messages plus booked video calls with people who *asked* to talk. No cold calls at all.\n\n### Source D: Content that brings PMs in (low effort, builds over time)\n\n**Time:** ~1 hour a week. Don't skip this, but don't let it eat into outreach time either.\n\n- **1 LinkedIn post a week** from your profile, aimed at PMs. Ideas:\n  - Before/after: a messy changelog \u2192 a clean customer note (use public examples, with permission or anonymized)\n  - \"I looked at the changelogs of 20 B2B SaaS companies. Here's what the good ones do.\"\n  - Lessons from customer calls (\"PMs told me release notes take 3\u20135 hours per release\")\n- Every post makes your DMs work better, because PMs who check your profile see you know the subject.\n- **Free lead magnet (optional, Month 2+):** a \"Release Notes Grader\": paste a changelog URL and get a score plus a rewrite. It brings PMs in, not developers.\n\n### Sources to skip for now\n\nCold calling (you said no, and it isn't needed), paid ads (no budget or time to optimize them), conferences, partnerships, and SEO (too slow for a 6-month goal). Look at them again after Month 6.\n\n---\n\n## 4. The sales process\n\nThe goal is **DM \u2192 20-minute call \u2192 trial/pilot \u2192 paid in \u2264 14 days**, with as little of your time as possible.\n\n### Stage 1: Discovery + live demo call (20\u201325 min)\n\n**Before the call (5 min):** look at their public changelog or release notes, run one real entry through Patchnote, and have the result open.\n\n**Call outline:**\n\n1. **(2 min) Set the agenda:** \"I'll ask a few questions about how you ship updates today, show you your own changelog through Patchnote, and if it fits, we'll talk next steps. If it doesn't fit, I'll say so. Sound good?\"\n2. **(8 min) Discovery questions (pick 4\u20135):**\n   - \"Walk me through what happens after engineering ships a feature. How do customers find out?\"\n   - \"Who writes the release notes? How long does it take per release?\"\n   - \"What happens when it doesn't get done?\" (Look for: low feature adoption, support tickets, sales doesn't know what shipped.)\n   - \"What have you tried? What are you using now?\"\n   - \"If this were solved, what would it be worth to you or the team?\"\n   - \"Who else would need to agree to adopt something like this? Is there a budget for tools like this?\"\n3. **(8 min) Demo:** show *their* changelog \u2192 *their* in-app note + email. Only show what connects to the pain they just told you about.\n4. **(5 min) Next step:** don't end the call without one:\n   - \"Want to try it on your next release? I'll set up a 14-day trial and check in after you send your first update.\"\n   - Set a **date for the follow-up check-in** before you hang up.\n\n**Qualification (all 4 should be true before you spend more time on a deal):**\n- **Pain:** they spend real time on release notes, or know that updates are being missed\n- **Fit:** B2B SaaS, 50\u2013300 people, ships regularly\n- **Authority:** the person can buy, or will bring in the person who can (expect up to ~USD 500/month on a PM's card or budget)\n- **Timing:** a release is coming in the next 2\u20134 weeks they can test with\n\nIf one of these is missing, send them to self-serve with a friendly note and move on.\n\n### Stage 2: Trial / pilot (\u226414 days)\n\n- Use their real release as the test. That's the \"aha\" moment.\n- **Day 1:** send a setup email with a 3-step checklist.\n- **During the trial:** offer a 15-minute setup call if they get stuck. Don't wait for them to ask.\n- **After their first real release goes out:** \"How did it land? Did customers open or click it? Ready to keep it running?\"\n- **Day 12:** \"Your trial ends Friday. Which plan fits? I'd suggest [X] because [what they told you on the call].\"\n\n### Stage 3: Close\n\n- Send a payment link (Stripe) for the plan you recommend. Don't send a long proposal for deals under USD 500/month.\n- **Offer annual billing:** \"Annual is 2 months free.\" This brings cash in sooner and reduces churn.\n- If a manager has to approve: send a short summary they can forward (problem \u2192 what Patchnote does \u2192 price \u2192 what changed during the trial).\n\n### Follow-up rules\n\n- Every open deal has a next step with a date. No exceptions.\n- If they go quiet: follow up after 3 days, again after 7, then a breakup message after 14. Then move them to \"nurture\" and check back in 60 days.\n\n---\n\n## 5. Pricing and packaging (a key lever)\n\nYour current range is USD 99\u2013499. To reach USD 20k you need the **average new deal around USD 400+**. Suggested structure, changed based on what your 6 customers tell you:\n\n| Plan | Price/month | For | Key limits |\n|---|---|---|---|\n| Free | 0 | Developers testing it | Patchnote branding, no customer emails, 1 project |\n| Starter | USD 149 | Small team, 1 product | In-app widget, limited emails, 1 seat |\n| **Team** (lead with this) | **USD 399** | The usual 50\u2013300 person SaaS | Emails, analytics, no branding, 5 seats, approval workflow |\n| Growth | USD 799 | Multiple products, many customers | Multiple projects, segments, integrations, priority support |\n\n**How to use it:**\n- **In calls, recommend Team by default.** Only go down if they push back on budget.\n- **Annual billing:** 2 months free. Push it on every close.\n- **Current customers:** keep their current price for now (grandfathered). Offer an upgrade when they need a feature from a higher plan.\n- **Test prices on new prospects first.** If \u2265 70% of qualified prospects accept USD 399 without pushback, your price is probably too low.\n\n---\n\n## 6. The weekly 10-hour routine\n\nSame time slots every week. Put them in your calendar as fixed blocks.\n\n| Block | Time | Activity |\n|---|---|---|\n| Mon | 1.5 h | Build the list + 25 personalized connection requests / DMs |\n| Tue | 2 h | Sales calls (book them here: up to 4 \u00d7 25 min) |\n| Wed | 1.5 h | Follow-ups on all open conversations + 25 more DMs + new free sign-ups that fit the ICP |\n| Thu | 2 h | Sales calls + trial check-ins + before/after examples for top accounts |\n| Fri | 1 h | Write 1 LinkedIn post + schedule it; reply to comments |\n| Fri | 0.5 h | **Weekly review:** update the tracking sheet, check the numbers against targets (Section 7) |\n| Flexible | 1.5 h | Customer success / referral asks / extra calls as demand grows |\n\n**As you get more calls (Month 3+):** cut list-building and DMs before you cut calls. Calls are closer to revenue. If you have more than 6 calls a week, that's a sign you need more hours or help (see Section 8).\n\n---\n\n## 7. Tracking: the only numbers that matter\n\nUse one simple spreadsheet or a free CRM (HubSpot free, Attio, or Folk). **Columns:** Name, Company, Size, Source (A/B/C/D), Stage, Date of last touch, Next step + date, Plan/value, Notes.\n\n**Stages:** Target \u2192 Contacted \u2192 Replied \u2192 Call booked \u2192 Call done (qualified?) \u2192 Trial \u2192 Won / Lost (reason) / Nurture\n\n**Weekly scorecard (check every Friday):**\n\n| Metric | Weekly target (Month 1\u20132) | Weekly target (Month 3+) |\n|---|---|---|\n| Messages / connection requests sent | 50 | 50\u201375 |\n| Replies | 8\u201312 | 10\u201315 |\n| Calls held | 2\u20133 | 4\u20136 |\n| Trials started | 1\u20132 | 3\u20134 |\n| New customers | 1 | 2\u20133 |\n| New MRR | ~USD 350 | ~USD 900+ |\n| Average deal size | \u2265 USD 300 | \u2265 USD 400 |\n\n*These are starting guesses. Your real numbers from the first 4 weeks will tell you more. Set the targets again based on them.*\n\n**How to tell what's broken:**\n- **Few replies** \u2192 the message or the list is wrong. Change the personalization or the opening question; check the ICP.\n- **Replies but no calls** \u2192 the call ask is too big or vague. Offer the before/after example instead.\n- **Calls but no trials** \u2192 poor qualification, or the demo doesn't connect to their pain. Review your discovery notes.\n- **Trials but no closes** \u2192 onboarding friction, or the trial doesn't run during a real release. Time trials to their release dates.\n- **Closes but small deals** \u2192 you're offering low plans too soon. Lead with Team.\n\n---\n\n## 8. Checkpoints and when to change course\n\n**End of Week 4:**\n- Did you hit \u2265 3 new customers and \u2265 150 messages sent?\n- What are your actual reply, call, and close rates? Rewrite the targets in Section 1 based on them.\n- Which source gives the best leads per hour: A, B, or C? Shift time toward it.\n\n**End of Week 12 (Month 3), the honest check:**\n- **On track (\u2265 USD 7k MRR):** keep going and add a referral program.\n- **Behind (USD 4\u20137k MRR):** pick one: (a) raise prices or move more customers to annual billing, (b) add ~5 hours a week for sales, or (c) get help: a part-time SDR or VA for list-building and first messages (~USD 1\u20132k/month, paid for by about 4 customers).\n- **Far behind (< USD 4k MRR):** stop and re-check the basics. Is the ICP right? Is the pain strong enough? Talk to 10 lost deals before you do more outreach.\n\n**Signs it's time to bring in help:** more than 6 calls a week, a full pipeline you can't follow up on, or MRR above ~USD 10k with a clear repeatable funnel.\n\n---\n\n## 9. Handling common objections\n\n| Objection | Response |\n|---|---|\n| \"We just write them in Notion/Docs.\" | \"Makes sense. How long does that take per release? Most teams I talk to spend 2\u20134 hours. Patchnote starts from what engineering already wrote, so you edit instead of writing from scratch.\" *(Replace with real numbers from your customers.)* |\n| \"We already use Beamer/Headway/etc.\" | \"Those are good for showing notes. Patchnote writes them from your changelog. Who writes the content that goes into Beamer today?\" |\n| \"Too expensive.\" | \"What's an hour of PM time worth to you? If it saves 3 hours a release and you ship every 2 weeks, it pays for itself in the first month. We also have Starter at USD 149 if you only need the in-app widget.\" |\n| \"Not a priority right now.\" | \"Totally fair. When's your next big release? I'll check back a week before, and we can test Patchnote on it then.\" *(Put the date in your sheet.)* |\n| \"I need to ask my manager.\" | \"Of course. I'll send a short summary you can forward. Would it help if I joined a 10-minute call with them?\" |\n| \"Our engineers can build this.\" | \"They could. Do they want to maintain it? Most teams figure out it's a 2\u20133 week project plus ongoing upkeep. Patchnote costs less than one day of engineering time a month.\" |\n\n---\n\n## 10. Day 1 checklist\n\n- [ ] Write a summary of the 6 current customers (Section 2). Update the ICP and pitch based on it.\n- [ ] Book 20-minute check-in calls with all 6 customers. Ask for referrals and quotes.\n- [ ] Export the 300 free sign-ups, look up their companies, and score them against the ICP. Pick the top 20.\n- [ ] Set up the tracking sheet or CRM with the stages above.\n- [ ] Get Sales Navigator (or use free LinkedIn search to start).\n- [ ] Write your 3-message sequence. Make 2 before/after examples from real public changelogs.\n- [ ] Block the 10 hours in your calendar for the next 4 weeks.\n- [ ] Decide on the new pricing (Section 5) and update your pricing page before your first new sales call.\n- [ ] Add 1\u20132 product changes to the backlog: \"Invite your PM\" in onboarding, and paid-only features for PMs.\n\n---\n\n*Review this playbook at each checkpoint. It's a starting point. Your real data from the first month counts for more than any of the estimates here.*\n"}]}, "haiku-1": {"cost_usd": 0.01261, "seconds": 86, "output_tokens": 16688, "files": [{"name": "sales-playbook.md", "text": "# Patchnote Founder-Led Sales Playbook\n\nDraft, 2026-10-08. Goal: $20k MRR in 6 months (from $1.4k). Built for one founder with 10 hours a week, no cold calls.\n\n## 1. Reality check on the goal\n\n- Gap to close: $20k - $1.4k = $18.6k MRR in about 26 weeks.\n- Assumed average price for new deals: $300/month. (Current average is $233; tiers are $99-$499.)\n- New paying teams needed: about 62 at $300, 41 at $450, 37 at $499. Churn is not included, so real numbers are higher.\n- That is about 2.4 new paying teams every week for 6 months.\n\nEstimated outcomes of this playbook. These rest on assumed conversion rates. Replace them with your real numbers after 4 weeks.\n\n| Scenario | Reply rate / call booked / call to paid | Outreach touches (26 weeks) | New teams | MRR at month 6 |\n|---|---|---|---|---|\n| Baseline | 12% / 40% / 27% | ~1,000 (40 a week) | ~13 | ~$5k |\n| Improved | 25% / 40% / 30% | ~1,000 | ~30 | ~$10k |\n| Needed for goal | | | ~62 | $20k |\n\nWhat this means: at 10 hours a week, DM outreach alone will not reach $20k. The gap has to come from three places, and the plan below targets all three:\n\n1. Warm pipeline you already have: 300 free sign-ups and the networks of your 6 paying teams. These convert far better than cold DMs.\n2. A higher reply rate from a sample-first hook (the teardown in section 4), instead of a pitch.\n3. A higher average price, tested in the first 60 days.\n\nIf MRR is well behind the straight-line path at week 8 (about $7k), change something structural: price, segment, or hours. Decide at week 8, not month 6.\n\n## 2. Who to sell to\n\n**Buyer:** Head of Product, Product Manager, or Product Ops at a B2B SaaS company with 50-300 employees that ships releases customers should hear about.\n\n**Qualify before spending time. All three must be true:**\n- They ship a change at least every two weeks.\n- They have a public or in-app changelog, a release-notes email, or have said they want one.\n- They own release communication, or someone who reports to them does.\n\n**Start with the six paying customers.** For each one, write down: title, company size, how they announce releases today, what made them buy now, where they first heard of Patchnote, and what happened in their first month. Find the two or three traits they share. Build the target list from those traits. This is the most important work in the first two weeks.\n\n**Build a target list of 150-200 accounts.** Use LinkedIn search (or Sales Navigator if you have it) for titles like \"Product Manager,\" \"Head of Product,\" \"Product Operations,\" and \"Director of Product,\" filtered to 50-300 employees. Check each company for a changelog page (its domain plus /changelog, /releases, or /updates). Companies with active changelogs are the best targets.\n\n## 3. Pipeline sources, in priority order\n\n1. **The 300 free sign-ups.** This is your warmest pool. In week 1, triage them: for each, check the email domain and company size. Email the ones that match. For developers at companies that don't match, ask whether they have a PM who ships releases. Developers can champion Patchnote internally.\n2. **Referrals from the six paying teams.** Ask each one for one introduction in the next 30 days, with a specific ask: \"Who in your network ships a changelog?\" For a referral that converts, give the referrer one free month of credit. Do not discount the referred deal.\n3. **LinkedIn outreach using the teardown hook (section 4).** All six paying customers came from DMs, so this channel is proven. Keep sends manual. LinkedIn restricts automation.\n4. **Public teardowns**, once you have three or more. Post them on LinkedIn as a second reach channel. Only do this with the company's permission or with the company anonymized.\n\nSkip paid ads, cold email lists, and marketplaces until the DM motion converts.\n\n## 4. The hook: the teardown\n\nA DM that says \"I built a tool for release notes\" gets ignored. Show the prospect their own output instead.\n\n**Process, about 15 minutes per account:**\n1. Take their last three public changelog entries.\n2. Run them through Patchnote to generate the in-app notes and the email.\n3. Record a 60-90 second Loom: \"Here's your last release as your customers would see it. One thing I noticed: [specific observation].\"\n4. Send the video with one question.\n\nRules:\n- Use only public changelog content. Don't claim to know their internal process.\n- If you can't produce a good sample from their changelog within 15 minutes, drop that account. The list is probably wrong.\n\n## 5. Messages\n\nKeep first messages short. No attachments in the first message. No \"hope you're well.\"\n\n**LinkedIn connection note:**\n> Hi {first}, the write-up for {company}'s {specific release} was clear. I'm working on tools for PMs who ship often. Would be glad to connect.\n\n**First DM, sent after they accept:**\n> {first}, I ran {company}'s last three changelog entries through Patchnote. Here's a 60-second video of what your customers would see in-app and in email: {link}. The thing I'd check first is {one specific observation}. Is release communication on your plate right now?\n\n**Follow-up 1 (day 3):**\n> Did the video land? Happy to adjust the tone to match your brand if that's useful.\n\n**Follow-up 2 (day 8):**\n> Last one from me. If release notes are handled, ignore this. If not, what's the usual bottleneck: writing, approvals, or distribution?\n\n**Free sign-up email (plain text, from your address):**\n> Hi {first}, I saw you signed up for Patchnote on {date}. What are you trying to ship: release notes for your customers, or something else? If it's for a team, I'm glad to walk through it in 15 minutes on Zoom.\n\n**Booking ask after a reply:**\n> Great. Would {option A} or {option B} work for a 20-minute video call? I'll run your last three releases through it live so you can see the output.\n\n## 6. The call (20 minutes, video only)\n\nStructure:\n1. (2 min) Confirm what they want from the call.\n2. (8 min) Discovery questions.\n3. (5 min) Show their own release, live.\n4. (5 min) Ask for the next step.\n\n**Discovery questions:**\n- How do you announce releases today, and who writes the notes?\n- From start to finish, how long does release communication take for a typical release?\n- What happened the last time a release went out with no notes or with wrong ones?\n- Where do customers see updates today: in-app, email, a changelog page?\n- If this worked perfectly, what would change for your team next quarter?\n- Who else would need to approve a monthly spend?\n\nPitch only after they've named the cost of their current process. If they haven't, the call isn't finished.\n\n**Objections:**\n- **\"We'll build it ourselves.\"** Ask: \"What's the plan for keeping it running after the first three releases?\" Maintenance is where build-it-yourself plans usually fall apart.\n- **\"It's too expensive.\"** Compare the price to the hours they just described. Then recommend the tier that covers their release volume. Don't discount.\n- **\"We already use a changelog or announcement tool.\"** Ask: \"Does that tool write the note, or does someone write it by hand first?\" If someone writes by hand, that's your opening.\n- **\"Send me something.\"** Send the video you already recorded, and propose a specific next step: \"I'll check back Thursday at 2pm.\" Don't send PDFs.\n- **\"Not now.\"** Ask: \"When would be the right time to look again?\" and \"Who do you know with this problem?\" Put the date in your CRM.\n\n## 7. Closing and pricing\n\n- **Trial:** offer 14 days connected to their real changelog. Write the success criteria in the first call, for example: \"Three releases go out with generated notes and need at most minor edits.\"\n- **Close:** at the end of the trial, ask directly: \"Do you want to keep this going on the {tier} plan?\" Then send the checkout link. Don't wait for them to bring it up.\n- **Pricing:** choose the tier from their release volume and team size. Track which tier each new customer picks and why. Don't discount to close. After your first three new deals, test a $299 entry tier for 50-300 person companies and see whether close rates hold.\n\n## 8. Weekly operating rhythm (10 hours)\n\n| Block | Hours | What |\n|---|---|---|\n| Outreach | 4 | About 40 personalized touches (connection notes, DMs, follow-ups), about 6 minutes each using pre-built teardowns |\n| Warm pipeline | 1 | Free sign-up emails, referral asks, replies |\n| Teardowns | 2 | Build 6-8 teardowns for next week's list |\n| Calls and demos | 2 | 3-4 calls including prep and notes |\n| Review | 1 | Metrics, CRM cleanup, next week's list (Friday) |\n\n## 9. Metrics and decision points\n\nTrack these weekly in one sheet, one row per week: touches sent, replies, reply rate, calls booked, calls held, trials started, paid conversions, MRR, churned MRR.\n\n- **Week 2:** Is the reply rate at least 10%? If not, rewrite the first DM and test the teardown on a different segment.\n- **Week 4:** Are calls booked at 40% or more of replies, and held at 70% or more of bookings? If not, fix the booking ask and the discovery script. Has the first new deal closed? If not, check whether your targets match the traits of the six paying customers.\n- **Week 8:** Compare MRR to the straight-line path (about $7k). If you're more than $2k below, change the price, the segment, or your hours. Don't keep running the same motion and hoping.\n\n## 10. First four weeks\n\n- **Week 1:** Analyze the six paying customers. Triage the 300 free sign-ups. Build the first 50 accounts. Record and review one teardown yourself.\n- **Week 2:** Send about 40 touches a week. Ask each paying customer for one introduction. Email the target-size free sign-ups.\n- **Week 3:** Ship 10 teardowns. Hold the first calls. Start the metrics sheet.\n- **Week 4:** First metrics review. Adjust the hook, segment, or ask based on the numbers.\n\n## 11. Open questions to answer in week 1\n\n- What do the six paying teams have in common (title, size, trigger, how they found you)?\n- Can a prospect start paying without a call? If not, that's a bottleneck to fix first.\n- How many of the 300 free sign-ups work at 50-300 person B2B SaaS companies?\n- Is $99 a real price point for these buyers, or a placeholder?\n- How many hours a week can you keep this up for six months? The plan assumes 10 hours, and it will fail if that slips.\n\n## 12. What to avoid\n\n- Cold calls and phone sequences.\n- Generic pitch DMs (\"I'd love to show you Patchnote\").\n- Discounting to close.\n- Building features for one prospect without a commitment.\n- Paid ads and cold email blasts before the DM motion converts.\n"}]}, "haiku-2": {"cost_usd": 0.01135, "seconds": 76, "output_tokens": 14717, "files": [{"name": "founder-sales-playbook.md", "text": "# Patchnote: Founder-Led Sales Playbook\n\nWritten 2026-10-08. Owner: solo founder, 10 hours/week on sales.\nAssumptions are marked as such. Replace them with measured numbers after week 2.\n\n## 1. The math\n\nStarting point: $1,400 MRR, 6 paying teams, about $233 per team.\nTarget: $20,000 MRR by 2027-04-08 (26 weeks).\n\nAssuming zero churn and that the 6 existing teams stay at about $233:\n\n| Avg price of new teams | New paying teams needed | New teams per week |\n|---|---|---|\n| $233 (today) | 80 | 3.1 |\n| $350 | 54 | 2.1 |\n| $400 | 47 | 1.8 |\n| $499 (top tier) | 38 | 1.5 |\n\nAt today's price, the goal needs about 3 new paid teams every week for six months. That is not achievable with 10 hours of outreach. The realistic path depends on two things:\n\n1. **Price per new team rises to about $400 or more.** Lead with the higher tiers and move teams up when they hit plan limits. Pricing is part of sales work here, not a separate project.\n2. **Outreach converts at the planning rates in section 7.** Measure them in weeks 1-2.\n\nChurn raises every number above. Measure it from the first week.\n\n## 2. Who to sell to\n\n**Buyer:** the Head of Product or PM at a 50-300 person B2B SaaS company that ships changes to its own customers. Developers use Patchnote. PMs buy it.\n\n**Good signals:**\n- The company publishes a changelog or release notes page.\n- The PM owns customer-facing release communication, not only marketing.\n- Frequent releases (roughly weekly or biweekly).\n\n**Disqualify:**\n- Outside 50-300 employees, unless a non-matching team becomes a paying customer.\n- No customer-facing product.\n- Only developers are involved and no one owns release comms.\n\n## 3. Weeks 1-2: work the warm list first\n\nThe 300 free sign-ups have already shown intent. Work them before any new outreach.\n\n1. **Segment the sign-ups (about 3 hours, week 1).** Look up each company's size (LinkedIn or domain lookup). Keep the ones with 50-300 employees. Note which ones actually sent a changelog through the product.\n2. **Find the PM for each company.** Search LinkedIn for \"Head of Product\" or \"Product Manager\" at that company.\n3. **Send a sample, not a pitch.** Run the company's public changelog through Patchnote and send the result with one question: \"Worth 15 minutes to see if this would save your team time?\" The sample is the hook. Do not ask for a call in the first message.\n4. **Ask the developers one question.** For the sign-ups who did integrate, send a short message: \"Who on your side owns release announcements?\" This gets names with almost no effort from the developer.\n5. **Interview the 6 paying teams (week 1, 3-4 calls).** Ask what made them buy, what they tried before, and how they described the problem internally. Their answers are the source of your messages and your pricing story. At the end of each call, ask for:\n   - A one-line quote and permission to name them.\n   - One introduction to a PM at a peer company.\n\n## 4. Channel: LinkedIn first, no cold calls\n\nThe 6 paying teams came from LinkedIn DMs. Keep that channel and do not phone-cold anyone. Calls only happen after someone replies in writing.\n\n**Rules:**\n- Every message names something specific about the company or its changelog. No pure template sends.\n- Batch outreach into two or three sessions per week. LinkedIn throttles invitations, so do not blast.\n- Maximum three messages to anyone without a reply: the first message plus two follow-ups, spaced four to five business days apart.\n\n**Templates**\n\n*A. Warm list (sample-first):*\n> Hi {first}, you tried Patchnote a while back. I ran your public changelog through it: {one sample line}. Worth 15 minutes to see if it would save your team time on release notes? Happy to just send the full sample if easier.\n\n*B. New prospect (changelog-based):*\n> Hi {first}, I noticed {company} publishes release notes at {url}. We build Patchnote, which turns changelogs like that into in-app notes and emails without manual writing. Would it be useful to see it run on one of your recent releases?\n\n*C. Follow-up (once, after four to five business days):*\n> Quick bump, {first}. Still glad to run one of your recent entries through it if useful. No pressure either way.\n\n## 5. The 20-minute call\n\nBook only after a reply. The call has one job: find out whether there is a release-notes problem worth money, and get a next step with a date.\n\n| Minutes | Section | What to do |\n|---|---|---|\n| 0-2 | Frame | \"I want to understand how you write release notes today and whether Patchnote fits. Sound right?\" |\n| 2-7 | Current process | Ask the questions below. Let them talk. |\n| 7-15 | Live run | Run one of their changelog entries live. Show the in-app and email output. Do not show features they did not mention. |\n| 15-18 | Price | \"Based on {N} releases a month, the {tier} plan is {$}. Want to start a trial with your team this week?\" |\n| 18-20 | Next step | Lock a date and a named owner. Never end with \"let me know.\" |\n\n**Questions that surface pain in hours and dollars:**\n- How often do you ship customer-facing changes?\n- Who writes the release notes, and how long does each one take?\n- What happens after the notes are written? Who sees them?\n- What got missed or delayed last time?\n- What did you try before, and how long did it take?\n\n**Objections**\n\n| Objection | Response |\n|---|---|\n| \"We'll build it ourselves.\" | \"How long did the last internal attempt take, and who maintains it now?\" Compare the answer to the plan price. |\n| \"Too expensive.\" | \"What does one release's notes cost you in PM hours today?\" Compare against that. |\n| \"Not now.\" | \"When's your next release? Can I check back after it ships?\" Book the follow-up before hanging up. |\n| \"Send me some info.\" | Send the sample built from their changelog, then ask for 15 minutes. |\n| \"I need to check with engineering.\" | \"Who else needs to be on it? Happy to do 15 minutes with both of you.\" |\n\n## 6. Conversion from free to paid\n\nFree developers are not converting. The likely reason is that the person who would pay (the PM) never sees the value. Test one change at a time, one week per test:\n\n- **Invite test:** after a developer's first successful run, prompt them to invite a teammate to review the email preview. Measure how many invites go to non-developers.\n- **Warm-list test:** route free sign-ups at 50-300 person companies to the sample outreach in section 3 instead of waiting for them to upgrade.\n\nDo not spend more than two hours a week on free-tier product work during the next six weeks. The free list is a source of leads, not a funnel to fix yet.\n\n**Pricing:**\n- Keep list prices unchanged for the first six weeks.\n- Test annual billing with a discount on new deals only.\n- If the first five calls close without price objections, raise the entry tier. If calls stall on price, that is a signal to change positioning, not just price.\n\n## 7. Weekly operating rhythm\n\n| Block | Hours | Work |\n|---|---|---|\n| Outreach | 4 | About 80 personalized messages and follow-ups |\n| Calls | 3 | 6-7 calls of 20 minutes, plus prep |\n| Trials and follow-ups | 1.5 | Samples, trial setup, nudges |\n| Customer work | 1 | Interviews, quotes, introductions |\n| Friday review | 0.5 | Update the pipeline sheet and choose next week's three changes |\n\n**Planning assumptions (replace with measured values after week 2):**\n\n| Stage | Assumed rate | Weekly result at 80 messages |\n|---|---|---|\n| Message to reply | 20% | 16 replies |\n| Reply to call booked | 40% | 6-7 calls |\n| Call to paid | 30% | About 1.9 new paid teams |\n\nAt 1.9 teams per week, a $400 average price is on track. At $233, the same funnel gives about $450 of new MRR a week, which is not enough.\n\n**Pipeline sheet columns:** date, company, company size, role, source (warm list or new), message sent, reply, call date, trial started, price quoted, closed, MRR, churned.\n\n**Friday review (30 minutes):** conversion by stage, objections heard this week, one message change to test, one customer to ask for a quote or introduction.\n\n## 8. Customer-led growth\n\n- Every paying team gets a request each month: a quote, permission to name them, or one introduction. Target: 6 introductions in the first 6 weeks.\n- Once you have a measured result from a customer (for example, hours saved per release), write a one-page case study and use it in every message.\n- Optional, about one hour a week: post a before-and-after of a real public changelog and its generated release note on LinkedIn. It brings inbound PM interest without cold outreach.\n\n## 9. Milestones and decision gates\n\nPlanning path at a $400 average for new teams:\n\n| Date | Week | Cumulative new paid teams | Approx. MRR |\n|---|---|---|---|\n| 2026-10-22 | 2 | 3 | $2.6k |\n| 2026-11-19 | 6 | 11 | $5.8k |\n| 2026-12-31 | 12 | 22 | $10.2k |\n| 2027-02-11 | 18 | 32 | $14.2k |\n| 2027-04-08 | 26 | 47 | $20.2k |\n\n**Gates:**\n\n- **Week 2:** The warm list is worked, the six customer interviews are done, and the conversion rates in section 7 are measured. If the reply rate is below 10%, change the message before increasing volume.\n- **Week 6:** At least 11 new teams in trial or proposal. If not, the offer or the ICP is wrong. Revisit the customer interview findings before sending more messages.\n- **Week 12:** If MRR is below about $9k, the 10-hour model will not reach $20k by April. The options are to add part-time outreach help, to add a second channel, or to reset the target to something like $10k in six months. Make this decision at week 12, not later.\n\n## 10. What not to do\n\n- Do not run cold calls. The channel that worked is LinkedIn, and the calls are reserved for warm replies.\n- Do not message developers about buying. Use them to find the PM.\n- Do not rebuild the free-tier product while the paid funnel is still unmeasured.\n- Do not send templates without personalization. The response rate will not survive it.\n"}]}, "haiku-3": {"cost_usd": 0.01025, "seconds": 69, "output_tokens": 12913, "files": [{"name": "sales-playbook.md", "text": "# Patchnote Founder-Led Sales Playbook\n\nWritten 2026-10-08. Goal: $20k MRR by 2027-04-08. Starting point: $1.4k MRR from 6 paying teams.\n\n## 1. The math\n\n- Average revenue per paying team today: about $233/month ($1.4k / 6).\n- $20k at today's average needs about 86 paying teams. At $350 average, about 57.\n- Your time: 10 hours/week for 26 weeks is about 260 hours.\n\nThe funnel rates below are assumptions, not measurements. Replace them with real numbers in week 1.\n\n| Stage | Cold DM (no sample) | Sample-led (this playbook) |\n|---|---|---|\n| Reply rate | 5-10% | 25% (target) |\n| Reply to call | 50% | 60% (target) |\n| Call to paid or pilot | 25% | 30% (target) |\n| Touches per paying team | about 100 | about 22 |\n\nAt 30 sample-led touches a week, you get about 780 touches in six months, which is roughly 35 new teams. That lands near $8k-12k MRR depending on deal size. Closing the gap to $20k means one of three things:\n\n- About 50 new teams at about $370 average, or about 35 new teams at about $530 average (mostly top-tier deals).\n- More touches per week, which the 10-hour budget limits.\n- A channel that does not need you: referrals from paying teams and self-serve signups.\n\nPlan for $20k, but judge progress against the milestones in section 10. Month 6 at $20k needs deals to get bigger, not just more of them.\n\n## 2. Who to sell to\n\n- **Buyer:** the product manager (or head of product) at a 50-300 person B2B SaaS company that ships changes regularly. The six paying teams confirm this profile.\n- **Champion:** a developer who already tried Patchnote. Developers sign up; PMs buy. Use the developer's signup to open the door to the PM.\n- **Strongest signal:** a company where someone has made real API calls, not just signed up.\n\n## 3. Lists, in priority order\n\n1. **The 300 free signups, filtered.** Pull the list, look up each company's size, and keep those with 50-300 employees. Sort by API activity. For each match, identify the PM on LinkedIn.\n2. **The six paying teams' networks.** Ask each of them for two intros to peers who own release notes. Ask for a short quote once they see results.\n3. **Verified outbound.** Build lists of 50-300 person SaaS companies with a public changelog page. Check each company by hand. Do not buy unverified lists.\n\nOnly contact people about their company's use of Patchnote. Keep records of where each contact came from. If you email anyone, follow the opt-out rules that apply to where they live.\n\n## 4. The offer: sample first\n\nDo not open with a pitch. Open with a sample of their own release notes, generated from their public changelog and shown in the format their users would see. The sample is the hook. The call is the second step.\n\nGenerate samples in a batch on Monday using the Patchnote API. Aim for 10-15 samples a week. Each sample should use the last three real changelog entries from that company.\n\n## 5. Outreach sequence\n\n**Touch 1 (day 0), LinkedIn DM or email to the PM:**\n\n> Hi {first}, I ran {Company}'s recent changelog entries through Patchnote as a test. Here is what they look like as in-app release notes: {link to sample}. It takes 30 seconds to look. If it would help your next release, I can set it up on your real changelog. Would 20 minutes next week make sense?\n\n**Touch 1 variant, to the champion's PM:**\n\n> Hi {first}, {developer name} on your team tried Patchnote last month. I put together a sample of what your release notes could look like: {link}. No pitch attached. If it is not useful, tell me and I will stop.\n\n**Touch 2 (day 3):**\n\n> Quick follow-up on the sample. PMs often tell me release notes go out late or not at all because they come after the release is done. If that is true for you, a 20-minute setup fixes it. If not, I will leave you alone.\n\n**Touch 3 (day 7):**\n\n> Last note from me. If release notes are not on your plan this half, that is fine. Tell me and I will check back in {month}.\n\nRules: keep messages under 80 words, personalize the first line with something specific from their changelog, and never send the same message to two people at one company.\n\n## 6. The 20-minute call\n\nBook calls only after a reply. Do not cold call.\n\n1. **Frame (2 min):** \"I want to understand how your release notes work today. If Patchnote is not a fit, I will say so.\"\n2. **Current process (5 min):** \"Walk me through your last release. Who wrote the notes? How long did it take? Where do they appear?\"\n3. **Cost of the status quo (5 min):** \"What happens when notes are late? Have customers asked about something they did not know existed?\"\n4. **Show the sample (4 min):** Walk through their sample against their real changelog.\n5. **Next step (4 min):** \"If this fits, the next step is a 14-day pilot on your real changelog. What would have to be true for you to say yes?\"\n\nWrite down: the pain in their words, who owns release notes today, the tool they use now, release frequency, budget, the decision maker, and their deadline.\n\n## 7. Objections\n\n- **\"We'll build it ourselves.\"** \"How long did your last version take, and who maintains it now?\"\n- **\"Not a priority this quarter.\"** \"When is your next release? I can set up the sample for that release only.\"\n- **\"Send me some info.\"** \"I will. What is the one question you want answered? I will send a two-minute video and the sample, not a deck.\"\n- **\"Too expensive.\"** Tie it to hours saved per release. Do not discount first. If budget is the real issue, offer the lower tier.\n- **\"We need a security review.\"** Send a one-page security summary the same day. Set a date for the review and keep it short.\n\n## 8. Pricing and closing\n\n- Propose the tier that matches their release volume. If they ship weekly or more often, open with the top tier.\n- Offer a 14-day pilot on their real changelog, with the card taken up front at the tier they choose. Refund in full if they cancel within 14 days. Free pilots rarely convert, and paid pilots keep the buyer honest.\n- Ask for a named decision maker and the date of their next real release. Get the first release through the system before the pilot ends.\n- Give annual prepay a discount only in exchange for annual payment.\n\n## 9. After the sale\n\n- Get the first live release out within seven days of signing.\n- Day 14: check in on what their users saw and whether the PM is happy.\n- Day 30: ask for two peer intros and, if they are happy, a short quote for a case study.\n\n## 10. Weekly rhythm (10 hours)\n\n| Block | Hours | Work |\n|---|---|---|\n| Monday | 2 | Build list, generate 10-15 samples |\n| Tuesday to Thursday | 2.5 | Send about 30 touches (about 5 minutes each) |\n| Calls | 1.5 | 3-4 calls, including prep |\n| Pilot support | 2 | Onboarding and first releases for active pilots |\n| Friday | 1 | Pipeline review, follow-ups, one LinkedIn post with a sample |\n| Buffer | 1 | Overflow |\n\n## 11. Metrics and checkpoints\n\nTrack every week: touches sent, reply rate, calls booked, calls held, pilots started, paid conversions, MRR, and churned MRR.\n\n- **Day 30:** If the sample-led reply rate is under 10%, rewrite the first line and the sample format. If you are booking fewer than 3 calls a week, change the list.\n- **Day 60:** At least 6 pilots live. If not, revisit the offer.\n- **Day 90:** MRR of at least $8k. If not, revisit pricing and the non-founder channels (referrals and self-serve).\n\nMilestones: Month 1 $2.5k, Month 2 $4.5k, Month 3 $8k, Month 4 $12k, Month 5 $16k, Month 6 $20k.\n\n## 12. What not to do\n\n- No cold calls.\n- No automated bulk LinkedIn tools. They risk the account, and the sample is what makes the outreach work.\n- No demos before discovery.\n- No discounts before a real budget objection.\n- No custom features for a prospect unless the feature unlocks at least two pilots.\n\n## 13. Product changes that help sales\n\nThese are suggestions for you to test. They are not part of the sales motion.\n\n- A shareable sample page that a developer can send to their PM.\n- An \"invite your PM\" action in the dashboard.\n- A usage alert to you when a signup reaches a set number of API calls. That is your hottest lead.\n\n## 14. First 30 days\n\n- **Week 1:** Pull and filter the 300 signups. Message the six paying teams for intros and a quote. Reconstruct how each of the six came in: which message, how many touches, and how long to close. Write the sample generator.\n- **Week 2:** Send the first 30 sample-led touches. Book the first 3 calls.\n- **Week 3:** Run calls. Start one pilot. Publish the first LinkedIn post with a sample.\n- **Week 4:** Review the metrics against the day-30 checkpoint. Adjust the message and the list.\n\n## Numbers that would sharpen this\n\n- How many DMs did the six paying teams take to close, and which message worked?\n- The price for each tier and what share of customers are on each.\n- Monthly churn so far.\n- How many of the 300 signups have made API calls, and how many are at companies of 50-300 people.\n"}]}}, "verdicts": [{"vs": "opus", "pair": "1", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "haiku+skill", "why": "A better respects buyer evidence and product facts, despite materially understating required new customers and lacking six-month execution detail. B provides stronger outreach and pilots but invents non-conversion causes, capabilities, and setup times; its workload accounting is incomplete."}, {"vs": "opus", "pair": "1", "judge": "gpt", "order": "opus shown first", "winner": "haiku+skill", "why": "B better respects buyer evidence and product facts. A invents non-conversion causes and capabilities, and budgets 11 hours. B materially understates additional customers needed and lacks complete outreach, implementation, and six-month execution details."}, {"vs": "opus", "pair": "1", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "tie", "why": "Both fully meet all criteria, correctly using evidence, arithmetic, and founder constraints."}, {"vs": "opus", "pair": "1", "judge": "deepseek", "order": "opus shown first", "winner": "tie", "why": "Both playbooks fully meet all criteria, correctly targeting PMs, using evidence, respecting constraints, and providing actionable plans."}, {"vs": "haiku", "pair": "1", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "haiku+skill", "why": "A better follows proven buyers and separates hypotheses from evidence. B assumes free-user conversion potential and omits technical onboarding and its time cost. Both materially undercount required customers; A also lacks complete outreach assets and six-month execution."}, {"vs": "haiku", "pair": "1", "judge": "gpt", "order": "haiku shown first", "winner": "tie", "why": "A offers stronger outreach assets; B better respects buyer evidence and uncertainty. Both lack practical API onboarding and complete six-month capacity planning. A undercounts required customers; B incorrectly applies hypothetical prices to existing teams when calculating net additions."}, {"vs": "haiku", "pair": "1", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "tie", "why": "Both fully meet all criteria, correctly using evidence, arithmetic, and founder constraints."}, {"vs": "haiku", "pair": "1", "judge": "deepseek", "order": "haiku shown first", "winner": "tie", "why": "Both fully meet all criteria, offering actionable, evidence-based plans fitting the constraints."}, {"vs": "opus", "pair": "2", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "haiku+skill", "why": "A follows buyer evidence and tests hypotheses. B assumes sign-ups prove pain, invents capabilities, and overloads the time budget. Both lack concrete API activation steps; A lacks six-month execution detail and understates the required additional teams at $499."}, {"vs": "opus", "pair": "2", "judge": "gpt", "order": "opus shown first", "winner": "haiku+skill", "why": "B better separates evidence from hypotheses and avoids invented capabilities. A overstates pricing necessity and underbudgets sample production. Both lack technical activation criteria; B lacks six-month execution detail and understates required total teams at $499: 44, retaining current revenue."}, {"vs": "opus", "pair": "2", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "tie", "why": "Both fully meet all criteria with correct arithmetic, actionable plans, and evidence-based targeting."}, {"vs": "opus", "pair": "2", "judge": "deepseek", "order": "opus shown first", "winner": "opus", "why": "A provides more actionable, detailed instructions and a concrete weekly plan, while B is more abstract and cautious."}, {"vs": "haiku", "pair": "2", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "haiku+skill", "why": "A follows proven buyer evidence and tests non-conversion causes more carefully. B overprioritizes free users and underbudgets research, personalization, and onboarding. Both lack concrete API activation criteria; A lacks six-month progression, and neither states the $3,100 monthly net-MRR requirement."}, {"vs": "haiku", "pair": "2", "judge": "gpt", "order": "haiku shown first", "winner": "haiku+skill", "why": "B better separates evidence from hypotheses and budgets realistic outreach. A underbudgets research and support. Both lack concrete API activation steps. B\u2019s $499 scenario requires 44 total teams, not 43; neither states the $3,100 monthly net-MRR requirement."}, {"vs": "haiku", "pair": "2", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "tie", "why": "Both fully meet all criteria, correctly using evidence, arithmetic, and founder constraints."}, {"vs": "haiku", "pair": "2", "judge": "deepseek", "order": "haiku shown first", "winner": "haiku", "why": "A provides a more detailed, executable weekly plan and conversion funnel that fully respects the 10-hour constraint."}, {"vs": "opus", "pair": "3", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "haiku+skill", "why": "A better grounds targeting and discovery in evidence. Its $400 scenario understates required new teams. B invents non-conversion causes, capabilities and customer claims, overloads the time budget, and suggests annual billing remedies MRR shortfalls. Both omit practical API implementation."}, {"vs": "opus", "pair": "3", "judge": "gpt", "order": "opus shown first", "winner": "haiku+skill", "why": "B better separates evidence from hypotheses. A invents non-conversion causes and customer claims. Both omit developer-led API implementation and strain capacity. B\u2019s revenue scenario is wrong: 44 new $400 teams yield $19,000 MRR, retaining current revenue."}, {"vs": "opus", "pair": "3", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "tie", "why": "Both fully meet all criteria, correctly targeting PMs, using LinkedIn, fitting 10 hours, and using correct arithmetic."}, {"vs": "opus", "pair": "3", "judge": "deepseek", "order": "opus shown first", "winner": "tie", "why": "Both fully meet all criteria with actionable, evidence-based plans fitting the constraints."}, {"vs": "haiku", "pair": "3", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "haiku+skill", "why": "A better grounds targeting and discovery in evidence. Both omit technical implementation ownership and capacity priorities. A\u2019s $400 scenario mishandles retained revenue; B conflates pilots with customers and invents PM testimony and a 20-minute outcome."}, {"vs": "haiku", "pair": "3", "judge": "gpt", "order": "haiku shown first", "winner": "haiku+skill", "why": "B better investigates non-conversion and tests value hypotheses. A has stronger outreach but invents customer claims. Both omit developer implementation and capacity priorities. Revenue scenarios fall short: B\u2019s 44 new $400 teams yield $19,000; A counts pilots as paying teams."}, {"vs": "haiku", "pair": "3", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "tie", "why": "Both meet all rubric criteria, offering actionable, evidence-based plans fitting the founder's constraints."}, {"vs": "haiku", "pair": "3", "judge": "deepseek", "order": "haiku shown first", "winner": "tie", "why": "Both meet all rubric criteria, offering actionable, evidence-based plans fitting the constraints."}]}, {"id": "cold-2", "family": "cold-email", "prompt": "Draft a first outreach email to the CFO of a 250-person manufacturing company. Our product details are in product.md.", "input": "product.md", "skill": "braun-poke-the-bear", "input_text": "Product: Spendlane, catches duplicate supplier invoices and price deviations from contracts before payment.\nTypical finding: 0.8-1.5% of supplier spend paid twice or above contract price. Customers: 11 mid-size manufacturers in Germany and Austria.\nThe target CFO runs a company with about EUR 90M supplier spend, uses SAP Business One, and recently posted on LinkedIn that they are \"under pressure on margins with energy costs\".\nPrice: EUR 2,500/month, or a share of recovered money for the first 6 months.\n", "rubric": [{"id": "c1", "criterion": "Explain the product\u2019s relevance to the CFO in concrete business terms.", "full_marks_when": "Clearly connects catching duplicate supplier invoices and prices above contract before payment with reducing avoidable supplier costs and protecting margins.", "zero_when": "Leaves the product\u2019s function unclear, misrepresents it, or offers only generic efficiency claims."}, {"id": "c2", "criterion": "Make the outreach specific to this CFO\u2019s situation.", "full_marks_when": "Naturally connects the CFO\u2019s stated pressure on margins from energy costs with an actionable opportunity to reduce supplier-payment leakage; any reference to SAP Business One is relevant and does not imply an unverified integration.", "zero_when": "Reads as interchangeable mass outreach, ignores the margin concern, or claims the product reduces energy prices."}, {"id": "c3", "criterion": "Keep financial claims accurate and distinguish potential findings from guaranteed savings.", "full_marks_when": "If quantifying the opportunity, correctly calculates 0.8\u20131.5% of EUR 90M as EUR 720,000\u20131,350,000 annually, framed as an illustrative exposure based on typical findings rather than confirmed losses or guaranteed recoveries. If stating subscription cost, uses EUR 2,500/month or EUR 30,000/year.", "zero_when": "Miscalculates the amounts, uses the wrong spend base or period, or presents the typical finding as a verified loss or guaranteed saving for this company."}, {"id": "c4", "criterion": "Use credible evidence without inventing capabilities or commercial terms.", "full_marks_when": "Any customer proof accurately reflects 11 mid-size manufacturers in Germany and Austria. Any pricing reference accurately describes EUR 2,500/month or a share of recovered money for the first six months, without inventing a percentage, free trial, named customers, SAP integration, or implementation promises.", "zero_when": "Fabricates customer results, endorsements, technical capabilities, or pricing terms, or materially contradicts the supplied facts."}, {"id": "c5", "criterion": "Give the CFO an easy, proportionate next step.", "full_marks_when": "Ends with one clear, low-friction invitation to assess relevance, such as a short conversation, without demanding sensitive invoice data, extensive preparation, or an immediate purchase.", "zero_when": "Has no actionable next step, makes multiple competing requests, or asks for a burdensome commitment before establishing interest."}, {"id": "c6", "criterion": "Deliver a usable first outreach email suited to a busy executive.", "full_marks_when": "Provides a concise subject line and email body with a professional, direct tone, a clear reason for contacting the CFO, and minimal placeholders for unavailable names or sender details; requires no substantial rewriting.", "zero_when": "Provides advice instead of an email, buries the point in a long pitch, uses hype or pressure tactics, or invents a prior relationship."}], "runs": {"haiku-skill-1": {"cost_usd": 0.00387, "seconds": 17, "output_tokens": 2718, "files": [{"name": "cfo-outreach-email.md", "text": "**Subject:** Duplicate supplier payments at [Company]\n\nHi [First name],\n\nI saw your LinkedIn post about margin pressure from energy costs. With about EUR 90M in supplier spend, even small payment leaks add up quickly.\n\nSpendlane checks supplier invoices before they're paid, flagging duplicate invoices and prices that deviate from contract terms. Across our 11 mid-size manufacturers in Germany and Austria, we typically find 0.8\u20131.5% of supplier spend paid twice or above contract price. At your spend level, that would be roughly EUR 0.7\u20131.35M a year, if the pattern holds for [Company].\n\nPricing is EUR 2,500 per month, or a share of what we recover during the first six months.\n\nWould a 20-minute call next week be worth your time?\n\nBest regards,\n[Your name]\n[Title], Spendlane\n[Phone] | [Email]\n"}]}, "haiku-skill-2": {"cost_usd": 0.00391, "seconds": 15, "output_tokens": 2765, "files": [{"name": "cfo_outreach_email.md", "text": "Subject: Duplicate invoices and contract price deviations at [Company]\n\nHi [First name],\n\nYour LinkedIn post about margin pressure from energy costs stood out to me.\n\nSpendlane checks supplier invoices against contract prices before they're paid, and flags duplicates. In our work with 11 mid-size manufacturers in Germany and Austria, we typically find 0.8-1.5% of supplier spend paid twice or above contract price. On your roughly EUR 90M of supplier spend, that range would be roughly EUR 0.7-1.35M a year.\n\nFor the first six months, we can work on a contingency basis: we take a share of what we recover, so there's no fixed cost while you check our results.\n\nWould a 20-minute call next week make sense? I can walk you through what we would check in your supplier invoices.\n\nBest regards,\n[Your name]\n[Title], [Company]\n"}]}, "haiku-skill-3": {"cost_usd": 0.00406, "seconds": 19, "output_tokens": 3478, "files": [{"name": "cfo_outreach_email.md", "text": "Subject: Supplier payments at [Company]: a quick check\n\nHi [First name],\n\nYour recent LinkedIn post about margins under pressure from energy costs stood out. When margins are tight, money already spent on suppliers is worth a second look.\n\nWe're Spendlane. We check supplier invoices against contract prices and earlier invoices before they're paid. For mid-size manufacturers we work with in Germany and Austria, the typical finding is 0.8\u20131.5% of supplier spend paid twice or above contract price. At your scale of roughly EUR 90M in supplier spend, that range comes to about EUR 0.7\u20131.4M a year. It's a typical range, not a promise, and we would verify it against your own payment data first.\n\nWe can start on a success-share basis for the first six months, so the first step carries no fixed cost.\n\nWould a 20-minute call next week make sense? I can also show you what a check of your last 12 months of supplier payments would involve.\n\nBest regards,\n[Your name]\n[Title], Spendlane\n[Phone] | [Email]\n"}]}, "opus-1": {"cost_usd": 0.07582, "seconds": 21, "output_tokens": 1610, "files": [{"name": "cfo-outreach-email.md", "text": "# First Outreach Email: CFO, 250-person manufacturer\n\n**Subject line options**\n1. A margin lever that isn't energy\n2. ~EUR 1M of your supplier spend, paid twice?\n3. Duplicate invoices at ~EUR 90M spend\n\n---\n\nHi [First name],\n\nI saw your LinkedIn post about margin pressure from energy costs. You can't do much about energy prices. Money leaking out through accounts payable is a different story, and you can fix it.\n\nAcross the 11 mid-size manufacturers in Germany and Austria we work with, Spendlane typically finds that 0.8\u20131.5% of supplier spend was paid twice or above the contracted price. With supplier spend of around EUR 90M, that would be roughly **EUR 720K to 1.35M a year**.\n\nSpendlane checks supplier invoices against earlier payments and your contract prices *before* they're paid, so duplicates and price deviations get caught before the money leaves, not months later in an audit.\n\nTo make this low-risk: for the first 6 months you can pay us a share of what we recover instead of a fixed fee. If we don't find anything, you don't pay.\n\nWould you be open to a 20-minute call in the next couple of weeks? I can walk you through what we usually find at companies your size.\n\nBest regards,\n[Your name]\n[Title], Spendlane\n[Phone] \u00b7 [Email]\n\n---\n\n## Notes before sending\n\n- **The math:** 0.8\u20131.5% \u00d7 EUR 90M = EUR 720K\u20131.35M. The standard price (EUR 2,500/month = EUR 30K/year) is about 2\u20134% of the low end, so mention it if they ask, but leave it out of the first email.\n- **SAP Business One:** The CFO uses SAP B1, but product.md doesn't say whether Spendlane integrates with it. If it does, add a line like *\"We connect directly to SAP Business One, so your team doesn't change how it works.\"* This is often the main objection, so it's worth confirming.\n- **Language:** The existing customers are in DACH. If this CFO is too, consider sending a German version.\n- **Proof:** If any of the 11 customers will let you name them or share a result (e.g., \"a 300-person Austrian machine builder recovered EUR X\"), add one sentence to paragraph 2. A specific example works better than a range.\n- **Revenue-share terms:** product.md doesn't give the percentage. Leave it out of this email, but have the number ready for the call.\n"}]}, "opus-2": {"cost_usd": 0.07716, "seconds": 19, "output_tokens": 1664, "files": [{"name": "outreach-email-cfo.md", "text": "# First outreach email: CFO, 250-person manufacturer\n\n**To:** [CFO name]\n**From:** [Your name], Spendlane\n\n## Subject line options\n1. Margin you've already paid for\n2. Before energy costs, check what you're paying suppliers twice\n3. [Company]: an estimated \u20ac0.7\u20131.35M in supplier overpayments\n\n---\n\n**Subject:** Margin you've already paid for\n\nHi [First name],\n\nI saw your LinkedIn post about margin pressure from energy costs. You can't do much about energy prices. There is another leak that usually gets less attention, and it's in accounts payable.\n\nAcross the 11 mid-size manufacturers in Germany and Austria we work with, we usually find that **0.8\u20131.5% of supplier spend was paid twice or above the contracted price**. With supplier spend of around \u20ac90M, that would put [Company] at roughly **\u20ac720K to \u20ac1.35M a year**. That money goes straight to margin if it's caught.\n\nSpendlane checks supplier invoices before payment. It flags duplicates and any line priced above what the contract says, so the money never leaves your account and nobody has to chase recoveries afterwards.\n\nTo make this low-risk to try, we can work on a share of what we recover for the first 6 months instead of a fixed fee. If we find nothing, you pay nothing.\n\nWould you be open to a 20-minute call in the next two weeks? I can show you what this kind of check usually turns up in a company your size.\n\nBest regards,\n[Your name]\n[Title], Spendlane\n[Phone] \u00b7 [Email]\n\n---\n\n## Notes before sending\n\n- **Numbers:** \u20ac720K\u20131.35M is 0.8\u20131.5% of the ~\u20ac90M supplier spend in product.md. It's an estimate based on what other customers saw, and the email says so (\"would put... roughly\").\n- **SAP Business One:** I left it out on purpose. product.md doesn't say whether Spendlane integrates with SAP B1. If it does, add one line after the third paragraph, e.g. \"It works on top of SAP Business One, so your AP team keeps its current process.\" Being specific about their setup helps a lot, but only if the claim is true.\n- **Pricing:** The email only mentions the performance-based option, because it's the easiest thing for a first-time contact to agree to. Save the \u20ac2,500/month price (\u20ac30K/year, about 2\u20134% of the estimated finding) for the call.\n- **Language:** If the CFO is based in Germany or Austria, a German version (formal \"Sie\") will probably get more replies. I can draft one.\n- **Customer names:** If any of the 11 customers will act as a reference, naming one (or a similar company) will make the email more credible.\n- **Fill in:** [CFO name], [First name], [Company], and your signature details.\n"}]}, "opus-3": {"cost_usd": 0.08219, "seconds": 23, "output_tokens": 1908, "files": [{"name": "outreach-email-cfo.md", "text": "# First outreach email: CFO, 250-person manufacturer\n\n**Subject:** Margin you may already have paid out\n\n*(Alternatives: \"Duplicate invoices at ~EUR 90M supplier spend\" / \"Re: your post on energy costs and margins\")*\n\n---\n\nHi [First name],\n\nI saw your LinkedIn post about the pressure energy costs are putting on margins. You can't do much about energy prices. Money that goes out the door by mistake is a different story.\n\nSpendlane checks supplier invoices before they're paid. It flags duplicates and any prices above what your contracts say. We work with 11 mid-size manufacturers in Germany and Austria, and we usually find that 0.8\u20131.5% of supplier spend was paid twice or above the contract price.\n\nFor a company with around EUR 90M in supplier spend, that range works out to roughly **EUR 700k\u20131.35M a year**. That's my estimate based on our typical findings. Your number could be higher or lower.\n\nTo keep the risk low, you can choose how you pay for the first six months: a flat EUR 2,500 a month, or a share of whatever we actually recover. If we find nothing, the second option costs you nothing.\n\nWould a 20-minute call in the next couple of weeks make sense? I can show you what we usually turn up and what we'd need from your side to run a first check.\n\nBest regards,\n[Your name]\n[Title], Spendlane\n[Phone] \u00b7 [Email]\n\n---\n\n## Notes before sending\n\n- **SAP Business One:** product.md doesn't say whether Spendlane connects to SAP Business One, so the email doesn't mention it. If there's a native connector or a proven export route, add one line after the second paragraph, e.g. \"We work directly with SAP Business One, so your team doesn't have to set anything up.\" That line would remove a big objection.\n- **Language:** The target is probably in the DACH region. Think about sending this in German (Sie form), or at least sending the English version with a German signature.\n- **The EUR figure** is simply 0.8\u20131.5% \u00d7 EUR 90M. Keep the hedge (\"my estimate\") so it doesn't come across as a promise.\n- **Pricing in a first email:** I included it because the recovery-share option is the strongest hook for a CFO. If you'd rather keep price for the call, replace that paragraph with: \"We can also work on a share of recovered money, so there's no upfront cost.\"\n- **Social proof:** If any of the 11 customers is willing to be named, or you have one concrete result (e.g. \"a 300-person Austrian machine builder recovered EUR 410k\"), that would beat the generic range.\n"}]}, "haiku-1": {"cost_usd": 0.0038, "seconds": 15, "output_tokens": 2604, "files": [{"name": "cfo-outreach-email.md", "text": "**Subject:** Duplicate supplier payments at [Company]?\n\nHi [First name],\n\nI saw your LinkedIn post about margin pressure from energy costs. At roughly EUR 90M in supplier spend, small billing errors add up quickly.\n\nWe built Spendlane, which checks supplier invoices against contract prices and payment history before they are paid. Across the mid-size manufacturers we work with in Germany and Austria, we typically find 0.8\u20131.5% of supplier spend paid twice or above contract price. At your scale, that would be roughly EUR 0.7\u20131.35M a year, though the real number depends on your invoice flow.\n\n[Confirm before sending: Spendlane works with SAP Business One.]\n\nWould a 20-minute call next week make sense? If useful, I can also run the check on a sample of your recent supplier invoices so you can see the findings before we talk. [Optional offer. Confirm before sending.]\n\nBest regards,\n\n[Your name]\n[Title], Spendlane\n[Phone] \u00b7 [Email]\n"}]}, "haiku-2": {"cost_usd": 0.00361, "seconds": 17, "output_tokens": 2798, "files": [{"name": "cfo-outreach-email.md", "text": "**Subject:** Supplier payments and the energy-cost squeeze\n\nHi [First name],\n\nYou mentioned on LinkedIn that margins are under pressure from energy costs. Many mid-size manufacturers are in the same position, and supplier payments are one place where money leaks without much notice: duplicate invoices, and prices that drift above what was agreed in the contract.\n\nSpendlane checks supplier invoices before they are paid and flags both. Across the 11 mid-size manufacturers in Germany and Austria we work with, we typically find 0.8\u20131.5% of supplier spend paid twice or above contract price. If your supplier spend is around EUR 90M a year, that range is roughly EUR 0.7\u20131.4M.\n\nPricing is EUR 2,500 a month. For the first six months you can instead pay a share of what we recover.\n\nWould a 20-minute call next week make sense? I would like to hear how supplier invoices are checked before payment today, and whether a closer look is worth your time.\n\nBest regards,\n\n[Your name]\n[Title], Spendlane\n[Phone] | [Email]\n"}]}, "haiku-3": {"cost_usd": 0.00379, "seconds": 14, "output_tokens": 2597, "files": [{"name": "cfo-outreach-email.md", "text": "Subject: Duplicate supplier payments at a manufacturer your size\n\nHi [First name],\n\nI read your LinkedIn post about margin pressure from energy costs, and I wanted to share something that often helps with exactly that.\n\nSpendlane checks supplier invoices before payment for duplicates and for prices above contract terms. Across the eleven mid-size manufacturers in Germany and Austria we work with, we typically find 0.8 to 1.5% of supplier spend paid twice or above contract price. On your roughly EUR 90M of supplier spend, that range would be about EUR 0.7M to 1.35M.\n\nI don't know yet whether the same pattern shows up in your payables. If it's worth finding out, would a 20-minute call next week work? I can show you what we look for and how it would run against your SAP Business One invoice data.\n\nBest regards,\n\n[Your name]\n[Title], Spendlane\n[Phone] | [Email]\n"}]}}, "verdicts": [{"vs": "haiku", "pair": "1", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "haiku+skill", "why": "A is accurate, conditional and ready to send; EUR 0.7M is reasonable rounding of EUR 720,000. B includes an unverified SAP capability, an unsupported sample-check offer and competing next steps, requiring edits before use."}, {"vs": "haiku", "pair": "1", "judge": "gpt", "order": "haiku shown first", "winner": "haiku+skill", "why": "B is ready to send, uses accurate customer proof and pricing, and offers one simple next step. A includes unresolved SAP compatibility and sample-check offers, requires editing, and adds an invoice-data request. Both reasonably round the illustrative financial range."}, {"vs": "haiku", "pair": "1", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "haiku+skill", "why": "Deliverable B introduces an unverified integration claim and an optional, potentially burdensome next step, violating C4."}, {"vs": "haiku", "pair": "1", "judge": "deepseek", "order": "haiku shown first", "winner": "haiku+skill", "why": "B accurately states pricing and customer count; A has a placeholder requiring confirmation about SAP."}, {"vs": "opus", "pair": "1", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "haiku+skill", "why": "A is concise, specific, and appropriately conditional; its figures are reasonably rounded. B invents a no-findings/no-payment promise, misstates annual pricing as 2\u20134% of the low-end opportunity, and requires removing notes and selecting a subject."}, {"vs": "opus", "pair": "1", "judge": "gpt", "order": "opus shown first", "winner": "haiku+skill", "why": "B is concise, tailored, and explicitly conditional about potential findings; its rounded estimate is reasonable. A invents a no-findings/no-payment promise, misstates the fee-to-opportunity ratio in its notes, and needs subject selection and tone cleanup."}, {"vs": "opus", "pair": "1", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "haiku+skill", "why": "B's 'Notes' section suggests adding unverified SAP integration and customer proof, violating c4."}, {"vs": "opus", "pair": "1", "judge": "deepseek", "order": "opus shown first", "winner": "tie", "why": "Both emails are highly effective, meeting all criteria with accurate, specific, and professional outreach."}, {"vs": "opus", "pair": "2", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "haiku+skill", "why": "A is concise and ready to send, although its lower estimate should be EUR 720K. B calculates correctly but overstates invoice flagging as ensuring money never leaves and adds unnecessary subject options and sending notes."}, {"vs": "opus", "pair": "2", "judge": "gpt", "order": "opus shown first", "winner": "haiku+skill", "why": "B is concise, relevant, and frames the rounded financial range conditionally. A implies company-specific leakage more strongly and adds an unsupported \u201cfind nothing, pay nothing\u201d guarantee. Its extra subject options and notes also require more cleanup."}, {"vs": "opus", "pair": "2", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "tie", "why": "Both emails fully meet all rubric criteria with equal clarity, accuracy, and appropriateness."}, {"vs": "opus", "pair": "2", "judge": "deepseek", "order": "opus shown first", "winner": "opus", "why": "A's financial calculation is precise; B rounds incorrectly, losing a point on accuracy."}, {"vs": "haiku", "pair": "2", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "tie", "why": "Both connect margin pressure to supplier-payment leakage, use supplied evidence and terms, and offer a proportionate call. Both frame estimates conditionally but round the opportunity instead of giving the rubric\u2019s EUR 720,000\u20131,350,000 annual range."}, {"vs": "haiku", "pair": "2", "judge": "gpt", "order": "haiku shown first", "winner": "tie", "why": "Both connect invoice leakage to margin pressure, accurately describe the product and customer evidence, and offer a proportionate call. Their approximate ranges reasonably round EUR 720,000\u20131,350,000 without guaranteeing savings. Both pricing descriptions reflect the supplied alternatives."}, {"vs": "haiku", "pair": "2", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "tie", "why": "Both emails are accurate, specific, and provide a clear, low-friction next step. Differences are minor."}, {"vs": "haiku", "pair": "2", "judge": "deepseek", "order": "haiku shown first", "winner": "haiku", "why": "Deliverable B misstates pricing by omitting the EUR 2,500/month option, violating C4."}, {"vs": "opus", "pair": "3", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "haiku+skill", "why": "A is more directly usable. Both round the correct EUR 720,000\u20131,350,000 opportunity and avoid guarantees. B adds unnecessary alternatives and notes, including suggested SAP wording that overpromises setup requirements even if an export route exists."}, {"vs": "opus", "pair": "3", "judge": "gpt", "order": "opus shown first", "winner": "haiku+skill", "why": "B is more concise and ready to send. Both round rather than state the exact EUR 720,000\u20131,350,000 range. A adds unnecessary alternatives and notes, including suggested implementation and commercial claims beyond the supplied facts."}, {"vs": "opus", "pair": "3", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "tie", "why": "Both emails fully meet all criteria, are accurate, specific, and offer a clear, low-friction next step."}, {"vs": "opus", "pair": "3", "judge": "deepseek", "order": "opus shown first", "winner": "opus", "why": "A's financial calculation is more precise (EUR 700k\u20131.35M) and framing is clearer; B rounds less accurately and is slightly less direct."}, {"vs": "haiku", "pair": "3", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "haiku", "why": "B gives exact customer proof and a more accurate upper estimate. Both appropriately qualify potential findings, but neither states the precise EUR 720,000\u20131,350,000 annual range; B also omits the period."}, {"vs": "haiku", "pair": "3", "judge": "gpt", "order": "haiku shown first", "winner": "haiku+skill", "why": "B makes the annual period and uncertainty clearer. Both round the correct EUR 720,000\u20131,350,000 range. A implies unsupported SAP compatibility; B suggests an unverified retrospective payment-review capability. Both offer an appropriate, simple next step."}, {"vs": "haiku", "pair": "3", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "haiku", "why": "B's financial calculation is more precise (EUR 0.7M-1.35M vs. A's rounded EUR 0.7-1.4M)."}, {"vs": "haiku", "pair": "3", "judge": "deepseek", "order": "haiku shown first", "winner": "tie", "why": "Both emails are excellent, meeting all criteria with accurate, specific, and professional outreach."}]}, {"id": "cold-3", "family": "cold-email", "prompt": "Write the cold email I'll send to Heads of Customer Support at e-commerce brands. Details in brief.md.", "input": "brief.md", "skill": "braun-poke-the-bear", "input_text": "Product: Returnly Desk, an inbox add-on that auto-answers \"where is my return/refund\" tickets using carrier and payment data.\nData from 9 customers: these tickets are 22-35% of support volume; we resolve 70% of them without an agent.\nTarget: Head of Support at a Shopify brand doing USD 20-80M revenue with an in-house team of 8-25 agents, using Gorgias or Zendesk.\nPeak season (November-December) is coming; support teams hire temps every year.\nPrice: USD 1,200/month.\n", "rubric": [{"id": "c1", "criterion": "Provides a complete, sendable cold email.", "full_marks_when": "Includes a relevant subject line and concise email body with a clear opening, value proposition, and next step; requires only basic sender or recipient substitutions.", "zero_when": "Provides advice, an outline, or a template requiring substantial writing instead of the requested email."}, {"id": "c2", "criterion": "Addresses the actual buyer and operating context.", "full_marks_when": "Speaks naturally to a Head of Support at a Shopify e-commerce brand with an in-house team using Gorgias or Zendesk, without awkwardly listing every targeting parameter or inventing recipient-specific facts.", "zero_when": "Targets consumers, unrelated businesses, or a different decision-maker, or fabricates knowledge of the recipient\u2019s revenue, staffing, or software."}, {"id": "c3", "criterion": "Makes the seasonal problem and practical benefit clear.", "full_marks_when": "Connects the approaching November\u2013December peak and recurring temporary hiring to reducing repetitive return/refund inquiries and freeing agent capacity, without promising that temporary hiring will become unnecessary.", "zero_when": "Omits the seasonal staffing relevance or guarantees eliminating temporary hires or a specific staffing reduction."}, {"id": "c4", "criterion": "Explains the product accurately and concretely.", "full_marks_when": "Describes Returnly Desk as an inbox add-on that automatically answers return/refund status tickets using carrier and payment data; keeps its scope clear and avoids unsupported implementation or integration claims.", "zero_when": "Misrepresents it as processing refunds, managing all support tickets, replacing the support platform, or offering unprovided capabilities or guarantees."}, {"id": "c5", "criterion": "Uses evidence and arithmetic faithfully.", "full_marks_when": "Uses the nine-customer evidence accurately: return/refund status tickets constitute 22\u201335% of support volume, and 70% of those tickets are resolved without an agent. If translating this into total support volume, states 15.4\u201324.5%, with appropriate qualification. Any price mentioned is USD 1,200/month; no savings, ROI, or broader results are invented.", "zero_when": "Claims 70% of all support tickets are resolved, gives incorrect figures or calculations, presents the customer results as guaranteed, or invents financial outcomes."}, {"id": "c6", "criterion": "Makes responding easy and keeps the tone credible.", "full_marks_when": "Ends with one clear, low-friction request relevant to evaluating the product; uses direct, professional language without hype, false urgency, invented familiarity, or unsupported offers.", "zero_when": "Has no actionable next step, makes multiple competing requests, demands a burdensome commitment, or relies on deceptive or exaggerated sales language."}], "runs": {"haiku-skill-1": {"cost_usd": 0.00322, "seconds": 13, "output_tokens": 2103, "files": [{"name": "cold-email.md", "text": "Subject: Where-is-my-refund tickets before peak\n\nHi {{first_name}},\n\nWhere-is-my-return and refund-status tickets are usually 22-35% of support volume for brands like {{company}}. Across 9 customers, Returnly Desk resolves about 70% of them without an agent. It answers from carrier tracking and payment data, so the customer gets their actual status without waiting in a queue.\n\nNovember and December are weeks away. If you hire temps for peak every year, this handles a large share of that volume, so you may need fewer of them and your permanent agents can spend their time on the tickets that need judgment.\n\nReturnly Desk is an inbox add-on at $1,200/month. Would a short call before peak be worth your time?\n\n{{your_name}}\n"}]}, "haiku-skill-2": {"cost_usd": 0.00412, "seconds": 13, "output_tokens": 2124, "files": [{"name": "cold-email.md", "text": "Subject: Peak season WISMO tickets, before the temps arrive\n\nHi [First name],\n\nWhere-is-my-return and refund tickets make up 22-35% of support volume at the brands we've worked with, and they're the easiest tickets to automate. Returnly Desk is an inbox add-on that answers them using carrier and payment data, so the customer gets the actual status instead of waiting in a queue.\n\nAcross 9 customers, about 70% of these tickets were resolved without an agent.\n\nWith November and December coming, your team is probably about to hire temps for the same repetitive questions. Returnly Desk is $1,200/month.\n\nWould it be useful to see what it would have handled in your last peak? I can run it on a sample of your tickets and send you the numbers.\n\nBest,\n[Your name]\n[Title, company]\n"}]}, "haiku-skill-3": {"cost_usd": 0.00308, "seconds": 11, "output_tokens": 1883, "files": [{"name": "cold-email.md", "text": "**Subject:** Where-is-my-refund tickets before peak\n\nHi [First name],\n\nReturn and refund-status tickets are 22-35% of support volume across our nine customers, and they resolve about 70% of those without an agent.\n\nReturnly Desk is an inbox add-on for Gorgias and Zendesk. It answers \"where is my return/refund\" tickets automatically, using carrier and payment data, so your agents only handle the ones that need a person.\n\nIf you're planning temp hires for November and December, this is worth a look before the volume arrives. It's USD 1,200 a month.\n\nWould a 15-minute call next week work?\n\n[Your name]\n[Title], [Company]\n"}]}, "opus-1": {"cost_usd": 0.09708, "seconds": 31, "output_tokens": 2470, "files": [{"name": "cold-email.md", "text": "# Cold email: Returnly Desk \u2192 Heads of Support\n\n**Audience:** Head of Support at a Shopify brand doing $20\u201380M in revenue, with an in-house team of 8\u201325 agents on Gorgias or Zendesk.\n**Send window:** now through late October, while peak-season temp hiring and training plans are still being decided.\n\n---\n\n## Subject line (pick one)\n\n1. `{{Company}} returns tickets before Black Friday`\n2. `Fewer \"where's my refund\" tickets this peak`\n3. `Before you hire this year's temps`\n\n---\n\n## Email\n\nHi {{FirstName}},\n\nOn the support teams we work with, \"where's my return / where's my refund\" tickets make up 22\u201335% of all volume. In November and December that's a big share of what the temps you're about to hire will spend their days answering.\n\nReturnly Desk is an add-on for {{Gorgias/Zendesk}}. It answers those tickets on its own, using the live carrier tracking and payment-processor data. Across our 9 customers, 70% of these tickets close without an agent ever touching them. That works out to roughly 15\u201325% of total ticket volume handled before it reaches your team.\n\nFor {{Company}}, that could mean fewer temps to hire and train this season, or the same headcount with time freed up for the tickets that actually need a person.\n\nWorth a 20-minute look at what it would do on your actual ticket mix?\n\n{{SenderName}}\n{{Title}}, Returnly Desk\n\n---\n\n## Follow-up 1 (send 3\u20134 business days later, same thread)\n\nHi {{FirstName}},\n\nOne more detail, since it usually comes up: Returnly Desk is a flat $1,200/month.\n\nIf you send me a rough count of your monthly return and refund status tickets, I'll tell you what share we'd expect to resolve, using the 70% rate our 9 customers see. It's a quick way to see whether it pays for itself before peak.\n\n{{SenderName}}\n\n---\n\n## Follow-up 2 (send ~1 week later, short breakup)\n\nHi {{FirstName}},\n\nI know peak prep is busy, so I'll close the loop. If \"where's my refund\" tickets are already under control at {{Company}}, no need to reply. If they're not, just reply \"later\" and I'll check back in January, once you can see what this season's volume actually looked like.\n\n{{SenderName}}\n\n---\n\n## Notes before sending\n\n- **Every claim comes from the brief:** 22\u201335% of volume, 70% resolved without an agent, 9 customers, $1,200/month. The \"15\u201325% of total volume\" figure is just 22\u201335% \u00d7 70% (15.4%\u201324.5%), rounded.\n- **Please confirm before sending:**\n  - **Integrations.** The email says it plugs into Gorgias/Zendesk. The brief only says \"inbox add-on\" and that targets *use* Gorgias or Zendesk. If either integration isn't live yet, change the line for that segment.\n  - **Setup time.** The peak-season angle only works if a prospect can be live before November. If onboarding is fast (e.g. \"live in a week\"), add that to the email, since it's the main objection this timing creates. If it's slow, move the pitch toward January planning.\n- **I left out:** dollar savings per temp (no data on temp costs) and customer names or logos (none in the brief). If any of the 9 customers would let you name them, a one-line reference (\"{{PeerBrand}} cut these tickets by X%\") would make this much stronger.\n- **Personalization tokens:** `{{FirstName}}`, `{{Company}}`, `{{Gorgias/Zendesk}}` (set per prospect from their helpdesk), `{{SenderName}}`, `{{Title}}`.\n"}]}, "opus-2": {"cost_usd": 0.08159, "seconds": 21, "output_tokens": 1841, "files": [{"name": "cold-email.md", "text": "# Cold email: Heads of Support at Shopify brands\n\n**Subject:** Before you hire peak-season temps\n\n(Other subject lines to test: \"Where is my refund?\" tickets in December / {{FirstName}}, about your return tickets)\n\n---\n\nHi {{FirstName}},\n\nAre you hiring temp agents for November and December again this year?\n\nIf so, here's something to check first. Across 9 Shopify brands we work with, \"where is my return?\" and \"where is my refund?\" tickets made up 22\u201335% of support volume. Those tickets get answered by looking up a carrier status and a payment record, then pasting the result back to the customer.\n\nReturnly Desk does that lookup and sends the reply from inside {{Gorgias/Zendesk}}. Across those 9 brands, it resolves 70% of these tickets without an agent touching them. That's roughly 15\u201325% of the whole inbox. Your agents get the rest, plus the cases that actually need a person.\n\nIt costs $1,200/month. Set that next to what a seasonal hire costs you to recruit, train and manage.\n\nWant to see it run on a sample of your own return tickets before you finalize temp headcount?\n\n{{YourName}}\n{{Title}}, Returnly Desk\n{{Calendar link or phone}}\n\n---\n\n## Notes before sending\n\n- **Merge fields:** `{{FirstName}}`, `{{Gorgias/Zendesk}}` (use whichever helpdesk the brand runs; leave it out if you don't know), sender details.\n- **Where the numbers come from:** 22\u201335% and 70% come from the 9-customer data in the brief. \"15\u201325% of the whole inbox\" is my own math (22% \u00d7 70% \u2248 15%, 35% \u00d7 70% \u2248 25%), not a measured figure. Drop that sentence if you only want to quote measured numbers.\n- **What I left out on purpose:** I didn't put a dollar figure on temp costs or claim a payback period. Nothing in the brief backs those up, so the email asks the reader to make the comparison themselves. If you have real temp-cost data from customers, add it.\n- **\"Shopify brands we work with\":** this assumes all 9 customers are Shopify brands. If they aren't, change it to \"Across 9 of our customers\".\n- **Timing:** temp hiring for peak season usually gets decided in October, so send this soon.\n- **Length:** about 150 words in the body, which reads fine on a phone.\n"}]}, "opus-3": {"cost_usd": 0.08197, "seconds": 22, "output_tokens": 1860, "files": [{"name": "cold-email.md", "text": "# Cold email: Heads of Support at Shopify brands\n\n**Subject:** {{Brand}}'s \"where's my refund\" tickets before November\n\n---\n\nHi {{FirstName}},\n\nHow many of the temps you hire for peak this year will spend their shifts answering \"where's my return?\" and \"when do I get my refund?\"\n\nAcross 9 Shopify brands using Returnly Desk, those two questions were 22\u201335% of all support tickets. Each answer is a lookup: check the carrier scan, check the payment processor, paste in the status.\n\nReturnly Desk adds onto {{Gorgias/Zendesk}} and does that lookup for you. It pulls the carrier and payment data and replies to the ticket. For those 9 brands, it resolved 70% of these tickets without an agent touching them. That works out to roughly 15\u201325% of total inbox volume gone.\n\nFor a team of {{8\u201325}} going into November, that could mean fewer temps to hire and train, or the same headcount spending peak on tickets that actually need a person.\n\nIt's $1,200/month.\n\nWorth a 20-minute call next week to see what share of {{Brand}}'s inbox this would cover?\n\n{{YourName}}\n{{Title}}, Returnly Desk\n\n---\n\n## Follow-up (send 4\u20135 business days later, same thread)\n\n**Subject:** Re: {{Brand}}'s \"where's my refund\" tickets before November\n\nHi {{FirstName}},\n\nOne quick way to size this: filter last December's tickets in {{Gorgias/Zendesk}} for \"return,\" \"refund,\" and \"tracking.\" If that comes to anything like the 22\u201335% our customers see, about 70% of it could have been answered without an agent.\n\nHappy to run that number with you on a short call.\n\n{{YourName}}\n\n---\n\n## Notes before sending\n\n- **Every number comes from the brief:** 9 customers, 22\u201335% of volume, 70% resolved without an agent, $1,200/month. The \"15\u201325% of total inbox volume\" is 22% \u00d7 70% \u2248 15% and 35% \u00d7 70% \u2248 25%.\n- **I left out a go-live promise.** The brief doesn't say how long setup takes. If you can get a brand live before Black Friday (Nov 27, 2026), say so in the email. With peak this close, that's the strongest line you could add.\n- **\"9 Shopify brands\"** assumes all 9 customers are Shopify brands. If they aren't, change it to \"9 of our customers.\"\n- **Price in the first email:** I put it in because $1,200/month is small next to the cost of even one seasonal hire, so it helps qualify leads. Cut it if you'd rather bring it up on the call.\n- **Personalization:** fill {{Gorgias/Zendesk}} from their tech stack (BuiltWith or job posts). If you know their team size, use the exact number instead of {{8\u201325}}.\n"}]}, "haiku-1": {"cost_usd": 0.00348, "seconds": 13, "output_tokens": 2114, "files": [{"name": "cold-email.md", "text": "Subject: Where-is-my-refund tickets this peak\n\nHi {{first_name}},\n\nPeak season is coming, and support teams hire temps for it every year. Most of the surge is \"where is my return or refund\" tickets.\n\nAcross 9 customers, those tickets were 22-35% of support volume, and 70% of them were resolved without an agent.\n\nReturnly Desk is an inbox add-on that answers those tickets automatically using carrier and payment data. It works with Gorgias and Zendesk and costs USD 1,200/month.\n\nWould a 15-minute call before November make sense? I can show you how it would handle your queue.\n\n{{your_name}}\n{{company}}\n"}]}, "haiku-2": {"cost_usd": 0.00318, "seconds": 11, "output_tokens": 2050, "files": [{"name": "cold-email.md", "text": "Subject: Where-is-my-refund tickets before peak\n\nHi {{first_name}},\n\nPeak season is about to push \"where is my return\" and \"where is my refund\" tickets to the top of your queue, and most teams cover that spike by hiring temps.\n\nReturnly Desk is an inbox add-on that answers those tickets automatically, using carrier and payment data. Across 9 customers, these tickets made up 22-35% of support volume, and about 70% of them were resolved without an agent.\n\nFor a team of 8-25 agents, that's a real share of the work to take off the queue before November.\n\nIt's USD 1,200 a month. Would a 15-minute call next week be worth it to check whether those numbers hold for {{company}}?\n\n{{your_name}}\n{{your_title}}\n"}]}, "haiku-3": {"cost_usd": 0.00301, "seconds": 10, "output_tokens": 1778, "files": [{"name": "cold-email.md", "text": "Subject: Where-is-my-refund tickets before peak\n\nHi {{First name}},\n\n\"Where is my return?\" and \"Where is my refund?\" make up 22-35% of support volume for the 9 brands we've worked with so far. Returnly Desk is an inbox add-on that answers those tickets automatically, using carrier and payment data, and it resolves about 70% of them without an agent.\n\nNovember and December are about to push that volume up, and most support teams bring in temps for it every year. If the refund and tracking tickets were handled before they reached an agent, your permanent team could stay on the harder cases and you might not need as many temps.\n\nWould a short call next week be useful to see whether it fits your setup? Pricing is USD 1,200/month.\n\nBest,\n{{Your name}}\n"}]}}, "verdicts": [{"vs": "opus", "pair": "1", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "haiku+skill", "why": "A is concise and sendable with accurate product scope. It slightly overgeneralizes the customer evidence. B assumes recipient hiring plans, asserts unverified integrations and live data, rounds the specified volume calculation, and adds unsupported evaluation offers."}, {"vs": "opus", "pair": "1", "judge": "gpt", "order": "opus shown first", "winner": "haiku+skill", "why": "B is immediately sendable and keeps product claims within scope. A requires integration verification, adds unsupported live-data claims, and rounds the specified volume calculation. B weakens buyer specificity and generalizes the nine-customer volume evidence."}, {"vs": "opus", "pair": "1", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "haiku+skill", "why": "Deliverable A is a complete, sendable email. Deliverable B provides a template and extra advice, not a single ready-to-send email."}, {"vs": "opus", "pair": "1", "judge": "deepseek", "order": "opus shown first", "winner": "opus", "why": "A provides a more complete, credible email with clearer next steps and follow-ups, while B's call-to-action is slightly less specific."}, {"vs": "opus", "pair": "2", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "haiku+skill", "why": "A preserves the evidence more faithfully. B invents a Shopify customer cohort, asserts unverified integration behavior, and rounds away the rubric\u2019s required 15.4\u201324.5% range. Both offer unsupported sample-ticket evaluations; A also calls these tickets the easiest to automate without evidence."}, {"vs": "opus", "pair": "2", "judge": "gpt", "order": "opus shown first", "winner": "haiku+skill", "why": "B describes the product and evidence more faithfully. A assumes all nine customers are Shopify brands, adds unsupported integration details, and rounds beyond the rubric\u2019s specified range. Both propose unsupported sample-ticket evaluations; B also omits the helpdesk context."}, {"vs": "opus", "pair": "2", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "tie", "why": "Both are complete, accurate, and contextually appropriate cold emails meeting all rubric criteria."}, {"vs": "opus", "pair": "2", "judge": "deepseek", "order": "opus shown first", "winner": "tie", "why": "Both are complete, accurate, and meet all criteria effectively; neither has a clear advantage."}, {"vs": "haiku", "pair": "1", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "haiku+skill", "why": "A connects automation to seasonal staffing and agent capacity without guarantees, though it overgeneralizes the sample. B invents that most surge tickets concern returns/refunds and asserts unprovided platform integrations. Its seasonal capacity benefit is less explicit."}, {"vs": "haiku", "pair": "1", "judge": "gpt", "order": "haiku shown first", "winner": "haiku+skill", "why": "B better connects peak hiring to agent capacity and accurately explains the product. It overgeneralizes the nine-customer volume findings. A invents a majority-of-surge claim and confirmed integrations, while leaving the practical staffing benefit implicit."}, {"vs": "haiku", "pair": "1", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "tie", "why": "Both are complete, accurate, and credible emails that meet all criteria equally well."}, {"vs": "haiku", "pair": "1", "judge": "deepseek", "order": "haiku shown first", "winner": "haiku", "why": "Deliverable B's suggestion to 'need fewer' temps overpromises, violating C3's zero-mark rule."}, {"vs": "haiku", "pair": "2", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "haiku", "why": "B accurately explains automation and offers a credible next step. A invents retrospective ticket-analysis capability and calls these tickets the easiest to automate without evidence. Neither clearly reflects the Shopify and Gorgias/Zendesk context."}, {"vs": "haiku", "pair": "2", "judge": "gpt", "order": "haiku shown first", "winner": "haiku", "why": "A offers a credible next step grounded in the brief. B invents a historical-ticket testing offer and claims these tickets are easiest to automate. Neither explicitly addresses Shopify or Gorgias/Zendesk; A awkwardly lists the staffing range."}, {"vs": "haiku", "pair": "2", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "haiku+skill", "why": "A's next step is lower-friction and more credible; B awkwardly inserts team size and uses a templated call request."}, {"vs": "haiku", "pair": "2", "judge": "deepseek", "order": "haiku shown first", "winner": "tie", "why": "Both are complete, accurate, and credible emails that meet all criteria effectively."}, {"vs": "haiku", "pair": "3", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "haiku", "why": "B accurately explains the product and evidence while connecting peak-season staffing to freed agent capacity. A asserts Gorgias and Zendesk compatibility, which the brief identifies as buyer context but never confirms as a product capability."}, {"vs": "haiku", "pair": "3", "judge": "gpt", "order": "haiku shown first", "winner": "haiku", "why": "A accurately explains the product, evidence, price, and seasonal capacity benefit without guaranteeing fewer hires. B is concise and relevant but asserts Gorgias and Zendesk compatibility that the brief identifies only as targeting context."}, {"vs": "haiku", "pair": "3", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "haiku+skill", "why": "B's 'you might not need as many temps' slightly overpromises, violating c3's zero condition."}, {"vs": "haiku", "pair": "3", "judge": "deepseek", "order": "haiku shown first", "winner": "tie", "why": "Both are complete, accurate, and meet all criteria equally well."}, {"vs": "opus", "pair": "3", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "haiku+skill", "why": "A is concise and sendable, with faithful evidence. Both assert unconfirmed platform compatibility. B invents Shopify customer provenance, rounds beyond the rubric\u2019s specified figures, and uses an overbroad follow-up filter that conflates status inquiries with other tickets."}, {"vs": "opus", "pair": "3", "judge": "gpt", "order": "opus shown first", "winner": "haiku+skill", "why": "B is sendable, concise, and faithful to the brief. A requires additional targeting substitutions, assumes all nine customers are Shopify brands, rounds beyond the rubric\u2019s specified figures, and broadens status-ticket evidence to keyword-filtered tickets in its follow-up."}, {"vs": "opus", "pair": "3", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "tie", "why": "Both are complete, accurate, and tailored to the buyer. A is more concise; B offers a follow-up and deeper analysis."}, {"vs": "opus", "pair": "3", "judge": "deepseek", "order": "opus shown first", "winner": "opus", "why": "A fully addresses seasonal context and uses arithmetic correctly; B is concise but underplays the seasonal problem and misstates the evidence."}]}, {"id": "home-2", "family": "homepage", "prompt": "We're redoing our website. Write the homepage. Everything is in notes.md.", "input": "notes.md", "skill": "pierri-homepage-positioning", "input_text": "Product: Fieldnote, a mobile app for property inspectors (rental move-in/move-out, insurance inspections). Photo-first checklists, auto-generated PDF reports with timestamps and GPS, client e-signature.\nCustomers: 210 small inspection businesses (1-10 inspectors), mostly UK. They switch from Word templates and generic form apps.\nProof: report time drops from about 90 minutes to 20; inspectors do 1-2 more inspections per day; 4.8 stars on the App Store (600 reviews).\nPrice: GBP 29 per inspector per month.\nCurrent homepage: lots of feature icons, hero says \"Inspections, reimagined.\"\n", "rubric": [{"id": "c1", "criterion": "The opening makes Fieldnote's purpose and value immediately clear.", "full_marks_when": "The hero identifies Fieldnote as an inspection app and gives a concrete benefit, such as turning inspection photos into client-ready reports faster, with a useful subheading rather than a vague slogan.", "zero_when": "The hero relies on generic claims such as 'Inspections, reimagined' and leaves visitors unsure what the product does or why they need it."}, {"id": "c2", "criterion": "The copy fits the actual buyers and their current workflow.", "full_marks_when": "Speaks directly to small property inspection businesses, reflects rental move-in/move-out and insurance inspections, and explains the benefit of switching from Word templates or generic form apps. Uses language and currency appropriate to a mainly UK audience.", "zero_when": "Targets unrelated audiences or enterprise operations, or offers generic productivity copy with no connection to property inspections and the tools buyers currently use."}, {"id": "c3", "criterion": "Explains how the product produces a useful inspection deliverable.", "full_marks_when": "Connects photo-first checklists, automatically generated PDF reports with timestamps and GPS, and client e-signature into an understandable workflow, explaining their practical benefits rather than merely listing features.", "zero_when": "Omits the core reporting workflow, presents only feature labels, or substitutes unsupported capabilities for the features in the notes."}, {"id": "c4", "criterion": "Uses the supplied evidence accurately and avoids invented claims.", "full_marks_when": "Accurately presents the approximate report-time change from 90 to 20 minutes, 1\u20132 additional inspections per day, 210 customer businesses, and a 4.8-star App Store rating from 600 reviews. Any calculated time saving is about 70 minutes per report, or 77.8% (approximately 78%). Does not turn reported results into guarantees or invent testimonials, endorsements, certifications, or capabilities.", "zero_when": "Makes material factual or arithmetic errors against the notes, such as claiming a 90% time reduction, misstates the rating or review count, or invents customer quotes or guaranteed results."}, {"id": "c5", "criterion": "Makes the price and next step clear without inventing commercial terms.", "full_marks_when": "States \u00a329 per inspector per month and includes a clear, relevant primary call to action. Avoids unsupported promises about free trials, discounts, cancellation terms, or a booking process.", "zero_when": "Omits or materially misstates the price, implies \u00a329 covers the whole business, provides no actionable next step, or promotes an invented offer."}, {"id": "c6", "criterion": "Delivers a complete homepage that can be used as copy.", "full_marks_when": "Provides coherent, concise homepage copy with a clear hero, benefits and workflow, credible proof, pricing, and calls to action in a sensible reading order. The text is specific enough to publish with minimal editing and does not require the user to write missing sections.", "zero_when": "Returns advice, a strategy, or an outline instead of the homepage, or supplies fragmented, repetitive, placeholder-heavy copy that needs substantial rewriting."}], "runs": {"haiku-skill-1": {"cost_usd": 0.01021, "seconds": 47, "output_tokens": 9228, "files": [{"name": "homepage.md", "text": "# Fieldnote homepage\n\n## Customer-facing copy\n\n### Hero\n\n**H1:** Finish inspection reports in 20 minutes, not 90.\n\n**Supporting line:** Fieldnote is the mobile app for property inspectors. Photo-first checklists for move-in, move-out and insurance inspections, with PDF reports built automatically, timestamped and GPS-tagged, and signed by your client in the app.\n\n**Primary CTA:** Download Fieldnote\n\n**Secondary link:** See how it works (anchors to the section below)\n\n**Product view:** Two screenshots side by side: a checklist item with photos attached, and the first page of the generated PDF report. *Asset needed.*\n\n### The current way\n\n**Heading:** Reports shouldn't take the rest of your day.\n\nMany inspectors still build reports in Word templates or generic form apps. Photos sit in one place, notes in another, and the report gets assembled afterwards. Every inspection ends with the same assembly job.\n\n### How Fieldnote works\n\n**Heading:** From first photo to signed report, in one app.\n\n1. **Check.** Work through your checklist and take photos against each item.\n2. **Report.** Fieldnote builds the PDF from your checklist and photos. Reports carry timestamps and GPS.\n3. **Sign.** Your client signs the report in the app.\n\n### Value: reports in about 20 minutes\n\n**Heading:** Report time falls from about 90 minutes to about 20.\n\nBecause the report is built from the inspection itself, there is no separate writing stage. Inspectors who have moved to Fieldnote report that a finished report now takes about 20 minutes, down from about 90.\n\n### Value: more inspections per day\n\n**Heading:** Finish the report, start the next inspection.\n\nShorter reports leave time for more work. Inspectors using Fieldnote fit in one to two more inspections a day.\n\n### Value: a record you can show\n\n**Heading:** Timestamps and GPS on every report.\n\nEach report shows when and where the inspection was done. Your client signs it in the app, so the signed version is the one you send.\n\n### Proof\n\n- 4.8 stars on the App Store, from 600 reviews.\n- Used by 210 inspection businesses, most of them in the UK, with one to ten inspectors each.\n- *[Customer quote: gap. Needs a named customer and permission.]*\n\n### Pricing\n\n**\u00a329 per inspector per month.**\n\n### Closing\n\n**Heading:** Stop writing inspection reports in Word.\n\nDownload Fieldnote and run your next inspection in the app. \u00a329 per inspector per month.\n\n**CTA:** Download Fieldnote\n\n---\n\n## Internal notes (remove before publishing)\n\n### Positioning decision\n\n- **Recommended bet:** Switch from the current way. The champion is the inspector who writes the reports at a firm of one to ten inspectors. The comparator is Word templates and generic form apps. The hero pairs the job (inspection reports) with a measurable change (about 20 minutes, not about 90).\n- **Why this over the alternatives:**\n  - *Category-led* (\"The inspection app for property inspectors\"). Kept as a fallback. The category name has no proof in the notes, and buyers using Word may not recognise it.\n  - *Insurance-inspection entry.* Narrower, and it depends on segment data that is not in the notes.\n- **Why not \"Inspections, reimagined\":** It names no job, no comparator, and no change. The icon grid also gave visitors nothing concrete to recognise themselves in.\n- **Deliberately left out:** Other customer segments and any enterprise motion. The notes do not cover them, and they would dilute the hero.\n\n### Claims to confirm before publishing\n\n| Claim | Gap |\n| --- | --- |\n| 90 to 20 minutes | Source, how it was measured, and whether it is a customer report or a timed test. The hero uses it, so this matters most. |\n| 1 to 2 more inspections a day | Source and sample. |\n| 4.8 stars, 600 reviews | Current App Store figures at launch. |\n| 210 businesses, mostly UK | Current count. |\n| Timestamps and GPS on every report | Confirm this holds for all report types. |\n| Client e-signature | Confirm it is in the current release. |\n\n### Missing inputs\n\n- **CTA destination:** App Store only, or Google Play too? Is there a free trial? The notes say neither.\n- **Screenshots:** Needed for the product view.\n- **Customer quote:** Needs a named customer and their permission.\n\n### Review status\n\nThis is a desk draft. No one outside the team has read it, and no inspector has checked the workflow description. Before launch, show the H1 and supporting line to three inspectors who are not on the team, and ask what Fieldnote is, who it is for, and what it replaces. If they cannot answer, return to the hero before changing the copy.\n\nThis page can make the offer clear. It cannot promise more sign-ups or revenue; that depends on the rest of the acquisition and product.\n"}]}, "haiku-skill-2": {"cost_usd": 0.0123, "seconds": 65, "output_tokens": 13288, "files": [{"name": "homepage.md", "text": "# Fieldnote homepage\n\nBracketed items are placeholders or confirmations. Remove them before publishing. See positioning-notes.md for the reasoning and open items.\n\n**Meta title:** Fieldnote: property inspection app with automatic PDF reports\n**Meta description:** Photo-first checklists, timestamped PDF reports with GPS, and client e-signature for move-in, move-out and insurance inspections.\n\n---\n\n## Hero\n\n# Property inspection reports in about 20 minutes, not 90.\n\nFieldnote is the photo-first app property inspectors use for move-in, move-out and insurance inspections. Capture photos and checklists on site, and get a timestamped PDF report with client e-signature.\n\n[Download Fieldnote] (App Store link to confirm)   [See how it works] (anchor to product view)\n\n---\n\n## Product view\n\n[Asset needed: short screen recording or screenshots showing one checklist item with photos attached, then the finished PDF report.]\n\nCaption: From the first photo on site to the finished PDF.\n\n---\n\n## Problem\n\n### Hours of write-up after every inspection.\n\nMost inspection businesses still build reports from Word templates and generic form apps. Photos get sorted, notes get retyped and the PDF gets put together by hand. A single report can take around 90 minutes, and that is time not spent on the next inspection.\n\n---\n\n## Solution\n\n### Do the inspection once. The report follows.\n\nFieldnote is built around photos. You work through the checklist on site and attach photos to the items they belong to. When you finish, the app generates the PDF report, so nothing has to be re-typed or re-assembled.\n\n---\n\n## Value 1: Photos stay with the checklist item\n\n### Each photo is filed against the item it proves.\n\nTake photos inside each checklist item. You don't have to scroll a camera roll later to work out which photo shows which room or meter.\n\n---\n\n## Value 2: Reports with timestamps and GPS\n\n### The report is generated, and it records when and where.\n\nEvery PDF report includes timestamps and GPS location, so the record shows when and where the inspection was done. [Confirm exactly what is stamped: per photo, per report, or both.]\n\n---\n\n## Value 3: Client sign-off in the app\n\n### Clients sign in the app, with no printing or scanning.\n\nClients add their e-signature to the report. There is no printed copy to sign and scan back in. [Confirm the signing flow: on the inspector's device, or via a link sent to the client.]\n\n---\n\n## Proof\n\n### Inspection businesses are switching to Fieldnote.\n\n- **210 inspection businesses** use Fieldnote, mostly in the UK.\n- **Report time drops from about 90 minutes to about 20.** [Confirm how this was measured before publishing.]\n- **Inspectors fit 1 to 2 more inspections into each day.** [Confirm this figure before publishing.]\n- **4.8 stars** from **600 reviews** on the App Store. [Re-check the current figures at launch.]\n\n[Customer quote: none approved yet. Add one with the customer's permission, or delete this line.]\n\n---\n\n## Who it's for\n\n### Built for the inspections you already do.\n\n- **Move-in and move-out inspections for rentals** \u2192 [link to rental inspections page]\n- **Insurance inspections** \u2192 [link to insurance inspections page]\n\n---\n\n## Pricing\n\n### One simple price.\n\n**\u00a329 per inspector per month.** Work solo or with a team of ten. [Confirm whether VAT is included or added at checkout.]\n\n[See pricing] (link to pricing page)\n\n---\n\n## Closing call to action\n\n### Your next report can take about 20 minutes.\n\nDownload Fieldnote and run your next inspection in it.\n\n[Download Fieldnote] (App Store link to confirm)\n"}, {"name": "positioning-notes.md", "text": "# Fieldnote homepage: positioning notes (internal, not for the page)\n\nStatus: provisional desk draft. No customer interviews, call notes, or analytics were available, so audience and comparator choices are assumptions to test.\n\n## Positioning\n\n- **Champion:** the owner-operator who does inspections and pays for seats. Assumed, because the notes don't name a role. Pricing is per inspector, so the person who does the work is probably the buyer.\n- **Job:** property inspection reports for move-in, move-out and insurance inspections.\n- **Comparator:** Word templates and generic form apps (from notes).\n- **Reference point:** the job, plus how the product improves on the current way (report time). This is the hero pattern from the skill, rather than a category claim.\n- **Differentiation:** photo-first checklists, automatic PDF reports with timestamps and GPS, and client e-signature. Generic form apps are not described as photo-first.\n- **Direct benefit:** less time writing up each report (about 90 minutes down to about 20, per notes).\n\nPosition sentence: For small UK property inspection businesses doing move-in, move-out and insurance inspections, Fieldnote is an inspection app whose photo-first checklists produce a timestamped PDF report with client e-signature, improving on Word templates by cutting report time.\n\nSelection status: recommended by the drafter, not chosen by a named stakeholder. The decision owner is not identified in the notes.\n\n## Removed from the current page\n\n- \"Inspections, reimagined.\" It could describe almost any software and names no job or comparator.\n- The feature icon grid. It lists features without an argument. The new page gives three value sections, each tied to a removed limitation.\n\n## Claims and evidence\n\n| Claim in copy | Source | Status |\n| --- | --- | --- |\n| 210 inspection businesses, mostly UK | Notes | Use. Confirm the count is current. |\n| Report time about 90 min to about 20 | Notes, no measurement method | Use with \"about\". Confirm how it was measured. Hold if it was self-estimated. |\n| 1 to 2 more inspections per day | Notes, unclear whether measured or self-reported | Confirm before publishing. |\n| 4.8 stars, 600 App Store reviews | Notes | Use. Re-check at launch. |\n| \u00a329 per inspector per month | Notes | Use. VAT treatment unknown. |\n| Photo-first checklists, timestamps, GPS, e-signature | Notes (product facts) | Use. Confirm the e-signature flow and what is stamped. |\n| Photos filed against checklist items; re-typing and assembling reports by hand | Inference from notes | Verify with a customer before publishing. |\n\n## Gaps\n\n- No approved customer quote. The proof section has a placeholder.\n- No screenshots or screen recording. The product view needs an asset.\n- App Store link is known. Google Play availability is not stated.\n- No free trial is mentioned in the notes, so the CTAs say \"Download\" rather than \"Start free trial\".\n- Champion and decision owner are not validated.\n- Nothing in the notes claims UK-specific compliance features. The copy does not claim any.\n\n## Review against the checklist (desk)\n\n- Evidence integrity: pass, with the inferences above flagged.\n- Correct comparator: pass. Word templates and generic form apps come from notes.\n- Causal value: pass. Each value section ties a feature to a removed limitation.\n- Honest outcomes: pass. No guaranteed conversion, no invented quotes, and the outcome figures are attributed as reported.\n- Reachable champion: pending. No acquisition data.\n- Comprehension check: pending. Test with someone outside the team: can they say what the product is, who it is for, and what it replaces?\n- Workflow check: pending. Ask two or three inspectors whether the photo-to-report flow matches their day.\n\n## Next steps\n\n1. Confirm the three bracketed figures and their measurement method.\n2. Get one customer quote with permission, and a screen recording.\n3. Test the hero with unfamiliar readers.\n"}]}, "haiku-skill-3": {"cost_usd": 0.01325, "seconds": 58, "output_tokens": 13001, "files": [{"name": "index.html", "text": "<!DOCTYPE html>\n<html lang=\"en-GB\">\n<head>\n<meta charset=\"utf-8\">\n<meta name=\"viewport\" content=\"width=device-width, initial-scale=1\">\n<title>Fieldnote: the photo-first inspection app for property inspectors</title>\n<meta name=\"description\" content=\"Fieldnote replaces Word templates with photo-first checklists and timestamped PDF reports with GPS, ready for client e-signature. \u00a329 per inspector per month.\">\n<style>\n  :root { --ink: #1d2328; --muted: #5b6670; --line: #e3e7ea; --accent: #1f6f5c; --bg: #ffffff; --soft: #f5f7f8; }\n  * { box-sizing: border-box; }\n  body { margin: 0; font-family: system-ui, -apple-system, \"Segoe UI\", Roboto, sans-serif; color: var(--ink); background: var(--bg); line-height: 1.6; }\n  .wrap { max-width: 960px; margin: 0 auto; padding: 0 24px; }\n  header { border-bottom: 1px solid var(--line); }\n  header .wrap { display: flex; align-items: center; justify-content: space-between; height: 64px; }\n  .logo { font-weight: 700; font-size: 20px; color: var(--ink); text-decoration: none; }\n  nav a { color: var(--muted); text-decoration: none; margin-left: 24px; font-size: 15px; }\n  .hero { padding: 80px 0 64px; }\n  h1 { font-size: 46px; line-height: 1.15; margin: 0 0 20px; max-width: 760px; letter-spacing: -0.01em; }\n  .lead { font-size: 20px; color: var(--muted); max-width: 680px; margin: 0 0 32px; }\n  .actions a { display: inline-block; padding: 14px 22px; border-radius: 6px; text-decoration: none; font-weight: 600; margin-right: 12px; }\n  .btn-primary { background: var(--accent); color: #fff; }\n  .btn-secondary { color: var(--accent); border: 1px solid var(--accent); }\n  .product-view { margin-top: 48px; height: 360px; border: 2px dashed #b9c2c9; border-radius: 10px; display: flex; align-items: center; justify-content: center; color: var(--muted); background: var(--soft); text-align: center; padding: 24px; }\n  section { padding: 64px 0; border-top: 1px solid var(--line); }\n  h2 { font-size: 32px; line-height: 1.2; margin: 0 0 16px; letter-spacing: -0.01em; }\n  h3 { font-size: 20px; margin: 0 0 8px; }\n  .value { display: grid; grid-template-columns: 1fr 1fr; gap: 32px; margin-top: 40px; }\n  .value article { background: var(--soft); border-radius: 10px; padding: 28px; }\n  .value p { margin: 0 0 12px; }\n  .benefit { font-weight: 600; color: var(--accent); margin: 0; }\n  .stats { display: grid; grid-template-columns: repeat(3, 1fr); gap: 24px; margin-top: 32px; }\n  .stat { border-top: 3px solid var(--accent); padding-top: 16px; }\n  .stat strong { display: block; font-size: 28px; margin-bottom: 4px; }\n  .stat span { color: var(--muted); }\n  .price { font-size: 40px; font-weight: 700; margin: 8px 0; }\n  .closing { text-align: left; }\n  footer { border-top: 1px solid var(--line); padding: 32px 0; color: var(--muted); font-size: 14px; }\n  @media (max-width: 720px) {\n    h1 { font-size: 34px; }\n    .value, .stats { grid-template-columns: 1fr; }\n    nav a { display: none; }\n  }\n</style>\n</head>\n<body>\n\n<header>\n  <div class=\"wrap\">\n    <a class=\"logo\" href=\"#\">Fieldnote</a>\n    <nav>\n      <a href=\"#how-it-works\">How it works</a>\n      <a href=\"#proof\">Results</a>\n      <a href=\"#pricing\">Pricing</a>\n    </nav>\n  </div>\n</header>\n\n<main>\n  <section class=\"hero\" style=\"border-top: none;\">\n    <div class=\"wrap\">\n      <h1>The photo-first inspection app that replaces Word templates</h1>\n      <p class=\"lead\">Fieldnote runs move-in, move-out and insurance inspections from your phone. Work through photo-first checklists, then send a PDF report with timestamps and GPS, ready for client e-signature.</p>\n      <div class=\"actions\">\n        <a class=\"btn-primary\" href=\"#\">Download on the App Store</a>\n        <a class=\"btn-secondary\" href=\"#pricing\">See pricing</a>\n      </div>\n      <!-- Asset needed: a finished report PDF showing timestamps, GPS and the signature block. Replace this placeholder before launch. -->\n      <div class=\"product-view\">[Screenshot needed: a finished PDF report showing timestamps, GPS and the e-signature block]</div>\n    </div>\n  </section>\n\n  <section>\n    <div class=\"wrap\">\n      <h2>Most inspection reports are still built by hand</h2>\n      <p>Small inspection businesses usually assemble each report from Word templates or generic form apps. Writing one takes about 90 minutes.</p>\n    </div>\n  </section>\n\n  <section id=\"how-it-works\">\n    <div class=\"wrap\">\n      <h2>Fieldnote takes the inspection from first photo to signed report</h2>\n      <p>Go through the checklist room by room, photograph each item as you go, and let Fieldnote generate the report.</p>\n      <div class=\"value\">\n        <article>\n          <h3>Photo-first checklists</h3>\n          <p>Each checklist item takes its photos on the spot. Photos stay with the item they document, so there is no separate folder to sort through afterwards.</p>\n          <p class=\"benefit\">Evidence is captured during the inspection, not rebuilt after it.</p>\n        </article>\n        <article>\n          <h3>Automatic PDF reports</h3>\n          <p>Fieldnote generates the PDF from your checklist, with timestamps and GPS included in the report.</p>\n          <p class=\"benefit\">Report time drops from about 90 minutes to about 20.</p>\n        </article>\n        <article>\n          <h3>Client e-signature</h3>\n          <p>Clients sign the report electronically in the app.</p>\n          <p class=\"benefit\">No printed copies to post, scan or chase.</p>\n        </article>\n        <article>\n          <h3>Built for small teams</h3>\n          <p>Designed for inspection businesses with 1 to 10 inspectors, covering rental move-in and move-out and insurance inspections.</p>\n          <p class=\"benefit\">Priced per inspector, so you pay for the people who inspect.</p>\n        </article>\n      </div>\n    </div>\n  </section>\n\n  <section id=\"proof\">\n    <div class=\"wrap\">\n      <h2>What changes after switching to Fieldnote</h2>\n      <div class=\"stats\">\n        <!-- Confirm source and method for the report-time figures before publishing. -->\n        <div class=\"stat\">\n          <strong>90 \u2192 20 min</strong>\n          <span>Time to produce a report</span>\n        </div>\n        <!-- Confirm how the inspections-per-day figure was measured before publishing. -->\n        <div class=\"stat\">\n          <strong>1\u20132 more</strong>\n          <span>Inspections completed per inspector each day</span>\n        </div>\n        <!-- Source: App Store ratings. -->\n        <div class=\"stat\">\n          <strong>4.8 \u2605</strong>\n          <span>On the App Store, from 600 reviews</span>\n        </div>\n      </div>\n    </div>\n  </section>\n\n  <section id=\"pricing\">\n    <div class=\"wrap\">\n      <h2>Simple pricing</h2>\n      <p class=\"price\">\u00a329 <span style=\"font-size:18px;font-weight:400;color:var(--muted);\">per inspector per month</span></p>\n      <p>For inspection businesses with 1 to 10 inspectors.</p>\n    </div>\n  </section>\n\n  <section class=\"closing\">\n    <div class=\"wrap\">\n      <h2>Run your next inspection in Fieldnote</h2>\n      <p>Replace the Word template with photo-first checklists and a report generated from your checklist.</p>\n      <div class=\"actions\">\n        <a class=\"btn-primary\" href=\"#\">Download on the App Store</a>\n      </div>\n    </div>\n  </section>\n</main>\n\n<footer>\n  <div class=\"wrap\">Fieldnote</div>\n</footer>\n\n</body>\n</html>\n"}]}, "opus-1": {"cost_usd": 0.20644, "seconds": 64, "output_tokens": 5943, "files": [{"name": "index.html", "text": "<!DOCTYPE html>\n<html lang=\"en-GB\">\n<head>\n<meta charset=\"utf-8\">\n<meta name=\"viewport\" content=\"width=device-width, initial-scale=1\">\n<title>Fieldnote: inspection reports done before you leave the property</title>\n<meta name=\"description\" content=\"Fieldnote is the mobile app for property inspectors. Photo-first checklists become timestamped, GPS-tagged PDF reports your client signs on site. About 20 minutes per report instead of 90.\">\n<style>\n  :root {\n    --ink: #1b1f1d;\n    --muted: #5a625e;\n    --line: #dfe3e0;\n    --bg: #ffffff;\n    --tint: #f4f6f3;\n    --accent: #1f6f4a;\n    --accent-ink: #ffffff;\n    --max: 1080px;\n  }\n  * { box-sizing: border-box; }\n  body {\n    margin: 0;\n    font-family: system-ui, -apple-system, \"Segoe UI\", Roboto, sans-serif;\n    color: var(--ink);\n    background: var(--bg);\n    line-height: 1.55;\n    font-size: 17px;\n  }\n  a { color: var(--accent); }\n  .wrap { max-width: var(--max); margin: 0 auto; padding: 0 24px; }\n  section { padding: 72px 0; }\n  h1, h2, h3 { line-height: 1.2; margin: 0 0 16px; }\n  h1 { font-size: clamp(2rem, 4.5vw, 3.1rem); letter-spacing: -0.02em; max-width: 18ch; }\n  h2 { font-size: clamp(1.5rem, 3vw, 2rem); letter-spacing: -0.01em; }\n  h3 { font-size: 1.1rem; }\n  p { margin: 0 0 16px; }\n  .lede { font-size: 1.2rem; color: var(--muted); max-width: 46ch; }\n  .muted { color: var(--muted); }\n\n  .btn {\n    display: inline-block;\n    padding: 14px 22px;\n    border-radius: 8px;\n    font-weight: 600;\n    text-decoration: none;\n    border: 2px solid var(--accent);\n  }\n  .btn-primary { background: var(--accent); color: var(--accent-ink); }\n  .btn-secondary { color: var(--accent); background: transparent; }\n  .actions { display: flex; flex-wrap: wrap; gap: 12px; margin-top: 28px; }\n\n  /* Header */\n  header { border-bottom: 1px solid var(--line); }\n  header .wrap { display: flex; align-items: center; justify-content: space-between; height: 64px; }\n  .logo { font-weight: 700; font-size: 1.2rem; color: var(--ink); text-decoration: none; }\n  nav a { margin-left: 24px; color: var(--ink); text-decoration: none; font-size: 0.95rem; }\n  nav a.btn { padding: 8px 16px; color: var(--accent-ink); }\n  @media (max-width: 640px) { nav a:not(.btn) { display: none; } }\n\n  /* Hero */\n  .hero { padding-top: 80px; }\n  .hero .note { margin-top: 16px; font-size: 0.95rem; }\n\n  /* Proof */\n  .proof { background: var(--tint); padding: 48px 0; }\n  .stats { display: grid; grid-template-columns: repeat(auto-fit, minmax(200px, 1fr)); gap: 32px; }\n  .stat strong { display: block; font-size: 2rem; letter-spacing: -0.02em; }\n  .stat span { color: var(--muted); }\n\n  /* Steps */\n  .steps { display: grid; grid-template-columns: repeat(auto-fit, minmax(260px, 1fr)); gap: 32px; margin-top: 40px; counter-reset: step; }\n  .step { border-top: 3px solid var(--accent); padding-top: 20px; }\n  .step::before { counter-increment: step; content: \"Step \" counter(step); display: block; font-size: 0.85rem; font-weight: 600; color: var(--accent); margin-bottom: 8px; text-transform: uppercase; letter-spacing: 0.05em; }\n\n  /* Compare */\n  .compare { display: grid; grid-template-columns: repeat(auto-fit, minmax(300px, 1fr)); gap: 24px; margin-top: 32px; }\n  .col { border: 1px solid var(--line); border-radius: 12px; padding: 28px; }\n  .col.after { border-color: var(--accent); }\n  .col ul { margin: 0; padding-left: 20px; }\n  .col li { margin-bottom: 10px; }\n\n  /* Uses */\n  .uses { display: grid; grid-template-columns: repeat(auto-fit, minmax(260px, 1fr)); gap: 24px; margin-top: 32px; }\n\n  /* Pricing */\n  .pricing { background: var(--tint); }\n  .price-card { background: var(--bg); border: 1px solid var(--line); border-radius: 12px; padding: 36px; max-width: 520px; }\n  .price { font-size: 2.6rem; font-weight: 700; letter-spacing: -0.02em; }\n  .price small { font-size: 1rem; font-weight: 400; color: var(--muted); }\n  .price-card ul { padding-left: 20px; margin: 20px 0 0; }\n  .price-card li { margin-bottom: 8px; }\n\n  /* Final CTA */\n  .final { text-align: center; }\n  .final h2 { max-width: 22ch; margin-left: auto; margin-right: auto; }\n  .final .actions { justify-content: center; }\n\n  footer { border-top: 1px solid var(--line); padding: 32px 0; font-size: 0.9rem; color: var(--muted); }\n</style>\n</head>\n<body>\n\n<header>\n  <div class=\"wrap\">\n    <a class=\"logo\" href=\"/\">Fieldnote</a>\n    <nav>\n      <a href=\"#how-it-works\">How it works</a>\n      <a href=\"#pricing\">Pricing</a>\n      <a class=\"btn btn-primary\" href=\"#get-the-app\">Get the app</a>\n    </nav>\n  </div>\n</header>\n\n<main>\n\n  <!-- Hero -->\n  <section class=\"hero\">\n    <div class=\"wrap\">\n      <h1>Finish the inspection report before you leave the property.</h1>\n      <p class=\"lede\">\n        Fieldnote turns your photos and checklist into a timestamped, GPS-tagged PDF report,\n        and your client signs it on your phone. Reports that used to take 90 minutes take about 20.\n      </p>\n      <div class=\"actions\" id=\"get-the-app\">\n        <!-- TODO: replace with real App Store link -->\n        <a class=\"btn btn-primary\" href=\"#\">Download on the App Store</a>\n        <a class=\"btn btn-secondary\" href=\"#how-it-works\">See how it works</a>\n      </div>\n      <p class=\"note muted\">\u00a329 per inspector per month. Built for inspection businesses with 1\u201310 inspectors.</p>\n    </div>\n  </section>\n\n  <!-- Proof -->\n  <section class=\"proof\" aria-label=\"Results\">\n    <div class=\"wrap stats\">\n      <div class=\"stat\">\n        <strong>90 \u2192 20 min</strong>\n        <span>typical time to produce a report</span>\n      </div>\n      <div class=\"stat\">\n        <strong>+1\u20132 inspections</strong>\n        <span>per inspector, per day</span>\n      </div>\n      <div class=\"stat\">\n        <strong>4.8 \u2605</strong>\n        <span>on the App Store, from 600 reviews</span>\n      </div>\n      <div class=\"stat\">\n        <strong>210 businesses</strong>\n        <span>run their inspections on Fieldnote, most of them in the UK</span>\n      </div>\n    </div>\n  </section>\n\n  <!-- How it works -->\n  <section id=\"how-it-works\">\n    <div class=\"wrap\">\n      <h2>The report writes itself while you walk the property</h2>\n      <p class=\"lede\">No typing up notes at your desk in the evening. You do the inspection and the report comes out of it.</p>\n      <div class=\"steps\">\n        <div class=\"step\">\n          <h3>Photograph as you go</h3>\n          <p class=\"muted\">Work through a photo-first checklist room by room. Each photo is attached to the item it shows, so you never have to match them up later.</p>\n        </div>\n        <div class=\"step\">\n          <h3>Get the PDF straight away</h3>\n          <p class=\"muted\">Fieldnote builds a clean, professional PDF report automatically. Every photo is stamped with the time and GPS location, so you can show when and where it was taken.</p>\n        </div>\n        <div class=\"step\">\n          <h3>Client signs on site</h3>\n          <p class=\"muted\">Hand over your phone for an e-signature before you leave. Nobody has to chase a signature by email, and the report is agreed while everyone is still in the room.</p>\n        </div>\n      </div>\n    </div>\n  </section>\n\n  <!-- Switching -->\n  <section>\n    <div class=\"wrap\">\n      <h2>Still using Word templates or a generic form app?</h2>\n      <p class=\"lede\">Most Fieldnote customers did too. This is what changes when they switch.</p>\n      <div class=\"compare\">\n        <div class=\"col\">\n          <h3>Before</h3>\n          <ul class=\"muted\">\n            <li>Photos on the camera roll, notes on paper, the report put together later</li>\n            <li>Copying, pasting and resizing images into a Word template</li>\n            <li>About 90 minutes of admin for every inspection</li>\n            <li>Signatures chased by email after the visit</li>\n          </ul>\n        </div>\n        <div class=\"col after\">\n          <h3>With Fieldnote</h3>\n          <ul>\n            <li>Photos captured inside the checklist, already in the right place</li>\n            <li>PDF report generated for you, with a time and GPS stamp on every photo</li>\n            <li>About 20 minutes per report</li>\n            <li>Client e-signature taken on site</li>\n          </ul>\n        </div>\n      </div>\n    </div>\n  </section>\n\n  <!-- Who it's for -->\n  <section>\n    <div class=\"wrap\">\n      <h2>Made for the inspections you actually do</h2>\n      <div class=\"uses\">\n        <div>\n          <h3>Rental move-in and move-out</h3>\n          <p class=\"muted\">Record the condition of the property with photos that carry a time and location stamp. When the tenancy ends, there's a clear record to compare against.</p>\n        </div>\n        <div>\n          <h3>Insurance inspections</h3>\n          <p class=\"muted\">Produce consistent reports with timestamped, GPS-tagged photos that show exactly what you saw and where.</p>\n        </div>\n        <div>\n          <h3>Small inspection businesses</h3>\n          <p class=\"muted\">Built for teams of 1\u201310 inspectors who need professional reports without a back office to write them.</p>\n        </div>\n      </div>\n    </div>\n  </section>\n\n  <!-- Pricing -->\n  <section class=\"pricing\" id=\"pricing\">\n    <div class=\"wrap\">\n      <h2>One simple price</h2>\n      <div class=\"price-card\">\n        <div class=\"price\">\u00a329 <small>per inspector / month</small></div>\n        <p class=\"muted\">Pay only for the inspectors who use it.</p>\n        <ul>\n          <li>Photo-first inspection checklists</li>\n          <li>Automatic PDF reports</li>\n          <li>Time and GPS stamps on every photo</li>\n          <li>Client e-signature on site</li>\n        </ul>\n        <div class=\"actions\">\n          <!-- TODO: replace with real App Store link -->\n          <a class=\"btn btn-primary\" href=\"#\">Download on the App Store</a>\n        </div>\n      </div>\n    </div>\n  </section>\n\n  <!-- Final CTA -->\n  <section class=\"final\">\n    <div class=\"wrap\">\n      <h2>Get your evenings back from report writing.</h2>\n      <p class=\"lede\" style=\"margin: 0 auto;\">Join the 210 inspection businesses that finish their reports on site.</p>\n      <div class=\"actions\">\n        <!-- TODO: replace with real App Store link -->\n        <a class=\"btn btn-primary\" href=\"#\">Download on the App Store</a>\n      </div>\n    </div>\n  </section>\n\n</main>\n\n<footer>\n  <div class=\"wrap\">\n    \u00a9 Fieldnote \u00b7 <a href=\"#\">Contact</a> \u00b7 <a href=\"#\">Privacy</a> \u00b7 <a href=\"#\">Terms</a>\n  </div>\n</footer>\n\n</body>\n</html>\n"}]}, "opus-2": {"cost_usd": 0.21155, "seconds": 66, "output_tokens": 6167, "files": [{"name": "index.html", "text": "<!DOCTYPE html>\n<html lang=\"en-GB\">\n<head>\n  <meta charset=\"utf-8\">\n  <meta name=\"viewport\" content=\"width=device-width, initial-scale=1\">\n  <title>Fieldnote: inspection reports finished on site</title>\n  <meta name=\"description\" content=\"Fieldnote is the mobile app for property inspectors. Photo-first checklists become timestamped, GPS-tagged PDF reports your client signs on site. Reports in about 20 minutes, not 90.\">\n  <style>\n    :root {\n      --ink: #1c2321;\n      --muted: #5a6662;\n      --line: #e3e7e5;\n      --bg: #ffffff;\n      --soft: #f5f7f6;\n      --accent: #1f6f50;\n      --accent-dark: #17563e;\n      --max: 1080px;\n    }\n    * { box-sizing: border-box; }\n    body {\n      margin: 0;\n      font-family: -apple-system, BlinkMacSystemFont, \"Segoe UI\", Roboto, \"Helvetica Neue\", Arial, sans-serif;\n      color: var(--ink);\n      background: var(--bg);\n      line-height: 1.55;\n      font-size: 17px;\n    }\n    a { color: var(--accent); }\n    .wrap { max-width: var(--max); margin: 0 auto; padding: 0 24px; }\n    section { padding: 72px 0; }\n    h1, h2, h3 { line-height: 1.15; margin: 0 0 16px; letter-spacing: -0.01em; }\n    h1 { font-size: clamp(2.2rem, 5vw, 3.4rem); max-width: 16ch; }\n    h2 { font-size: clamp(1.6rem, 3vw, 2.2rem); }\n    h3 { font-size: 1.15rem; }\n    p { margin: 0 0 16px; }\n    .lede { font-size: 1.2rem; color: var(--muted); max-width: 40em; }\n\n    .btn {\n      display: inline-block;\n      padding: 14px 22px;\n      border-radius: 8px;\n      font-weight: 600;\n      text-decoration: none;\n      border: 2px solid var(--accent);\n    }\n    .btn-primary { background: var(--accent); color: #fff; }\n    .btn-primary:hover { background: var(--accent-dark); border-color: var(--accent-dark); }\n    .btn-secondary { color: var(--accent); background: transparent; }\n    .ctas { display: flex; gap: 12px; flex-wrap: wrap; margin-top: 28px; }\n\n    /* Header */\n    header { border-bottom: 1px solid var(--line); }\n    header .wrap { display: flex; align-items: center; justify-content: space-between; height: 68px; }\n    .logo { font-weight: 700; font-size: 1.25rem; color: var(--ink); text-decoration: none; }\n    nav a { margin-left: 24px; color: var(--ink); text-decoration: none; font-size: 0.95rem; }\n    nav a.btn { margin-left: 24px; padding: 9px 16px; color: #fff; }\n    @media (max-width: 720px) { nav a:not(.btn) { display: none; } }\n\n    /* Hero */\n    .hero { padding-top: 88px; }\n    .hero-small { margin-top: 16px; font-size: 0.95rem; color: var(--muted); }\n\n    /* Proof strip */\n    .proof { background: var(--soft); border-top: 1px solid var(--line); border-bottom: 1px solid var(--line); padding: 40px 0; }\n    .proof ul { list-style: none; margin: 0; padding: 0; display: grid; grid-template-columns: repeat(auto-fit, minmax(200px, 1fr)); gap: 24px; }\n    .proof strong { display: block; font-size: 1.9rem; line-height: 1.1; }\n    .proof span { color: var(--muted); font-size: 0.95rem; }\n\n    /* Steps */\n    .steps { display: grid; grid-template-columns: repeat(auto-fit, minmax(260px, 1fr)); gap: 32px; margin-top: 40px; counter-reset: step; }\n    .step { border-top: 3px solid var(--accent); padding-top: 20px; }\n    .step::before { counter-increment: step; content: \"Step \" counter(step); display: block; font-size: 0.85rem; font-weight: 600; color: var(--accent); text-transform: uppercase; letter-spacing: 0.06em; margin-bottom: 8px; }\n    .step p { color: var(--muted); }\n\n    /* Comparison */\n    .compare { display: grid; grid-template-columns: 1fr 1fr; gap: 24px; margin-top: 32px; }\n    @media (max-width: 720px) { .compare { grid-template-columns: 1fr; } }\n    .compare div { border: 1px solid var(--line); border-radius: 12px; padding: 28px; }\n    .compare .after { border-color: var(--accent); background: #f2f8f5; }\n    .compare ul { margin: 0; padding-left: 20px; }\n    .compare li { margin-bottom: 8px; }\n\n    /* Use cases */\n    .uses { display: grid; grid-template-columns: repeat(auto-fit, minmax(280px, 1fr)); gap: 32px; margin-top: 32px; }\n    .uses p { color: var(--muted); }\n\n    /* Pricing */\n    .pricing { background: var(--soft); }\n    .price-card { background: #fff; border: 1px solid var(--line); border-radius: 12px; padding: 36px; max-width: 520px; margin-top: 32px; }\n    .price { font-size: 3rem; font-weight: 700; line-height: 1; }\n    .price small { font-size: 1rem; font-weight: 400; color: var(--muted); }\n    .price-card ul { padding-left: 20px; margin: 24px 0; }\n    .price-card li { margin-bottom: 8px; }\n\n    /* Final CTA */\n    .final { text-align: center; }\n    .final h2 { max-width: 22ch; margin-left: auto; margin-right: auto; }\n    .final .ctas { justify-content: center; }\n\n    footer { border-top: 1px solid var(--line); padding: 32px 0; color: var(--muted); font-size: 0.9rem; }\n  </style>\n</head>\n<body>\n\n<header>\n  <div class=\"wrap\">\n    <a class=\"logo\" href=\"/\">Fieldnote</a>\n    <nav>\n      <a href=\"#how-it-works\">How it works</a>\n      <a href=\"#pricing\">Pricing</a>\n      <a href=\"/sample-report.pdf\">Sample report</a>\n      <a class=\"btn btn-primary\" href=\"/signup\">Get started</a>\n    </nav>\n  </div>\n</header>\n\n<main>\n\n  <!-- Hero -->\n  <section class=\"hero\">\n    <div class=\"wrap\">\n      <h1>Finish the inspection report before you leave the property.</h1>\n      <p class=\"lede\">\n        Fieldnote is the inspection app for property inspectors. Work through a photo-first checklist on your phone, and Fieldnote builds the PDF report as you go, with every photo timestamped and GPS-tagged. Your client signs on screen before you leave.\n      </p>\n      <div class=\"ctas\">\n        <a class=\"btn btn-primary\" href=\"/signup\">Get started</a>\n        <a class=\"btn btn-secondary\" href=\"/sample-report.pdf\">See a sample report</a>\n      </div>\n      <p class=\"hero-small\">\u00a329 per inspector per month. Built for move-in, move-out and insurance inspections.</p>\n    </div>\n  </section>\n\n  <!-- Proof -->\n  <section class=\"proof\" aria-label=\"Results\">\n    <div class=\"wrap\">\n      <ul>\n        <li><strong>90 \u2192 20 min</strong><span>typical time to produce a report</span></li>\n        <li><strong>+1 to 2</strong><span>more inspections per inspector, per day</span></li>\n        <li><strong>4.8 \u2605</strong><span>on the App Store, from 600 reviews</span></li>\n        <li><strong>210</strong><span>inspection businesses, mostly in the UK</span></li>\n      </ul>\n    </div>\n  </section>\n\n  <!-- How it works -->\n  <section id=\"how-it-works\">\n    <div class=\"wrap\">\n      <h2>The report writes itself while you inspect</h2>\n      <p class=\"lede\">No more typing up notes at the kitchen table at night. When the walk-through is done, so is the report.</p>\n      <div class=\"steps\">\n        <div class=\"step\">\n          <h3>Photograph as you go</h3>\n          <p>Follow your checklist room by room. Take a photo, add a condition note, and move on. Each photo is stamped with the time and GPS location automatically.</p>\n        </div>\n        <div class=\"step\">\n          <h3>Get the PDF straight away</h3>\n          <p>Fieldnote lays out your photos and notes as a clean, consistent PDF report. There's nothing to format and nothing to paste into Word.</p>\n        </div>\n        <div class=\"step\">\n          <h3>Get it signed on site</h3>\n          <p>Your client, tenant or landlord signs on your phone. The signed report is ready to send before you get back in the car.</p>\n        </div>\n      </div>\n    </div>\n  </section>\n\n  <!-- Before / after -->\n  <section>\n    <div class=\"wrap\">\n      <h2>If you're still using Word templates, here's what changes</h2>\n      <div class=\"compare\">\n        <div>\n          <h3>Word templates and generic form apps</h3>\n          <ul>\n            <li>Photos copied off your phone and pasted in one by one</li>\n            <li>Around 90 minutes of write-up per report</li>\n            <li>Dates and locations you have to add yourself</li>\n            <li>Signatures chased by email afterwards</li>\n          </ul>\n        </div>\n        <div class=\"after\">\n          <h3>Fieldnote</h3>\n          <ul>\n            <li>Photos go straight into the right section of the report</li>\n            <li>About 20 minutes from first photo to finished PDF</li>\n            <li>Every photo timestamped and GPS-tagged automatically</li>\n            <li>Client signs on site, report sent the same day</li>\n          </ul>\n        </div>\n      </div>\n    </div>\n  </section>\n\n  <!-- Use cases -->\n  <section>\n    <div class=\"wrap\">\n      <h2>Made for the inspections you do every day</h2>\n      <div class=\"uses\">\n        <div>\n          <h3>Rental move-in and move-out</h3>\n          <p>Record the property's condition at check-in and check-out with dated, located photo evidence. When there's a deposit dispute, the report speaks for itself.</p>\n        </div>\n        <div>\n          <h3>Insurance inspections</h3>\n          <p>Give insurers a consistent report with photos that show exactly when and where they were taken, signed by the property owner at the visit.</p>\n        </div>\n        <div>\n          <h3>Small inspection teams</h3>\n          <p>Fieldnote is used by businesses with 1 to 10 inspectors. Every inspector produces the same clear, consistent report, so your firm looks professional on every job.</p>\n        </div>\n      </div>\n    </div>\n  </section>\n\n  <!-- Pricing -->\n  <section class=\"pricing\" id=\"pricing\">\n    <div class=\"wrap\">\n      <h2>One simple price</h2>\n      <p class=\"lede\">You pay for the inspectors on your team, and nothing else.</p>\n      <div class=\"price-card\">\n        <div class=\"price\">\u00a329 <small>per inspector, per month</small></div>\n        <ul>\n          <li>Photo-first inspection checklists</li>\n          <li>Automatic PDF reports</li>\n          <li>Timestamped, GPS-tagged photos</li>\n          <li>Client e-signature on site</li>\n        </ul>\n        <a class=\"btn btn-primary\" href=\"/signup\">Get started</a>\n      </div>\n    </div>\n  </section>\n\n  <!-- Final CTA -->\n  <section class=\"final\">\n    <div class=\"wrap\">\n      <h2>Spend your evenings on something other than reports.</h2>\n      <p class=\"lede\" style=\"margin: 0 auto;\">Join 210 inspection businesses that finish their reports on site with Fieldnote.</p>\n      <div class=\"ctas\">\n        <a class=\"btn btn-primary\" href=\"/signup\">Get started</a>\n        <a class=\"btn btn-secondary\" href=\"/sample-report.pdf\">See a sample report</a>\n      </div>\n    </div>\n  </section>\n\n</main>\n\n<footer>\n  <div class=\"wrap\">\n    \u00a9 Fieldnote. Inspection reports for property inspectors.\n  </div>\n</footer>\n\n</body>\n</html>\n"}]}, "opus-3": {"cost_usd": 0.24247, "seconds": 74, "output_tokens": 7360, "files": [{"name": "index.html", "text": "<!DOCTYPE html>\n<html lang=\"en-GB\">\n<head>\n<meta charset=\"utf-8\">\n<meta name=\"viewport\" content=\"width=device-width, initial-scale=1\">\n<title>Fieldnote: inspection reports finished on site, in 20 minutes</title>\n<meta name=\"description\" content=\"Fieldnote is the mobile app for property inspectors. Photo-first checklists, automatic PDF reports with timestamps and GPS, and client e-signature. \u00a329 per inspector per month.\">\n<style>\n  :root {\n    --ink: #1c2321;\n    --muted: #5a6662;\n    --line: #dde3e0;\n    --bg: #fbfaf7;\n    --panel: #ffffff;\n    --accent: #1f6f5c;\n    --accent-dark: #17594a;\n    --accent-soft: #e6f1ed;\n    --max: 1080px;\n  }\n  * { box-sizing: border-box; }\n  html { -webkit-text-size-adjust: 100%; }\n  body {\n    margin: 0;\n    font-family: -apple-system, BlinkMacSystemFont, \"Segoe UI\", Roboto, \"Helvetica Neue\", Arial, sans-serif;\n    color: var(--ink);\n    background: var(--bg);\n    line-height: 1.55;\n    font-size: 17px;\n  }\n  a { color: var(--accent); }\n  .wrap { max-width: var(--max); margin: 0 auto; padding: 0 24px; }\n\n  /* Header */\n  header.site {\n    border-bottom: 1px solid var(--line);\n    background: var(--panel);\n  }\n  header.site .wrap {\n    display: flex; align-items: center; justify-content: space-between;\n    height: 64px;\n  }\n  .logo { font-weight: 700; font-size: 20px; color: var(--ink); text-decoration: none; letter-spacing: -0.01em; }\n  nav a { color: var(--muted); text-decoration: none; margin-left: 24px; font-size: 15px; }\n  nav a:hover { color: var(--ink); }\n  nav a.btn { color: #fff; }\n\n  /* Buttons */\n  .btn {\n    display: inline-block; padding: 12px 20px; border-radius: 8px;\n    background: var(--accent); color: #fff; text-decoration: none; font-weight: 600;\n    border: 2px solid var(--accent);\n  }\n  .btn:hover { background: var(--accent-dark); border-color: var(--accent-dark); }\n  .btn.ghost { background: transparent; color: var(--accent); }\n  .btn.ghost:hover { background: var(--accent-soft); }\n  nav .btn { padding: 8px 14px; }\n\n  /* Hero */\n  .hero { padding: 80px 0 64px; }\n  .hero .wrap { display: grid; grid-template-columns: 1.15fr 0.85fr; gap: 56px; align-items: center; }\n  .eyebrow { color: var(--accent); font-weight: 600; font-size: 14px; letter-spacing: 0.04em; text-transform: uppercase; margin: 0 0 16px; }\n  h1 { font-size: 48px; line-height: 1.1; letter-spacing: -0.02em; margin: 0 0 20px; }\n  .lede { font-size: 20px; color: var(--muted); margin: 0 0 32px; max-width: 34em; }\n  .cta-row { display: flex; gap: 12px; flex-wrap: wrap; align-items: center; }\n  .cta-note { color: var(--muted); font-size: 14px; margin-top: 16px; }\n\n  /* Before/after card in hero */\n  .compare {\n    background: var(--panel); border: 1px solid var(--line); border-radius: 14px; padding: 28px;\n  }\n  .compare h2 { font-size: 15px; color: var(--muted); font-weight: 600; margin: 0 0 20px; }\n  .bar-row { margin-bottom: 20px; }\n  .bar-label { display: flex; justify-content: space-between; font-size: 15px; margin-bottom: 6px; }\n  .bar-label strong { font-variant-numeric: tabular-nums; }\n  .bar { height: 14px; border-radius: 7px; background: #c9d1ce; }\n  .bar.fn { background: var(--accent); width: 22.2%; }\n  .compare p { margin: 8px 0 0; font-size: 14px; color: var(--muted); }\n\n  /* Proof strip */\n  .proof { background: var(--panel); border-top: 1px solid var(--line); border-bottom: 1px solid var(--line); }\n  .proof .wrap { display: grid; grid-template-columns: repeat(3, 1fr); }\n  .stat { padding: 32px 24px; text-align: center; }\n  .stat + .stat { border-left: 1px solid var(--line); }\n  .stat .num { font-size: 34px; font-weight: 700; letter-spacing: -0.02em; font-variant-numeric: tabular-nums; }\n  .stat .what { color: var(--muted); font-size: 15px; }\n\n  /* Sections */\n  section.block { padding: 88px 0; }\n  .section-head { max-width: 40em; margin-bottom: 48px; }\n  h2.title { font-size: 34px; line-height: 1.2; letter-spacing: -0.015em; margin: 0 0 12px; }\n  .section-head p { color: var(--muted); font-size: 18px; margin: 0; }\n\n  /* Steps */\n  .steps { display: grid; grid-template-columns: repeat(3, 1fr); gap: 24px; counter-reset: step; list-style: none; padding: 0; margin: 0; }\n  .steps li { background: var(--panel); border: 1px solid var(--line); border-radius: 12px; padding: 28px; }\n  .steps li::before {\n    counter-increment: step; content: counter(step);\n    display: inline-flex; width: 32px; height: 32px; border-radius: 50%;\n    align-items: center; justify-content: center;\n    background: var(--accent-soft); color: var(--accent); font-weight: 700; margin-bottom: 16px;\n  }\n  .steps h3 { margin: 0 0 8px; font-size: 20px; }\n  .steps p { margin: 0; color: var(--muted); }\n\n  /* Switch section */\n  .switch { background: var(--accent-soft); }\n  .switch .wrap { display: grid; grid-template-columns: 1fr 1fr; gap: 56px; align-items: start; }\n  .switch ul { margin: 0; padding: 0; list-style: none; }\n  .switch li { padding: 16px 0; border-bottom: 1px solid rgba(31,111,92,0.18); }\n  .switch li:last-child { border-bottom: 0; }\n  .switch li strong { display: block; margin-bottom: 2px; }\n  .switch li span { color: var(--muted); }\n\n  /* Uses */\n  .uses { display: grid; grid-template-columns: repeat(3, 1fr); gap: 24px; }\n  .use { border-top: 3px solid var(--accent); padding-top: 18px; }\n  .use h3 { margin: 0 0 8px; font-size: 19px; }\n  .use p { margin: 0; color: var(--muted); }\n\n  /* Pricing */\n  .pricing .wrap { display: grid; grid-template-columns: 1fr 1fr; gap: 56px; align-items: center; }\n  .price-card { background: var(--panel); border: 1px solid var(--line); border-radius: 14px; padding: 36px; }\n  .price { font-size: 52px; font-weight: 700; letter-spacing: -0.02em; line-height: 1; }\n  .price small { font-size: 17px; font-weight: 500; color: var(--muted); letter-spacing: 0; }\n  .price-card ul { padding-left: 20px; margin: 24px 0 28px; }\n  .price-card li { margin-bottom: 6px; }\n  .examples { font-size: 15px; color: var(--muted); border-top: 1px solid var(--line); padding-top: 16px; margin-top: 4px; }\n  .examples table { width: 100%; border-collapse: collapse; font-variant-numeric: tabular-nums; }\n  .examples td { padding: 4px 0; }\n  .examples td:last-child { text-align: right; color: var(--ink); }\n\n  /* Final CTA */\n  .final { text-align: center; padding: 96px 0; background: var(--ink); color: #fff; }\n  .final h2 { font-size: 36px; margin: 0 0 12px; letter-spacing: -0.015em; }\n  .final p { color: #c3ccc9; font-size: 18px; margin: 0 0 32px; }\n  .final .btn { background: #fff; color: var(--ink); border-color: #fff; }\n  .final .btn:hover { background: #e9eeec; border-color: #e9eeec; }\n\n  footer.site { padding: 32px 0; color: var(--muted); font-size: 14px; }\n  footer.site .wrap { display: flex; justify-content: space-between; flex-wrap: wrap; gap: 12px; }\n\n  @media (max-width: 860px) {\n    body { font-size: 16px; }\n    nav a:not(.btn) { display: none; }\n    .hero { padding: 48px 0; }\n    .hero .wrap, .switch .wrap, .pricing .wrap { grid-template-columns: 1fr; gap: 36px; }\n    h1 { font-size: 36px; }\n    .lede { font-size: 18px; }\n    .proof .wrap, .steps, .uses { grid-template-columns: 1fr; }\n    .stat + .stat { border-left: 0; border-top: 1px solid var(--line); }\n    section.block { padding: 64px 0; }\n    h2.title { font-size: 28px; }\n  }\n</style>\n</head>\n<body>\n\n<header class=\"site\">\n  <div class=\"wrap\">\n    <a class=\"logo\" href=\"/\">Fieldnote</a>\n    <nav>\n      <a href=\"#how-it-works\">How it works</a>\n      <a href=\"#switching\">Switching</a>\n      <a href=\"#pricing\">Pricing</a>\n      <a class=\"btn\" href=\"#start\">Get started</a>\n    </nav>\n  </div>\n</header>\n\n<main>\n\n  <!-- Hero -->\n  <section class=\"hero\">\n    <div class=\"wrap\">\n      <div>\n        <p class=\"eyebrow\">For property inspectors</p>\n        <h1>Finish the report before you leave the property.</h1>\n        <p class=\"lede\">Fieldnote turns your photos and checklist into a signed PDF report, with timestamps and GPS on every photo. Reports that took 90 minutes at the desk now take about 20.</p>\n        <div class=\"cta-row\">\n          <a class=\"btn\" href=\"#start\">Get started</a>\n          <a class=\"btn ghost\" href=\"#how-it-works\">See how it works</a>\n        </div>\n        <p class=\"cta-note\">\u00a329 per inspector per month. Used by 210 inspection businesses.</p>\n      </div>\n\n      <div class=\"compare\" aria-label=\"Report time comparison\">\n        <h2>Time to finish one inspection report</h2>\n        <div class=\"bar-row\">\n          <div class=\"bar-label\"><span>Word template</span><strong>~90 min</strong></div>\n          <div class=\"bar\"></div>\n        </div>\n        <div class=\"bar-row\">\n          <div class=\"bar-label\"><span>Fieldnote</span><strong>~20 min</strong></div>\n          <div class=\"bar fn\"></div>\n        </div>\n        <p>About 70 minutes back on every job. Inspectors use that time to fit in 1\u20132 more inspections a day.</p>\n      </div>\n    </div>\n  </section>\n\n  <!-- Proof -->\n  <section class=\"proof\" aria-label=\"Results\">\n    <div class=\"wrap\">\n      <div class=\"stat\"><div class=\"num\">90 \u2192 20 min</div><div class=\"what\">typical time per report</div></div>\n      <div class=\"stat\"><div class=\"num\">+1\u20132</div><div class=\"what\">more inspections per inspector, per day</div></div>\n      <div class=\"stat\"><div class=\"num\">4.8 \u2605</div><div class=\"what\">on the App Store, from 600 reviews</div></div>\n    </div>\n  </section>\n\n  <!-- How it works -->\n  <section class=\"block\" id=\"how-it-works\">\n    <div class=\"wrap\">\n      <div class=\"section-head\">\n        <h2 class=\"title\">Inspect, sign, send. All from your phone.</h2>\n        <p>No notes to type up later. The report builds itself while you walk the property.</p>\n      </div>\n      <ol class=\"steps\">\n        <li>\n          <h3>Work through a photo-first checklist</h3>\n          <p>Go room by room. Take the photo, mark the condition, add a note if you need one. Each photo is stamped with the time and GPS location.</p>\n        </li>\n        <li>\n          <h3>Get the PDF report automatically</h3>\n          <p>Fieldnote lays out your photos, conditions and notes into a clean, consistent report. No copying pictures into Word, no reformatting.</p>\n        </li>\n        <li>\n          <h3>Have your client sign on the spot</h3>\n          <p>Your client e-signs on your phone before you leave. The signed report is done, and so is the job.</p>\n        </li>\n      </ol>\n    </div>\n  </section>\n\n  <!-- Switching -->\n  <section class=\"block switch\" id=\"switching\">\n    <div class=\"wrap\">\n      <div>\n        <h2 class=\"title\">Still using Word templates or a generic form app?</h2>\n        <p style=\"color: var(--muted); font-size: 18px;\">That\u2019s where most of our customers came from. Here\u2019s what changes when you switch.</p>\n      </div>\n      <ul>\n        <li>\n          <strong>No evening write-ups</strong>\n          <span>The report is finished on site, not typed up at your desk after the last job.</span>\n        </li>\n        <li>\n          <strong>Evidence that holds up</strong>\n          <span>Every photo carries a timestamp and GPS location, so there\u2019s no question about when and where it was taken.</span>\n        </li>\n        <li>\n          <strong>Built for inspections, not forms in general</strong>\n          <span>Checklists start with the photo, because in an inspection the photo is the record.</span>\n        </li>\n        <li>\n          <strong>Signed before you leave</strong>\n          <span>No chasing clients for signatures by email days later.</span>\n        </li>\n      </ul>\n    </div>\n  </section>\n\n  <!-- Use cases -->\n  <section class=\"block\">\n    <div class=\"wrap\">\n      <div class=\"section-head\">\n        <h2 class=\"title\">Made for the inspections you already do</h2>\n      </div>\n      <div class=\"uses\">\n        <div class=\"use\">\n          <h3>Rental move-in</h3>\n          <p>Record the property\u2019s condition at the start of a tenancy, with time-stamped photos of every room.</p>\n        </div>\n        <div class=\"use\">\n          <h3>Rental move-out</h3>\n          <p>Document the condition at check-out against the same checklist, ready for any deposit discussion.</p>\n        </div>\n        <div class=\"use\">\n          <h3>Insurance inspections</h3>\n          <p>Produce a clear photo report with location and time on every image, signed off before you leave.</p>\n        </div>\n      </div>\n    </div>\n  </section>\n\n  <!-- Pricing -->\n  <section class=\"block pricing\" id=\"pricing\" style=\"padding-top: 0;\">\n    <div class=\"wrap\">\n      <div>\n        <h2 class=\"title\">One simple price</h2>\n        <p style=\"color: var(--muted); font-size: 18px;\">Pay per inspector. Whether you work alone or run a team of ten, you only pay for the people out inspecting.</p>\n      </div>\n      <div class=\"price-card\">\n        <div class=\"price\">\u00a329 <small>per inspector / month</small></div>\n        <ul>\n          <li>Photo-first inspection checklists</li>\n          <li>Automatic PDF reports</li>\n          <li>Timestamp and GPS on every photo</li>\n          <li>Client e-signature</li>\n        </ul>\n        <div class=\"examples\">\n          <table>\n            <tr><td>1 inspector</td><td>\u00a329 / month</td></tr>\n            <tr><td>3 inspectors</td><td>\u00a387 / month</td></tr>\n            <tr><td>10 inspectors</td><td>\u00a3290 / month</td></tr>\n          </table>\n        </div>\n      </div>\n    </div>\n  </section>\n\n  <!-- Final CTA -->\n  <section class=\"final\" id=\"start\">\n    <div class=\"wrap\">\n      <h2>Get your evenings back.</h2>\n      <p>Join 210 inspection businesses that finish their reports on site.</p>\n      <a class=\"btn\" href=\"#\">Get started with Fieldnote</a>\n    </div>\n  </section>\n\n</main>\n\n<footer class=\"site\">\n  <div class=\"wrap\">\n    <span>\u00a9 Fieldnote</span>\n    <span>Inspection reports for property inspectors</span>\n  </div>\n</footer>\n\n</body>\n</html>\n"}]}, "haiku-1": {"cost_usd": 0.00881, "seconds": 48, "output_tokens": 9423, "files": [{"name": "index.html", "text": "<!DOCTYPE html>\n<html lang=\"en-GB\">\n<head>\n  <meta charset=\"utf-8\">\n  <meta name=\"viewport\" content=\"width=device-width, initial-scale=1\">\n  <title>Fieldnote | Inspection reports in 20 minutes</title>\n  <meta name=\"description\" content=\"Fieldnote is the mobile inspection app for rental move-ins, move-outs and insurance inspections. Photo-first checklists become a timestamped, GPS-stamped PDF report.\">\n  <style>\n    :root {\n      --ink: #1d2521;\n      --muted: #5b6660;\n      --paper: #fbfaf7;\n      --line: #e4e1d8;\n      --accent: #1f6f5c;\n      --accent-ink: #ffffff;\n      --accent-soft: #e6f0ec;\n    }\n    * { box-sizing: border-box; }\n    html { scroll-behavior: smooth; }\n    body {\n      margin: 0;\n      font-family: system-ui, -apple-system, \"Segoe UI\", Roboto, \"Helvetica Neue\", Arial, sans-serif;\n      color: var(--ink);\n      background: var(--paper);\n      line-height: 1.55;\n      -webkit-font-smoothing: antialiased;\n    }\n    img, svg { display: block; max-width: 100%; }\n    a { color: inherit; }\n    .wrap { width: min(1080px, calc(100% - 40px)); margin: 0 auto; }\n\n    .nav {\n      display: flex; align-items: center; justify-content: space-between;\n      padding: 20px 0;\n    }\n    .logo { font-weight: 700; font-size: 1.15rem; letter-spacing: -0.01em; text-decoration: none; }\n    .nav-links { display: flex; align-items: center; gap: 24px; }\n    .nav-links a { text-decoration: none; color: var(--muted); font-size: 0.95rem; }\n    .nav-links a:hover { color: var(--ink); }\n\n    .btn {\n      display: inline-block; padding: 14px 22px; border-radius: 10px;\n      font-weight: 600; font-size: 1rem; text-decoration: none; border: 1px solid transparent;\n      transition: background 0.15s ease, border-color 0.15s ease;\n    }\n    .btn-primary { background: var(--accent); color: var(--accent-ink); }\n    .btn-primary:hover { background: #185a4a; }\n    .btn-ghost { border-color: var(--line); color: var(--ink); background: #fff; }\n    .btn-ghost:hover { border-color: #c9c5b9; }\n    .btn-small { padding: 9px 16px; font-size: 0.9rem; }\n\n    .hero { padding: 64px 0 72px; }\n    .eyebrow {\n      display: inline-block; margin: 0 0 18px; padding: 6px 12px; border-radius: 999px;\n      background: var(--accent-soft); color: var(--accent); font-size: 0.85rem; font-weight: 600;\n    }\n    h1 {\n      font-size: clamp(2.3rem, 5.4vw, 3.8rem); line-height: 1.07; letter-spacing: -0.025em;\n      margin: 0 0 20px; max-width: 16ch;\n    }\n    .lede { font-size: clamp(1.05rem, 2vw, 1.25rem); color: var(--muted); max-width: 48ch; margin: 0 0 32px; }\n    .cta-row { display: flex; flex-wrap: wrap; gap: 12px; }\n\n    .stats {\n      display: grid; grid-template-columns: repeat(3, 1fr); gap: 0;\n      margin-top: 56px; border-top: 1px solid var(--line);\n    }\n    .stat { padding: 24px 24px 0 0; }\n    .stat + .stat { padding-left: 24px; border-left: 1px solid var(--line); }\n    .stat strong { display: block; font-size: clamp(1.8rem, 3.5vw, 2.4rem); letter-spacing: -0.02em; color: var(--accent); }\n    .stat span { color: var(--muted); font-size: 0.95rem; }\n\n    section { padding: 72px 0; border-top: 1px solid var(--line); }\n    .section-head { max-width: 640px; margin-bottom: 40px; }\n    h2 { font-size: clamp(1.7rem, 3.4vw, 2.4rem); line-height: 1.15; letter-spacing: -0.02em; margin: 0 0 14px; }\n    .section-head p { margin: 0; color: var(--muted); font-size: 1.05rem; }\n\n    .steps {\n      list-style: none; padding: 0; margin: 0;\n      display: grid; grid-template-columns: repeat(3, 1fr); gap: 20px;\n    }\n    .steps li {\n      background: #fff; border: 1px solid var(--line); border-radius: 14px; padding: 28px;\n    }\n    .step-num {\n      display: inline-flex; align-items: center; justify-content: center;\n      width: 32px; height: 32px; border-radius: 50%;\n      background: var(--ink); color: #fff; font-weight: 700; font-size: 0.9rem; margin-bottom: 18px;\n    }\n    .steps h3 { margin: 0 0 8px; font-size: 1.15rem; }\n    .steps p { margin: 0; color: var(--muted); }\n\n    .features { display: grid; gap: 56px; }\n    .feature {\n      display: grid; grid-template-columns: 1fr 1fr; gap: 56px; align-items: center;\n    }\n    .feature:nth-child(even) .feature-text { order: 2; }\n    .feature h3 { font-size: clamp(1.4rem, 2.6vw, 1.8rem); line-height: 1.2; letter-spacing: -0.015em; margin: 0 0 14px; }\n    .feature p { margin: 0 0 12px; color: var(--muted); font-size: 1.05rem; }\n    .feature ul { margin: 16px 0 0; padding: 0; list-style: none; color: var(--ink); font-size: 0.97rem; }\n    .feature li { padding: 8px 0; border-top: 1px solid var(--line); }\n    .feature li:last-child { border-bottom: 1px solid var(--line); }\n    .panel {\n      background: #fff; border: 1px solid var(--line); border-radius: 18px; padding: 28px;\n      min-height: 260px; display: flex; flex-direction: column; justify-content: center;\n    }\n    .panel-label { font-size: 0.8rem; font-weight: 600; text-transform: uppercase; letter-spacing: 0.06em; color: var(--muted); margin: 0 0 12px; }\n    .panel-value { font-size: 1.6rem; font-weight: 700; letter-spacing: -0.02em; margin: 0; }\n    .panel-meta { color: var(--muted); font-size: 0.92rem; margin: 6px 0 0; }\n\n    .proof { background: var(--ink); color: #f1f3ef; border-top: none; }\n    .proof .section-head p { color: #aab4ad; }\n    .proof-grid {\n      display: grid; grid-template-columns: repeat(3, 1fr); gap: 20px;\n    }\n    .proof-card { border: 1px solid #323d37; border-radius: 14px; padding: 28px; }\n    .proof-card strong { display: block; font-size: 2.4rem; letter-spacing: -0.02em; margin-bottom: 6px; }\n    .proof-card span { color: #aab4ad; font-size: 0.97rem; }\n\n    .pricing-card {\n      max-width: 520px; background: #fff; border: 1px solid var(--line); border-radius: 18px; padding: 36px;\n    }\n    .price { font-size: 3rem; font-weight: 700; letter-spacing: -0.03em; margin: 0; line-height: 1; }\n    .price small { font-size: 1rem; font-weight: 500; color: var(--muted); letter-spacing: 0; }\n    .pricing-card .lead { color: var(--muted); margin: 12px 0 24px; }\n    .pricing-card ul { list-style: none; padding: 0; margin: 0 0 28px; }\n    .pricing-card li { padding: 10px 0; border-top: 1px solid var(--line); }\n    .pricing-card .btn { width: 100%; text-align: center; }\n\n    .final { text-align: center; padding: 96px 0; }\n    .final h2 { margin-left: auto; margin-right: auto; max-width: 18ch; }\n    .final p { color: var(--muted); font-size: 1.1rem; margin: 0 auto 32px; max-width: 46ch; }\n\n    footer { border-top: 1px solid var(--line); padding: 28px 0; color: var(--muted); font-size: 0.9rem; }\n    .footer-row { display: flex; justify-content: space-between; flex-wrap: wrap; gap: 12px; }\n\n    @media (max-width: 860px) {\n      .steps, .proof-grid { grid-template-columns: 1fr; }\n      .feature { grid-template-columns: 1fr; gap: 28px; }\n      .feature:nth-child(even) .feature-text { order: 0; }\n      .stats { grid-template-columns: 1fr; }\n      .stat, .stat + .stat { padding: 20px 0; border-left: none; border-bottom: 1px solid var(--line); }\n      .stat:last-child { border-bottom: none; }\n    }\n    @media (max-width: 560px) {\n      .nav-links a:not(.btn) { display: none; }\n      .hero { padding-top: 40px; }\n    }\n  </style>\n</head>\n<body>\n\n  <header class=\"wrap nav\">\n    <a href=\"#\" class=\"logo\">Fieldnote</a>\n    <nav class=\"nav-links\" aria-label=\"Main\">\n      <a href=\"#how-it-works\">How it works</a>\n      <a href=\"#pricing\">Pricing</a>\n      <a href=\"#\" class=\"btn btn-primary btn-small\">Get the app</a>\n    </nav>\n  </header>\n\n  <main>\n    <div class=\"wrap hero\">\n      <span class=\"eyebrow\">For property inspectors in the UK</span>\n      <h1>Finish the report before you leave the property.</h1>\n      <p class=\"lede\">Fieldnote is the inspection app for rental move-ins, move-outs and insurance inspections. Photograph as you go, and get a timestamped, GPS-stamped PDF that your client can sign on the spot.</p>\n      <div class=\"cta-row\">\n        <a href=\"#\" class=\"btn btn-primary\">Get the app</a>\n        <a href=\"#pricing\" class=\"btn btn-ghost\">See pricing</a>\n      </div>\n\n      <div class=\"stats\">\n        <div class=\"stat\">\n          <strong>90 to 20 min</strong>\n          <span>Average time to write up a report</span>\n        </div>\n        <div class=\"stat\">\n          <strong>1 to 2 more</strong>\n          <span>Inspections per inspector, per day</span>\n        </div>\n        <div class=\"stat\">\n          <strong>4.8 stars</strong>\n          <span>On the App Store, from 600 reviews</span>\n        </div>\n      </div>\n    </div>\n\n    <section id=\"how-it-works\">\n      <div class=\"wrap\">\n        <div class=\"section-head\">\n          <h2>Built for how inspections actually happen.</h2>\n          <p>You're on your feet, going room to room. Fieldnote is designed around that, not around a desk.</p>\n        </div>\n        <ol class=\"steps\">\n          <li>\n            <span class=\"step-num\">1</span>\n            <h3>Photograph as you walk</h3>\n            <p>Start from a checklist and take photos directly against each item. The photos are the record.</p>\n          </li>\n          <li>\n            <span class=\"step-num\">2</span>\n            <h3>Tick off the checklist</h3>\n            <p>Record condition, notes and issues while you're still in the room, so nothing gets remembered wrong later.</p>\n          </li>\n          <li>\n            <span class=\"step-num\">3</span>\n            <h3>Send the signed PDF</h3>\n            <p>The report generates itself. Your client signs on your phone, and the PDF goes out before you drive away.</p>\n          </li>\n        </ol>\n      </div>\n    </section>\n\n    <section id=\"features\">\n      <div class=\"wrap\">\n        <div class=\"section-head\">\n          <h2>Everything a report needs. Nothing it doesn't.</h2>\n          <p>Stop rebuilding Word templates and wrestling generic form apps into shape for property work.</p>\n        </div>\n        <div class=\"features\">\n          <div class=\"feature\">\n            <div class=\"feature-text\">\n              <h3>Photo-first checklists</h3>\n              <p>Checklists are built around photos, so evidence sits next to every line item instead of in a camera roll you sort through later.</p>\n            </div>\n            <div class=\"panel\" aria-hidden=\"true\">\n              <p class=\"panel-label\">Kitchen, sink</p>\n              <p class=\"panel-value\">Chipped near tap</p>\n              <p class=\"panel-meta\">3 photos attached</p>\n            </div>\n          </div>\n\n          <div class=\"feature\">\n            <div class=\"feature-text\">\n              <h3>Reports that write themselves</h3>\n              <p>Every photo is stamped with the time and GPS location. The PDF is assembled automatically from your checklist.</p>\n              <ul>\n                <li>Timestamps on every photo</li>\n                <li>GPS location on every photo</li>\n                <li>Auto-generated PDF, ready to share</li>\n              </ul>\n            </div>\n            <div class=\"panel\" aria-hidden=\"true\">\n              <p class=\"panel-label\">Report</p>\n              <p class=\"panel-value\">Move-out, 14 Elm Road</p>\n              <p class=\"panel-meta\">Generated automatically from checklist</p>\n            </div>\n          </div>\n\n          <div class=\"feature\">\n            <div class=\"feature-text\">\n              <h3>Client sign-off on site</h3>\n              <p>Clients sign the report on your phone while you're still there. No chasing emails, no printing and scanning.</p>\n            </div>\n            <div class=\"panel\" aria-hidden=\"true\">\n              <p class=\"panel-label\">Signed</p>\n              <p class=\"panel-value\">Client signature captured</p>\n              <p class=\"panel-meta\">Report sent before you leave</p>\n            </div>\n          </div>\n        </div>\n      </div>\n    </section>\n\n    <section class=\"proof\" id=\"proof\">\n      <div class=\"wrap\">\n        <div class=\"section-head\">\n          <h2>Inspection businesses are switching to Fieldnote.</h2>\n          <p>Teams moving off Word templates and generic form apps, from solo inspectors to teams of ten.</p>\n        </div>\n        <div class=\"proof-grid\">\n          <div class=\"proof-card\">\n            <strong>210</strong>\n            <span>Inspection businesses using Fieldnote</span>\n          </div>\n          <div class=\"proof-card\">\n            <strong>4.8</strong>\n            <span>Average App Store rating, from 600 reviews</span>\n          </div>\n          <div class=\"proof-card\">\n            <strong>20 min</strong>\n            <span>Typical report time, down from about 90</span>\n          </div>\n        </div>\n      </div>\n    </section>\n\n    <section id=\"pricing\">\n      <div class=\"wrap\">\n        <div class=\"section-head\">\n          <h2>One simple price.</h2>\n          <p>No tiers to compare. Pay for the inspectors who use it.</p>\n        </div>\n        <div class=\"pricing-card\">\n          <p class=\"price\">\u00a329 <small>per inspector, per month</small></p>\n          <p class=\"lead\">For inspection businesses with 1 to 10 inspectors.</p>\n          <ul>\n            <li>Photo-first checklists</li>\n            <li>Timestamped, GPS-stamped PDF reports</li>\n            <li>Client e-signature</li>\n          </ul>\n          <a href=\"#\" class=\"btn btn-primary\">Get the app</a>\n        </div>\n      </div>\n    </section>\n\n    <section class=\"final\">\n      <div class=\"wrap\">\n        <h2>Your next report can be finished before you leave.</h2>\n        <p>Move in, move out or inspect for insurance. Fieldnote gets the report done while the details are still fresh.</p>\n        <a href=\"#\" class=\"btn btn-primary\">Get the app</a>\n      </div>\n    </section>\n  </main>\n\n  <footer>\n    <div class=\"wrap footer-row\">\n      <span>&copy; Fieldnote</span>\n      <span>\n        <a href=\"#\">Privacy</a> &middot; <a href=\"#\">Terms</a> &middot; <a href=\"#\">Contact</a>\n      </span>\n    </div>\n  </footer>\n\n</body>\n</html>\n"}]}, "haiku-2": {"cost_usd": 0.00877, "seconds": 41, "output_tokens": 9961, "files": [{"name": "index.html", "text": "<!DOCTYPE html>\n<html lang=\"en-GB\">\n<head>\n  <meta charset=\"utf-8\">\n  <meta name=\"viewport\" content=\"width=device-width, initial-scale=1\">\n  <title>Fieldnote: property inspection app with photo-first checklists and PDF reports</title>\n  <meta name=\"description\" content=\"Fieldnote is a mobile app for property inspectors. Photo-first checklists, timestamped and GPS-tagged PDF reports, and client e-signature. \u00a329 per inspector per month.\">\n  <style>\n    :root {\n      --ink: #14213d;\n      --muted: #55607a;\n      --paper: #fbfaf7;\n      --card: #ffffff;\n      --line: #e4e1d8;\n      --accent: #1f7a5a;\n      --accent-ink: #ffffff;\n    }\n\n    * { box-sizing: border-box; }\n\n    body {\n      margin: 0;\n      font-family: -apple-system, BlinkMacSystemFont, \"Segoe UI\", Roboto, \"Helvetica Neue\", Arial, sans-serif;\n      color: var(--ink);\n      background: var(--paper);\n      line-height: 1.6;\n      -webkit-font-smoothing: antialiased;\n    }\n\n    img, svg { display: block; max-width: 100%; }\n    a { color: inherit; }\n\n    .wrap {\n      width: min(1080px, 100% - 40px);\n      margin-inline: auto;\n    }\n\n    header.site {\n      border-bottom: 1px solid var(--line);\n      background: var(--paper);\n    }\n\n    .site-inner {\n      display: flex;\n      align-items: center;\n      justify-content: space-between;\n      gap: 16px;\n      padding: 18px 0;\n    }\n\n    .logo {\n      font-weight: 700;\n      font-size: 1.15rem;\n      text-decoration: none;\n      letter-spacing: -0.01em;\n    }\n\n    nav.links {\n      display: flex;\n      align-items: center;\n      gap: 20px;\n      font-size: 0.95rem;\n    }\n\n    nav.links a {\n      text-decoration: none;\n      color: var(--muted);\n    }\n\n    nav.links a:hover { color: var(--ink); }\n\n    .btn {\n      display: inline-block;\n      padding: 12px 22px;\n      border-radius: 8px;\n      background: var(--accent);\n      color: var(--accent-ink);\n      font-weight: 600;\n      text-decoration: none;\n      font-size: 1rem;\n    }\n\n    .btn:hover { filter: brightness(0.94); }\n\n    .btn.secondary {\n      background: transparent;\n      color: var(--ink);\n      border: 1px solid var(--line);\n    }\n\n    .btn.small { padding: 9px 16px; font-size: 0.9rem; }\n\n    .hero {\n      padding: 72px 0 56px;\n    }\n\n    .eyebrow {\n      font-size: 0.85rem;\n      font-weight: 600;\n      letter-spacing: 0.08em;\n      text-transform: uppercase;\n      color: var(--accent);\n      margin: 0 0 14px;\n    }\n\n    h1 {\n      font-size: clamp(2.2rem, 5vw, 3.4rem);\n      line-height: 1.1;\n      letter-spacing: -0.02em;\n      margin: 0 0 20px;\n      max-width: 16ch;\n    }\n\n    .lead {\n      font-size: 1.15rem;\n      color: var(--muted);\n      max-width: 36em;\n      margin: 0 0 32px;\n    }\n\n    .cta-row {\n      display: flex;\n      flex-wrap: wrap;\n      gap: 12px;\n    }\n\n    .proof {\n      display: grid;\n      grid-template-columns: repeat(3, 1fr);\n      border-top: 1px solid var(--line);\n      border-bottom: 1px solid var(--line);\n    }\n\n    .proof > div {\n      padding: 28px 24px;\n      border-left: 1px solid var(--line);\n    }\n\n    .proof > div:first-child { border-left: 0; padding-left: 0; }\n\n    .proof .num {\n      font-size: 2rem;\n      font-weight: 700;\n      letter-spacing: -0.02em;\n      line-height: 1.1;\n      margin-bottom: 6px;\n    }\n\n    .proof .label {\n      font-size: 0.95rem;\n      color: var(--muted);\n      margin: 0;\n    }\n\n    section.block {\n      padding: 80px 0;\n    }\n\n    section.block.tinted {\n      background: var(--card);\n      border-top: 1px solid var(--line);\n      border-bottom: 1px solid var(--line);\n    }\n\n    h2 {\n      font-size: clamp(1.6rem, 3.2vw, 2.2rem);\n      line-height: 1.2;\n      letter-spacing: -0.015em;\n      margin: 0 0 16px;\n    }\n\n    .section-lead {\n      color: var(--muted);\n      max-width: 40em;\n      margin: 0 0 40px;\n      font-size: 1.05rem;\n    }\n\n    .steps {\n      list-style: none;\n      margin: 0;\n      padding: 0;\n      display: grid;\n      gap: 24px;\n      grid-template-columns: 1fr;\n      counter-reset: step;\n    }\n\n    .steps li {\n      border-top: 2px solid var(--ink);\n      padding-top: 18px;\n    }\n\n    .steps .step-no {\n      display: block;\n      font-size: 0.85rem;\n      font-weight: 600;\n      color: var(--accent);\n      letter-spacing: 0.06em;\n      text-transform: uppercase;\n      margin-bottom: 8px;\n    }\n\n    .steps h3 {\n      font-size: 1.2rem;\n      margin: 0 0 8px;\n    }\n\n    .steps p {\n      margin: 0;\n      color: var(--muted);\n    }\n\n    .uses {\n      display: grid;\n      gap: 20px;\n      grid-template-columns: 1fr;\n    }\n\n    .use {\n      background: var(--paper);\n      border: 1px solid var(--line);\n      border-radius: 12px;\n      padding: 28px;\n    }\n\n    .use h3 {\n      font-size: 1.2rem;\n      margin: 0 0 8px;\n    }\n\n    .use p {\n      margin: 0;\n      color: var(--muted);\n    }\n\n    .switch {\n      display: grid;\n      gap: 40px;\n      grid-template-columns: 1fr;\n      align-items: start;\n    }\n\n    .switch p {\n      color: var(--muted);\n      font-size: 1.05rem;\n      margin: 0 0 16px;\n    }\n\n    .price-card {\n      border: 1px solid var(--ink);\n      border-radius: 14px;\n      padding: 40px 36px;\n      background: var(--card);\n      max-width: 520px;\n    }\n\n    .price {\n      font-size: 3rem;\n      font-weight: 700;\n      letter-spacing: -0.02em;\n      line-height: 1;\n      margin: 0 0 8px;\n    }\n\n    .price small {\n      font-size: 1rem;\n      font-weight: 500;\n      color: var(--muted);\n      letter-spacing: 0;\n    }\n\n    .price-card p {\n      color: var(--muted);\n      margin: 0 0 24px;\n    }\n\n    .final {\n      padding: 88px 0;\n      text-align: center;\n    }\n\n    .final h2 { margin-inline: auto; max-width: 20em; }\n\n    .final p {\n      color: var(--muted);\n      margin: 0 auto 28px;\n      max-width: 34em;\n    }\n\n    footer.site {\n      border-top: 1px solid var(--line);\n      padding: 28px 0;\n      font-size: 0.9rem;\n      color: var(--muted);\n    }\n\n    .foot-inner {\n      display: flex;\n      flex-wrap: wrap;\n      justify-content: space-between;\n      gap: 12px;\n    }\n\n    footer.site a {\n      color: var(--muted);\n      margin-left: 18px;\n    }\n\n    @media (min-width: 720px) {\n      .steps { grid-template-columns: repeat(3, 1fr); gap: 32px; }\n      .uses { grid-template-columns: repeat(2, 1fr); }\n      .switch { grid-template-columns: 1fr 1fr; gap: 64px; }\n    }\n\n    @media (max-width: 719px) {\n      nav.links a.hide-mobile { display: none; }\n      .proof { grid-template-columns: 1fr; }\n      .proof > div {\n        border-left: 0;\n        padding-left: 0;\n        border-top: 1px solid var(--line);\n      }\n      .proof > div:first-child { border-top: 0; }\n      .hero { padding: 48px 0 40px; }\n      section.block { padding: 56px 0; }\n    }\n  </style>\n</head>\n<body>\n\n  <header class=\"site\">\n    <div class=\"wrap site-inner\">\n      <a class=\"logo\" href=\"#\">Fieldnote</a>\n      <nav class=\"links\" aria-label=\"Main\">\n        <a class=\"hide-mobile\" href=\"#how\">How it works</a>\n        <a class=\"hide-mobile\" href=\"#pricing\">Pricing</a>\n        <a class=\"btn small\" href=\"#get\">Get the app</a>\n      </nav>\n    </div>\n  </header>\n\n  <main>\n\n    <section class=\"hero\">\n      <div class=\"wrap\">\n        <p class=\"eyebrow\">For property inspectors</p>\n        <h1>Finish the report before you leave the property.</h1>\n        <p class=\"lead\">Fieldnote is the inspection app for small inspection businesses. Work through photo-first checklists on your phone, then send a timestamped PDF report that your client can sign on the spot.</p>\n        <div class=\"cta-row\">\n          <a class=\"btn\" id=\"get\" href=\"#\">Get Fieldnote</a>\n          <a class=\"btn secondary\" href=\"#pricing\">See pricing</a>\n        </div>\n      </div>\n    </section>\n\n    <div class=\"wrap\">\n      <div class=\"proof\" aria-label=\"Results from Fieldnote users\">\n        <div>\n          <div class=\"num\">90 \u2192 20 min</div>\n          <p class=\"label\">Time to write a report, down from about 90 minutes.</p>\n        </div>\n        <div>\n          <div class=\"num\">1\u20132 more</div>\n          <p class=\"label\">Inspections per inspector, per day.</p>\n        </div>\n        <div>\n          <div class=\"num\">4.8 \u2605</div>\n          <p class=\"label\">Rating on the App Store, from 600 reviews.</p>\n        </div>\n      </div>\n    </div>\n\n    <section class=\"block\" id=\"how\">\n      <div class=\"wrap\">\n        <h2>From the first room to the signed report.</h2>\n        <p class=\"section-lead\">Fieldnote follows the order of a real inspection, so the report is built as you go.</p>\n        <ol class=\"steps\">\n          <li>\n            <span class=\"step-no\">Step 1</span>\n            <h3>Work through the checklist</h3>\n            <p>Take photos room by room, inside the checklist you are already working through.</p>\n          </li>\n          <li>\n            <span class=\"step-no\">Step 2</span>\n            <h3>The report builds itself</h3>\n            <p>Every photo carries its timestamp and GPS location. The PDF is assembled from your checklist.</p>\n          </li>\n          <li>\n            <span class=\"step-no\">Step 3</span>\n            <h3>Your client signs on the spot</h3>\n            <p>Clients sign electronically, so there is no paperwork to chase afterwards.</p>\n          </li>\n        </ol>\n      </div>\n    </section>\n\n    <section class=\"block tinted\">\n      <div class=\"wrap\">\n        <h2>Built for the inspections you already do.</h2>\n        <p class=\"section-lead\">Fieldnote is designed around the two inspection types our customers do most.</p>\n        <div class=\"uses\">\n          <article class=\"use\">\n            <h3>Move-in and move-out</h3>\n            <p>Rental inspections with photo-first checklists, so the condition at check-in and check-out is recorded room by room.</p>\n          </article>\n          <article class=\"use\">\n            <h3>Insurance inspections</h3>\n            <p>Timestamped, GPS-tagged photos and a signed PDF report you can hand over with the file.</p>\n          </article>\n        </div>\n      </div>\n    </section>\n\n    <section class=\"block\">\n      <div class=\"wrap switch\">\n        <div>\n          <h2>Leave the Word template behind.</h2>\n          <p>Many inspectors start with a Word file, a folder of photos, and a lot of retyping at the end of the day.</p>\n          <p>Fieldnote keeps the checklist, the photos, and the report together, so the report is built from the inspection rather than typed up after it.</p>\n        </div>\n      </div>\n    </section>\n\n    <section class=\"block tinted\" id=\"pricing\">\n      <div class=\"wrap\">\n        <h2>One simple price.</h2>\n        <p class=\"section-lead\">Built for small inspection businesses with 1 to 10 inspectors.</p>\n        <div class=\"price-card\">\n          <p class=\"price\">\u00a329 <small>per inspector, per month</small></p>\n          <p>Checklists with photos, timestamped and GPS-tagged PDF reports, and client e-signature.</p>\n          <a class=\"btn\" href=\"#get\">Get Fieldnote</a>\n        </div>\n      </div>\n    </section>\n\n    <section class=\"final\">\n      <div class=\"wrap\">\n        <h2>See how your next inspection goes.</h2>\n        <p>Download Fieldnote and run your next move-in, move-out, or insurance inspection through it.</p>\n        <a class=\"btn\" href=\"#get\">Get Fieldnote</a>\n      </div>\n    </section>\n\n  </main>\n\n  <footer class=\"site\">\n    <div class=\"wrap foot-inner\">\n      <span>&copy; 2026 Fieldnote</span>\n      <span>\n        <a href=\"#\">Privacy</a>\n        <a href=\"#\">Terms</a>\n        <a href=\"#\">Contact</a>\n      </span>\n    </div>\n  </footer>\n\n</body>\n</html>\n"}]}, "haiku-3": {"cost_usd": 0.00827, "seconds": 36, "output_tokens": 9142, "files": [{"name": "index.html", "text": "<!DOCTYPE html>\n<html lang=\"en-GB\">\n<head>\n  <meta charset=\"utf-8\">\n  <meta name=\"viewport\" content=\"width=device-width, initial-scale=1\">\n  <title>Fieldnote | Property inspection app for UK inspectors</title>\n  <meta name=\"description\" content=\"Fieldnote is the mobile app for property inspectors. Photo-first checklists, timestamped PDF reports with GPS, and client e-signature. Reports in 20 minutes, not 90.\">\n  <style>\n    :root {\n      --ink: #1d2328;\n      --muted: #5b6670;\n      --paper: #ffffff;\n      --wash: #f5f6f4;\n      --line: #e2e5e1;\n      --accent: #1f6f5c;\n      --accent-ink: #ffffff;\n      --radius: 12px;\n    }\n\n    * { box-sizing: border-box; }\n\n    html { -webkit-text-size-adjust: 100%; }\n\n    body {\n      margin: 0;\n      font-family: -apple-system, BlinkMacSystemFont, \"Segoe UI\", Roboto, \"Helvetica Neue\", Arial, sans-serif;\n      color: var(--ink);\n      background: var(--paper);\n      line-height: 1.6;\n      font-size: 17px;\n    }\n\n    img, svg { max-width: 100%; display: block; }\n    a { color: inherit; }\n\n    .wrap {\n      width: min(1120px, 100% - 40px);\n      margin-inline: auto;\n    }\n\n    /* Nav */\n    .nav {\n      display: flex;\n      align-items: center;\n      justify-content: space-between;\n      gap: 16px;\n      padding: 20px 0;\n    }\n    .logo {\n      font-weight: 700;\n      font-size: 20px;\n      text-decoration: none;\n      letter-spacing: -0.01em;\n    }\n    .nav-links {\n      display: flex;\n      align-items: center;\n      gap: 28px;\n      list-style: none;\n      margin: 0;\n      padding: 0;\n    }\n    .nav-links a {\n      text-decoration: none;\n      color: var(--muted);\n      font-size: 15px;\n    }\n    .nav-links a:hover { color: var(--ink); }\n\n    /* Buttons */\n    .btn {\n      display: inline-block;\n      padding: 14px 22px;\n      border-radius: 999px;\n      font-weight: 600;\n      font-size: 16px;\n      text-decoration: none;\n      border: 2px solid var(--accent);\n      background: var(--accent);\n      color: var(--accent-ink);\n    }\n    .btn:hover { filter: brightness(1.06); }\n    .btn-ghost {\n      background: transparent;\n      color: var(--accent);\n    }\n    .btn-small { padding: 9px 16px; font-size: 14px; }\n\n    /* Hero */\n    .hero {\n      padding: 56px 0 72px;\n    }\n    .eyebrow {\n      display: inline-block;\n      margin: 0 0 18px;\n      font-size: 14px;\n      font-weight: 600;\n      letter-spacing: 0.04em;\n      text-transform: uppercase;\n      color: var(--accent);\n    }\n    h1 {\n      font-size: clamp(38px, 6vw, 64px);\n      line-height: 1.08;\n      letter-spacing: -0.025em;\n      margin: 0 0 22px;\n      max-width: 16ch;\n    }\n    .lede {\n      font-size: clamp(18px, 2vw, 21px);\n      color: var(--muted);\n      max-width: 52ch;\n      margin: 0 0 32px;\n    }\n    .cta-row {\n      display: flex;\n      flex-wrap: wrap;\n      gap: 12px;\n      align-items: center;\n    }\n    .proof-line {\n      margin: 22px 0 0;\n      font-size: 15px;\n      color: var(--muted);\n    }\n\n    /* Stats */\n    .stats {\n      background: var(--wash);\n      border-block: 1px solid var(--line);\n      padding: 40px 0;\n    }\n    .stats-grid {\n      display: grid;\n      grid-template-columns: repeat(4, 1fr);\n      gap: 24px;\n    }\n    .stat-value {\n      font-size: clamp(32px, 4vw, 44px);\n      font-weight: 700;\n      letter-spacing: -0.02em;\n      line-height: 1.1;\n      margin: 0 0 6px;\n    }\n    .stat-label {\n      margin: 0;\n      font-size: 15px;\n      color: var(--muted);\n    }\n\n    /* Sections */\n    section.block {\n      padding: 88px 0;\n    }\n    .section-head {\n      max-width: 640px;\n      margin-bottom: 48px;\n    }\n    h2 {\n      font-size: clamp(30px, 4vw, 42px);\n      line-height: 1.15;\n      letter-spacing: -0.02em;\n      margin: 0 0 14px;\n    }\n    .section-head p {\n      margin: 0;\n      color: var(--muted);\n      font-size: 18px;\n    }\n\n    /* Switch from Word / form apps */\n    .switch-grid {\n      display: grid;\n      grid-template-columns: repeat(3, 1fr);\n      gap: 24px;\n    }\n    .card {\n      border: 1px solid var(--line);\n      border-radius: var(--radius);\n      padding: 28px;\n      background: var(--paper);\n    }\n    .card h3 {\n      font-size: 19px;\n      margin: 0 0 10px;\n      letter-spacing: -0.01em;\n    }\n    .card p {\n      margin: 0;\n      color: var(--muted);\n      font-size: 16px;\n    }\n    .before-after {\n      margin-top: 36px;\n      padding: 24px 28px;\n      border-left: 4px solid var(--accent);\n      background: var(--wash);\n      border-radius: 0 var(--radius) var(--radius) 0;\n      color: var(--ink);\n    }\n    .before-after p { margin: 0; }\n\n    /* Features */\n    .features {\n      display: grid;\n      grid-template-columns: repeat(3, 1fr);\n      gap: 24px;\n    }\n    .feature-index {\n      display: inline-block;\n      font-size: 14px;\n      font-weight: 600;\n      color: var(--accent);\n      margin-bottom: 14px;\n    }\n\n    /* Use cases */\n    .use-cases {\n      display: grid;\n      grid-template-columns: 1fr 1fr;\n      gap: 24px;\n    }\n    .use-case h3 { font-size: 22px; margin: 0 0 10px; }\n    .use-case p { margin: 0; color: var(--muted); }\n\n    /* Pricing */\n    .pricing {\n      background: var(--ink);\n      color: #f2f4f2;\n      border-radius: 20px;\n      padding: 56px 48px;\n      display: grid;\n      grid-template-columns: 1.2fr 1fr;\n      gap: 40px;\n      align-items: center;\n    }\n    .pricing h2 { color: #ffffff; }\n    .pricing p { color: #c7cdd1; margin: 0 0 12px; }\n    .price {\n      font-size: clamp(52px, 7vw, 72px);\n      font-weight: 700;\n      letter-spacing: -0.03em;\n      line-height: 1;\n      margin: 0 0 8px;\n    }\n    .price-unit {\n      font-size: 18px;\n      color: #c7cdd1;\n      margin: 0 0 28px;\n    }\n    .pricing .btn { background: #ffffff; color: var(--ink); border-color: #ffffff; }\n    .pricing .btn-ghost { background: transparent; color: #ffffff; border-color: #ffffff; }\n\n    /* Final CTA */\n    .final {\n      text-align: center;\n      padding: 96px 0;\n    }\n    .final h2 { margin-bottom: 14px; }\n    .final p {\n      color: var(--muted);\n      font-size: 18px;\n      margin: 0 auto 32px;\n      max-width: 48ch;\n    }\n    .final .cta-row { justify-content: center; }\n\n    /* Footer */\n    footer {\n      border-top: 1px solid var(--line);\n      padding: 32px 0;\n      font-size: 14px;\n      color: var(--muted);\n    }\n    .footer-row {\n      display: flex;\n      flex-wrap: wrap;\n      justify-content: space-between;\n      gap: 16px;\n      align-items: center;\n    }\n    footer ul {\n      list-style: none;\n      display: flex;\n      gap: 20px;\n      margin: 0;\n      padding: 0;\n    }\n    footer a { text-decoration: none; }\n    footer a:hover { color: var(--ink); }\n\n    /* Responsive */\n    @media (max-width: 900px) {\n      .stats-grid { grid-template-columns: repeat(2, 1fr); }\n      .switch-grid, .features { grid-template-columns: 1fr; }\n      .use-cases { grid-template-columns: 1fr; }\n      .pricing { grid-template-columns: 1fr; padding: 40px 28px; }\n    }\n\n    @media (max-width: 640px) {\n      body { font-size: 16px; }\n      .nav-links li:not(:last-child) { display: none; }\n      .hero { padding: 36px 0 56px; }\n      section.block { padding: 64px 0; }\n      .stats-grid { grid-template-columns: 1fr 1fr; gap: 20px; }\n      .cta-row .btn { width: 100%; text-align: center; }\n      .final { padding: 72px 0; }\n    }\n  </style>\n</head>\n<body>\n\n  <div class=\"wrap\">\n    <header class=\"nav\">\n      <a class=\"logo\" href=\"#top\">Fieldnote</a>\n      <ul class=\"nav-links\">\n        <li><a href=\"#features\">Features</a></li>\n        <li><a href=\"#pricing\">Pricing</a></li>\n        <li><a class=\"btn btn-small\" href=\"#download\">Download</a></li>\n      </ul>\n    </header>\n  </div>\n\n  <main id=\"top\">\n    <div class=\"wrap\">\n      <section class=\"hero\">\n        <span class=\"eyebrow\">For property inspectors</span>\n        <h1>Inspection reports in 20 minutes, not 90.</h1>\n        <p class=\"lede\">Fieldnote is the app for move-in, move-out and insurance inspections. Photo-first checklists, timestamped PDF reports with GPS, and client e-signature, all from your phone.</p>\n        <div class=\"cta-row\" id=\"download\">\n          <a class=\"btn\" href=\"#\">Download on the App Store</a>\n          <a class=\"btn btn-ghost\" href=\"#features\">See how it works</a>\n        </div>\n        <p class=\"proof-line\">4.8 stars from 600 App Store reviews. Used by 210 inspection businesses, mostly across the UK.</p>\n      </section>\n    </div>\n\n    <section class=\"stats\" aria-label=\"Results\">\n      <div class=\"wrap stats-grid\">\n        <div>\n          <p class=\"stat-value\">90 to 20 min</p>\n          <p class=\"stat-label\">Time to finish a report, down from about 90 minutes</p>\n        </div>\n        <div>\n          <p class=\"stat-value\">+1 to 2</p>\n          <p class=\"stat-label\">More inspections per inspector, per day</p>\n        </div>\n        <div>\n          <p class=\"stat-value\">4.8</p>\n          <p class=\"stat-label\">Stars on the App Store, from 600 reviews</p>\n        </div>\n        <div>\n          <p class=\"stat-value\">210</p>\n          <p class=\"stat-label\">Inspection businesses using Fieldnote</p>\n        </div>\n      </div>\n    </section>\n\n    <div class=\"wrap\">\n      <section class=\"block\">\n        <div class=\"section-head\">\n          <h2>Built for how inspections actually run</h2>\n          <p>Most inspectors start in a Word template or a generic form app. Fieldnote is made for the job itself, so you stop reformatting at the end of the day.</p>\n        </div>\n        <div class=\"switch-grid\">\n          <article class=\"card\">\n            <h3>Photos sit with the checklist</h3>\n            <p>Each item starts with a photo. Find the room and the defect in one place, not across a camera roll and a folder of files.</p>\n          </article>\n          <article class=\"card\">\n            <h3>The report builds itself</h3>\n            <p>Every photo carries a timestamp and GPS location. The PDF report is generated from your checklist, ready to send.</p>\n          </article>\n          <article class=\"card\">\n            <h3>Clients sign on the spot</h3>\n            <p>Your client signs the report on your phone before you leave. No chasing signatures or printing forms.</p>\n          </article>\n        </div>\n        <div class=\"before-after\">\n          <p>Before Fieldnote, a report took around 90 minutes of typing after the visit. With it, the same report takes about 20.</p>\n        </div>\n      </section>\n\n      <section class=\"block\" id=\"features\">\n        <div class=\"section-head\">\n          <h2>Everything from the visit to the signature</h2>\n          <p>Three things Fieldnote does, end to end.</p>\n        </div>\n        <div class=\"features\">\n          <article class=\"card\">\n            <span class=\"feature-index\">01</span>\n            <h3>Photo-first checklists</h3>\n            <p>Start every room with a photo, then add notes against it. The checklist keeps each finding tied to its photo.</p>\n          </article>\n          <article class=\"card\">\n            <span class=\"feature-index\">02</span>\n            <h3>Timestamped PDF reports</h3>\n            <p>Reports are generated automatically, with timestamps and GPS on every photo. Send the PDF straight from your phone.</p>\n          </article>\n          <article class=\"card\">\n            <span class=\"feature-index\">03</span>\n            <h3>Client e-signature</h3>\n            <p>Clients sign the finished report on your device, so the signed version is ready to send without printing.</p>\n          </article>\n        </div>\n      </section>\n\n      <section class=\"block\">\n        <div class=\"section-head\">\n          <h2>Made for the inspections you already do</h2>\n        </div>\n        <div class=\"use-cases\">\n          <article class=\"use-case card\">\n            <h3>Rental move-in and move-out</h3>\n            <p>Record the condition of each room at the start and end of a tenancy, with photos and signatures on both.</p>\n          </article>\n          <article class=\"use-case card\">\n            <h3>Insurance inspections</h3>\n            <p>Document property condition with time and location on every photo, and hand over a report your client can use.</p>\n          </article>\n        </div>\n      </section>\n\n      <section class=\"block\" id=\"pricing\">\n        <div class=\"pricing\">\n          <div>\n            <h2>Simple pricing, per inspector</h2>\n            <p>Priced for small teams of one to ten inspectors. Every inspector on your team is charged the same rate.</p>\n          </div>\n          <div>\n            <p class=\"price\">\u00a329</p>\n            <p class=\"price-unit\">per inspector, per month</p>\n            <a class=\"btn\" href=\"#download\">Download on the App Store</a>\n          </div>\n        </div>\n      </section>\n\n      <section class=\"final\">\n        <h2>Finish your next report in 20 minutes.</h2>\n        <p>Download Fieldnote and run your next inspection with it.</p>\n        <div class=\"cta-row\">\n          <a class=\"btn\" href=\"#\">Download on the App Store</a>\n        </div>\n      </section>\n    </div>\n  </main>\n\n  <footer>\n    <div class=\"wrap footer-row\">\n      <span>&copy; Fieldnote. Made for property inspectors.</span>\n      <ul>\n        <li><a href=\"#features\">Features</a></li>\n        <li><a href=\"#pricing\">Pricing</a></li>\n        <li><a href=\"#\">Privacy</a></li>\n        <li><a href=\"#\">Terms</a></li>\n      </ul>\n    </div>\n  </footer>\n\n</body>\n</html>\n"}]}}, "verdicts": [{"vs": "opus", "pair": "1", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "haiku+skill", "why": "A stays closer to the supplied facts, despite an unqualified hero claim and publishing placeholders. B invents photo-level timestamp/GPS guarantees, on-site completion by all 210 customers, and billing terms. Its finished HTML does not offset those inaccuracies."}, {"vs": "opus", "pair": "1", "judge": "gpt", "order": "opus shown first", "winner": "haiku+skill", "why": "B explicitly identifies the app and accurately uses all supplied evidence. A invents on-site completion for all 210 businesses and GPS stamps on every photo. B needs placeholder removal, but its copy is more faithful to the notes."}, {"vs": "opus", "pair": "1", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "tie", "why": "Both meet all criteria fully, delivering clear, accurate, and complete homepage copy."}, {"vs": "opus", "pair": "1", "judge": "deepseek", "order": "opus shown first", "winner": "opus", "why": "A is a complete, publishable homepage. B is a draft with missing sections and a vague CTA."}, {"vs": "haiku", "pair": "1", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "haiku+skill", "why": "A stays closer to the supplied facts, despite an unqualified hero and removable placeholders. B invents timestamps and GPS on every photo, promises completion before leaving, and asserts no pricing tiers. Both clearly explain the product and buyer workflow."}, {"vs": "haiku", "pair": "1", "judge": "gpt", "order": "haiku shown first", "winner": "haiku+skill", "why": "B follows the supplied facts and pricing more closely. A invents per-photo timestamp/GPS stamping, promises completion and sending before departure, and asserts no pricing tiers. B needs its asset and testimonial placeholders removed before publication."}, {"vs": "haiku", "pair": "1", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "tie", "why": "Both meet all criteria: clear purpose, audience fit, workflow explanation, accurate evidence, clear pricing, and complete copy."}, {"vs": "haiku", "pair": "1", "judge": "deepseek", "order": "haiku shown first", "winner": "haiku", "why": "A provides complete, publishable homepage copy. B is a draft with placeholders and internal notes."}, {"vs": "opus", "pair": "2", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "haiku+skill", "why": "A preserves the supplied results and flags uncertainties, though placeholders impede publication. B invents per-photo metadata, live report generation, on-site signing and all-inclusive pricing, and misrepresents report-writing time as the entire photo-to-PDF workflow."}, {"vs": "opus", "pair": "2", "judge": "gpt", "order": "opus shown first", "winner": "haiku+skill", "why": "B preserves the supplied evidence and price, but needs placeholders removed. A invents guaranteed on-site completion, misrepresents report time as first-photo-to-finished-PDF time, and adds an unsupported promise of no other charges."}, {"vs": "opus", "pair": "2", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "opus", "why": "B is a complete, publishable homepage with accurate evidence and clear pricing. A has placeholders and requires substantial editing."}, {"vs": "opus", "pair": "2", "judge": "deepseek", "order": "opus shown first", "winner": "opus", "why": "A is a complete, publishable homepage that meets all criteria. B is an outline with placeholders, not a finished deliverable."}, {"vs": "opus", "pair": "3", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "haiku+skill", "why": "A stays closer to the supplied facts, though it omits 210 customers and includes a screenshot placeholder. B\u2019s arithmetic is correct, but it invents guaranteed on-site completion, immediate signatures and timestamp/GPS stamping on every photo."}, {"vs": "opus", "pair": "3", "judge": "gpt", "order": "opus shown first", "winner": "haiku+skill", "why": "B explains the app and workflow more faithfully. A invents universal on-site completion and per-photo GPS guarantees. B omits 210 customer businesses and leaves a visible screenshot placeholder. Both prices are accurate, but primary CTA links are nonfunctional."}, {"vs": "opus", "pair": "3", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "tie", "why": "Both meet all criteria, accurately using the notes and presenting a complete, clear homepage."}, {"vs": "opus", "pair": "3", "judge": "deepseek", "order": "opus shown first", "winner": "opus", "why": "A is a complete, publishable homepage with accurate arithmetic and clear pricing. B has placeholder text, incomplete CTA, and minor factual errors."}, {"vs": "haiku", "pair": "3", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "haiku+skill", "why": "A explains the workflow with fewer unsupported claims, but omits 210 customers and includes a visible placeholder. B invents per-photo GPS and on-site signing details, implies guaranteed 20-minute reports, and repeats sections."}, {"vs": "haiku", "pair": "3", "judge": "gpt", "order": "haiku shown first", "winner": "haiku+skill", "why": "B handles report-time evidence more carefully, though it omits 210 customers and leaves a screenshot placeholder. A promises the next report in 20 minutes and invents per-photo GPS/timestamps. Both explain the workflow and pricing; A repeats sections unnecessarily."}, {"vs": "haiku", "pair": "3", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "tie", "why": "Both meet all criteria: clear purpose, audience fit, workflow explanation, accurate evidence, clear pricing, and complete copy."}, {"vs": "haiku", "pair": "3", "judge": "deepseek", "order": "haiku shown first", "winner": "haiku", "why": "A is a complete, publishable homepage. B has placeholder text and incomplete sections, requiring more work."}, {"vs": "haiku", "pair": "2", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "haiku+skill", "why": "A better matches the supplied workflow, audience and evidence, but publishing placeholders undermine completeness. B omits 210 customers and generic form apps, invents per-photo GPS/timestamps and on-the-spot signing, and uses a nonfunctional download CTA."}, {"vs": "haiku", "pair": "2", "judge": "gpt", "order": "haiku shown first", "winner": "tie", "why": "A provides usable copy but omits 210 businesses and generic form apps, overstates on-site completion, and has dead CTAs. B better matches the evidence and buyers, but extensive placeholders and confirmation requests leave substantial publishing work."}, {"vs": "haiku", "pair": "2", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "haiku", "why": "B is a complete, publishable homepage with accurate evidence and no placeholders."}, {"vs": "haiku", "pair": "2", "judge": "deepseek", "order": "haiku shown first", "winner": "haiku", "why": "A is a complete, publishable homepage. B is a draft with placeholders and missing sections."}]}, {"id": "home-3", "family": "homepage", "prompt": "Write our landing page copy. Product info in product.md.", "input": "product.md", "skill": "pierri-homepage-positioning", "input_text": "Product: Briefcraft, turns sales call recordings into a filled-in CRM opportunity (next steps, stakeholders, objections, close date) for B2B sales teams on HubSpot.\nBuyer: VP Sales / RevOps at 20-200 person B2B companies. Users: account executives.\nProof: AEs save about 4 hours/week on CRM updates; forecast calls use real data instead of guesses; 52 customers; native HubSpot app with 4.7 rating.\nAlternatives: Gong (expensive, built for big teams), AEs typing notes, generic AI note-takers that don't update the CRM fields.\n", "rubric": [{"id": "c1", "criterion": "Make the product\u2019s core value immediately clear.", "full_marks_when": "The headline and supporting copy explain that Briefcraft turns sales call recordings into filled-in HubSpot CRM opportunities, with a clear benefit such as less manual updating or better forecast data.", "zero_when": "The copy presents Briefcraft as a generic transcription or meeting-summary tool, or leaves readers unable to understand what it does."}, {"id": "c2", "criterion": "Speak to the specified buyers while making the benefit for users concrete.", "full_marks_when": "The copy addresses VP Sales and RevOps buyers at 20\u2013200-person B2B companies through relevant concerns such as CRM completeness and forecast confidence, while showing how account executives spend less time updating records.", "zero_when": "The copy targets unrelated audiences or relies entirely on generic productivity claims without connecting AE time savings to sales leadership needs."}, {"id": "c3", "criterion": "Describe the actual workflow and outputs specifically.", "full_marks_when": "The copy connects sales call recordings to populated CRM opportunity fields and names next steps, stakeholders, objections, and close date, making the difference from receiving a separate set of notes clear.", "zero_when": "The copy omits the CRM field updates or substitutes unsupported capabilities such as autonomous selling, lead generation, or guaranteed deal closure."}, {"id": "c4", "criterion": "Use the supplied facts and numbers accurately without inventing evidence.", "full_marks_when": "The copy accurately incorporates approximately 4 hours saved per AE per week on CRM updates, 52 customers, and a native HubSpot app rated 4.7; it supports forecast confidence without promising perfect forecasts and invents no testimonials, customer names, financial results, or rating denominator.", "zero_when": "The copy materially changes a supplied fact or number, turns approximate savings into a guarantee, makes arithmetic errors in derived claims, or fabricates supporting evidence."}, {"id": "c5", "criterion": "Give buyers a credible reason to choose Briefcraft over their current alternatives.", "full_marks_when": "The copy distinguishes direct CRM field updates from manual AE notes and generic AI note-takers, and positions Briefcraft for smaller B2B teams relative to Gong\u2019s big-team focus without inventing Briefcraft pricing or unsupported competitor limitations.", "zero_when": "The copy offers no meaningful differentiation, claims all alternatives lack features without support, or asserts unprovided price comparisons or cost savings."}, {"id": "c6", "criterion": "Deliver landing page copy that can be used with minimal editing.", "full_marks_when": "The answer supplies finished, coherent copy with a strong hero, scannable benefits, a concise explanation of how it works, proof, and a clear, consistent conversion CTA; it avoids planning notes, unresolved placeholders, and unsupported offers such as a free trial or no-credit-card signup.", "zero_when": "The answer provides advice or an outline instead of finished copy, lacks a clear next action, or requires substantial rewriting to remove placeholders, contradictions, or invented commercial terms."}], "runs": {"haiku-skill-1": {"cost_usd": 0.00697, "seconds": 33, "output_tokens": 6457, "files": [{"name": "landing-page-copy.md", "text": "# Briefcraft landing page copy\n\nDraft. Items marked **[CONFIRM]** need a fact, asset, or link before publishing.\n\n---\n\n## Meta\n\n- **Title:** Briefcraft | Sales call recordings, turned into HubSpot deal updates\n- **Description:** Briefcraft reads your sales call recordings and fills in next steps, stakeholders, objections, and close dates on the HubSpot deal. Built for B2B sales teams.\n\n---\n\n## Hero\n\n**H1:** Turn every sales call into an updated HubSpot deal.\n\n**Supporting line:** Briefcraft reads your call recordings and fills in next steps, stakeholders, objections, and close date, so your reps spend less time on CRM updates and your forecast runs on what was actually said.\n\n**Primary CTA:** Install on HubSpot **[CONFIRM: marketplace listing URL]**\n\n**Secondary CTA:** Book a demo **[CONFIRM: sales motion exists for this path]**\n\n**Product view:** Screenshot or short loop showing a call recording beside the HubSpot deal record, with next steps, stakeholders, objections, and close date already populated. **[CONFIRM: asset does not exist yet]**\n\n---\n\n## The problem\n\n**Heading:** Your forecast is only as good as the notes behind it.\n\n**Body:** After every call, account executives replay the recording, type up what matters, and hunt for the right fields in HubSpot. Updates arrive late or get skipped. By the time the forecast call starts, deal records describe what someone remembered, not what the buyer said.\n\n---\n\n## The alternatives\n\n**Heading:** Typing notes is slow. Note-takers don't finish the job.\n\n**Body:** Manual notes take time reps don't have. Generic AI note-takers produce a summary, but it lives in a separate tool and your HubSpot fields stay empty.\n\n---\n\n## The solution\n\n**Heading:** Recordings go in. Structured deal fields come out.\n\n**Body:** Briefcraft works from the call itself. It pulls out the outcomes your team needs and writes them to the HubSpot deal, so the record is current before anyone opens it. **[CONFIRM: how recordings reach Briefcraft and which recording platforms are supported]**\n\n---\n\n## Value 1\n\n**Heading:** Next steps and stakeholders, ready for the forecast call.\n\n**Body:** Briefcraft captures the agreed next steps and everyone involved in the deal. Managers see the same picture the rep saw, without a status meeting to reconstruct it.\n\n---\n\n## Value 2\n\n**Heading:** Objections and close dates, from the conversation itself.\n\n**Body:** Objections raised on the call and the close date discussed are recorded against the deal. Nobody re-listens to a recording to find out what the buyer said.\n\n---\n\n## Proof\n\n- **52 companies** use Briefcraft **[CONFIRM: count date]**\n- Account executives save **about 4 hours a week** on CRM updates **[CONFIRM: measurement method and sample]**\n- Rated **4.7** on the HubSpot marketplace **[CONFIRM: current rating and source]**\n\n**Customer quote:** [Pending: needs a customer quote with permission to publish.]\n\n---\n\n## Who it's for\n\n**Heading:** Built for B2B sales teams of 20 to 200 people that run on HubSpot.\n\n**Body:** Account executives use Briefcraft every day. VP Sales and RevOps leaders use it to get forecasts they can trust.\n\n**Secondary path:** For account executives: see how the deal updates happen **[CONFIRM: page exists or remove]**\n\n---\n\n## Closing CTA\n\n**Heading:** Put your call data to work in HubSpot.\n\n**Button:** Install on HubSpot **[CONFIRM: same destination as primary CTA]**\n\n---\n\n## Internal notes (remove before publishing)\n\n**Audience.** Champion is the VP Sales or RevOps leader who owns forecast accuracy. The user is the account executive. Both are in scope; the champion drives the purchase, so the forecast argument leads.\n\n**Market state (assumed M2).** Buyers already know call recording and AI note-taking. The copy therefore leads with the HubSpot outcome and does not explain what call notes are. Validate this with two or three buyer conversations.\n\n**Category reference.** \"HubSpot deal updates from call recordings\" describes the job in the buyer's terms. It avoids the \"conversation intelligence\" category, where Gong is the established reference point and the comparison would favor the larger product.\n\n**Position sentence.** For RevOps and sales leaders running HubSpot, Briefcraft is a call-recording tool that writes next steps, stakeholders, objections, and close dates into the deal, replacing manual notes and summaries that never reach the CRM.\n\n**Deliberately excluded.**\n- Head-to-head comparison with Gong. Publishing \"expensive\" or \"built for big teams\" needs sourced evidence first.\n- Naming competitors on the page.\n- Enterprise positioning. The audience is 20 to 200 people.\n\n**Reconsideration trigger.** If buyers in the target range mostly arrive by searching for Gong alternatives, the page needs a comparison section and sourced claims about cost and scope.\n\n---\n\n## Claims and gaps\n\n| Claim on page | Evidence | Status |\n| --- | --- | --- |\n| Writes next steps, stakeholders, objections, close date to HubSpot | Product description | Supported. Confirm the exact field names match the app. |\n| Native HubSpot app | Product description | Supported. Link needed. |\n| About 4 hours a week saved per AE | Internal figure | Revise if the measurement method is unknown. Add the source or qualify as \"customer-reported.\" |\n| 52 companies | Internal figure | Supported with date. |\n| 4.7 rating on HubSpot | Internal figure | Confirm against the live listing before publishing. |\n| Gong comparison (not on page) | None | Omitted until sourced. |\n| Forecasts use real data | Internal description | Supported as a qualitative claim. Backed by a customer quote when available. |\n\n**Open items**\n1. Marketplace listing URL and final CTA destination.\n2. Recording intake method and supported platforms.\n3. Product screenshot or short demo loop.\n4. Customer quote with publishing permission.\n5. Source for the 4-hour and 4.7 figures.\n6. Comprehension test: show the hero to two or three VP Sales or RevOps leaders and ask what the product does. Revise the H1 if they describe something different.\n"}]}, "haiku-skill-2": {"cost_usd": 0.00814, "seconds": 41, "output_tokens": 8311, "files": [{"name": "landing-page-copy.md", "text": "# Briefcraft landing page copy\n\nBracketed items are open decisions or facts to confirm before launch. Remove them from the final page.\n\n---\n\n## Hero\n\n**H1:** Your HubSpot pipeline, updated straight from the sales call.\n\n**Supporting line:** Briefcraft reads your account executives' call recordings and fills in next steps, stakeholders, objections, and close dates on the HubSpot opportunity. Your forecast runs on what was said on the call, not on what someone remembered to type.\n\n**Primary CTA:** Book a demo [confirm: demo vs. self-serve trial]\n\n**Product view:** A HubSpot opportunity record beside the call summary, with the next steps, stakeholders, objections, and close date fields filled in by Briefcraft. [Needs a real screenshot; use a sample account, not customer data.]\n\n---\n\n## The problem\n\n**Heading:** Pipeline data is only as good as the last thing someone typed.\n\n**Body:** The call ends and the account executive moves on to the next one. The next steps are in their head, the new stakeholder is in a calendar invite, and the objection is in a notebook. Updating HubSpot gets pushed to the end of the day, or skipped. The forecast call then starts with a debate about whose gut to trust.\n\n---\n\n## The solution\n\n**Heading:** Briefcraft does the CRM work after every call.\n\n**Body:** Briefcraft reads each sales call recording and writes the details your pipeline depends on directly into the HubSpot opportunity: next steps, stakeholders, objections, and the close date. Your account executives spend their time selling. Your VP Sales gets a record that matches the conversation. [Confirm how recordings reach Briefcraft and which recording tools are supported.]\n\n---\n\n## Value\n\n### 1. Opportunities with the details your forecast needs\n\nNext steps, stakeholders, objections, and close dates, pulled from what the buyer actually said. Forecast calls start from the record instead of from memory of the last call.\n\n### 2. About four hours a week back for every account executive\n\nAccount executives using Briefcraft save about four hours a week that used to go to CRM updates. That is time for prospecting, prep, and the calls that move deals. [Confirm how the four-hour figure was measured before using \"saves\" as a fact.]\n\n### 3. Built into HubSpot, sized for growing sales teams\n\nBriefcraft is a native HubSpot app with a 4.7 rating. It is built for B2B teams of 20 to 200 people that need an accurate pipeline now, without an enterprise rollout. [Confirm the rating source and review count.]\n\n---\n\n## Why not a generic AI note-taker?\n\n**Heading:** A transcript in another tool doesn't update your pipeline.\n\n**Body:** Note-takers give you a transcript or summary in a separate place. Briefcraft writes to the HubSpot fields your managers and forecast actually use.\n\n---\n\n## Proof\n\n- **52 customers** [confirm the count is current]\n- **4.7 rating** on HubSpot [confirm source]\n- **About four hours a week** saved per account executive [confirm methodology]\n\n> \"[Customer quote]\" \u2014 [Name, title, company] [Pending permission. Omit this block until a quote is approved.]\n\n---\n\n## Closing CTA\n\n**Heading:** See your next call in HubSpot.\n\n**Button:** Book a demo [same destination as the primary CTA]\n\n---\n\n## Internal notes (remove before publishing)\n\n### Positioning record\n\n- **Audience:** VP Sales or RevOps at 20 to 200 person B2B companies on HubSpot. Account executives are the daily users. Market state is unverified; provisionally a team that knows conversation-intelligence tools and has tried generic note-takers.\n- **Product identity:** A tool that writes call details into HubSpot opportunity fields. Reference point: CRM updates from sales calls.\n- **Problem:** CRM updates after calls are late or incomplete, so the forecast is built on guesses.\n- **Alternatives:** Account executives typing notes, Gong (enterprise-oriented), and generic AI note-takers that don't update CRM fields.\n- **Differentiation:** Briefcraft writes structured fields into the HubSpot opportunity. It is a native HubSpot app and targets smaller teams.\n- **Value:** Accurate pipeline data for the VP, and time back for account executives. Forecast quality is a goal to support, not a result to promise.\n\n**Position sentence:** For VP Sales and RevOps leaders at 20 to 200 person B2B companies on HubSpot, Briefcraft is a call-to-CRM tool that writes next steps, stakeholders, objections, and close dates from sales calls into HubSpot opportunities, replacing manual account executive updates.\n\n### Why this position\n\n- **Chosen:** Lead with pipeline accuracy for the VP, since they are the buyer. Support it with account executive time savings, since that is the daily pain they feel.\n- **Rejected:** Competing head-on with Gong on conversation intelligence. We cannot match its depth, and the homepage would invite a feature comparison we would lose.\n- **Rejected:** Positioning as an AI note-taker. That category is crowded, and our real difference is writing to CRM fields.\n- **Reconsider if:** Interviews show the buyer is mainly motivated by coaching or deal review rather than CRM accuracy.\n\n### Claims check\n\n| Claim on the page | Source | Status |\n| --- | --- | --- |\n| Turns call recordings into filled-in HubSpot opportunities | product.md | Supported |\n| Fields: next steps, stakeholders, objections, close date | product.md | Supported |\n| Native HubSpot app | product.md | Supported |\n| 52 customers | product.md | Supported; confirm current |\n| 4.7 rating | product.md | Supported; confirm source |\n| About four hours a week saved per AE | product.md | Revise to \"about four hours a week\" with methodology confirmed |\n| Forecast calls use real data | product.md | Revised to \"forecast runs on what was said on the call\"; outcome not promised |\n| Gong is expensive | product.md | Omitted; no pricing evidence |\n| Note-takers don't update CRM fields | product.md | Stated as what Briefcraft does, not as a knock on competitors |\n| Built for 20 to 200 person teams | product.md | Target market, not an exclusivity claim |\n\n### Gaps before launch\n\n- **Recording integrations:** Which recording tools are supported, and how recordings reach Briefcraft.\n- **Sales motion:** Demo-led or self-serve trial. This sets the primary CTA.\n- **Time-saved method:** How the four-hour figure was measured and over what sample.\n- **Accuracy:** How often the extracted fields are right, and whether account executives review them.\n- **Call recording consent and data handling:** Buyers will ask. Answer on the page or in an FAQ.\n- **Customer evidence:** At least one approved quote and one customer logo, if permitted.\n- **Visuals:** A real product screenshot built on sample data.\n- **Comprehension test:** Show the hero and supporting line to two or three VP Sales or RevOps leaders and ask what the product does. Revise if they can't repeat it back.\n"}]}, "haiku-skill-3": {"cost_usd": 0.01174, "seconds": 58, "output_tokens": 11968, "files": [{"name": "landing-page-copy.md", "text": "# Briefcraft landing page copy\n\nStatus: Provisional. Copy uses only the facts in product.md. Items marked [CONFIRM] need a source or asset before publishing. The internal notes at the end must be removed before the page goes live.\n\n---\n\n## Hero\n\n**H1:** Your sales calls, turned into complete HubSpot opportunities.\n\n**Supporting line:** Briefcraft reads your call recordings and fills in next steps, stakeholders, objections, and close date. Built for B2B sales teams of 20 to 200 people.\n\n**Primary CTA:** Add Briefcraft to HubSpot [CONFIRM: Marketplace listing URL]\n\n**Product view:** Screenshot of a real HubSpot opportunity filled from one call, showing next steps, stakeholders, objections, and close date. [CONFIRM: asset needed. Use real product output only.]\n\n---\n\n## The problem\n\n**Heading:** The CRM only knows what someone had time to type.\n\nAfter a call, an account executive either types up the notes, writes a summary somewhere else, or waits until later. Next steps slip, stakeholders go unrecorded, and objections stay in one rep's head. By the forecast call, the numbers are a best guess.\n\n---\n\n## The solution\n\n**Heading:** Briefcraft writes the call into the opportunity.\n\nBriefcraft analyzes each sales call recording and updates the matching HubSpot opportunity with what was said: the next steps agreed, the people involved, the objections raised, and the close date discussed. The account executive doesn't write the update. Managers and the forecast read it from the record.\n\n---\n\n## Value 1\n\n**Heading:** Next steps and stakeholders, recorded after every call.\n\nBriefcraft pulls the agreed next steps and the people involved into the opportunity. Nobody has to remember who signed off on pricing or who joined the last call. The record is current before the next conversation starts.\n\n---\n\n## Value 2\n\n**Heading:** Objections and close dates you can forecast on.\n\nObjections and close dates come from what was said on the call, not from what a rep recalls at the end of the week. Forecast calls start from the same record your team works in.\n\n---\n\n## Value 3\n\n**Heading:** Lives in HubSpot, not in another dashboard.\n\nBriefcraft is a native HubSpot app. Updates land in the opportunities your team already uses for pipeline and reporting, so there is no second place to check. [CONFIRM: native app listing details]\n\n---\n\n## Proof\n\n- **52 customers** using Briefcraft [CONFIRM: current count]\n- **AEs save about 4 hours a week** on CRM updates [CONFIRM: source and method before publishing. Without one, remove the number or describe it as an estimate.]\n- **Forecast calls use real data instead of guesses**\n- **4.7 rating** as a native HubSpot app [CONFIRM: marketplace and review count]\n\nCustomer quote: pending permission. Do not publish a placeholder.\n\n---\n\n## Secondary paths\n\n- For account executives: how Briefcraft handles your call notes [CONFIRM: page exists before linking]\n- Compare Briefcraft with Gong and AI note-takers [CONFIRM: separate page, not homepage content]\n\n---\n\n## Closing CTA\n\n**Heading:** Your next forecast call should start with what was actually said.\n\n**Button:** Add Briefcraft to HubSpot\n\n---\n---\n\n# Internal notes (remove before publishing)\n\n## Positioning record\n\n- **Champion:** VP Sales or RevOps at a 20 to 200 person B2B company. They own forecast accuracy and CRM data quality and can install the app. Account executives are the users. Their time savings are proof, not the lead.\n- **Market state:** M2 for the CRM-update activity. Manual entry is the current method. Some buyers already know Gong and AI note-takers, so parts of the audience are M3. The page therefore has to lead with CRM write-back, not with why call notes are useful.\n- **Category route:** Mature category (AI call analysis). The entry point is an underserved segment (20 to 200 people, HubSpot-native) and writing to structured opportunity fields.\n- **Comparator:** Account executives typing notes, and generic AI note-takers that produce summaries but do not update CRM fields.\n- **Position sentence:** For a RevOps or VP Sales leader at a 20 to 200 person B2B company who needs accurate HubSpot opportunities, Briefcraft is a call-analysis app that writes next steps, stakeholders, objections, and close date into the opportunity, so the CRM stays complete without rep typing.\n\n## Options considered\n\n1. **Recommended: CRM completeness for HubSpot teams** (RevOps or VP Sales). Risk: medium. The field-level claim has to be shown in the product view.\n2. **Forecast accuracy** (VP Sales). Risk: medium to high. The link between call data and forecast quality is not yet shown. Kept as Value 2, not the headline.\n3. **Rep time savings** (account executives). Risk: low for proof, but account executives are not the buyer. Used as proof.\n4. **Conversation intelligence for mid-market, against Gong.** Risk: high. Gong is established, and product.md does not cover coaching or analytics features we could match. Rejected for the homepage.\n\n## Open items\n\n| Item | Why it matters | Next step |\n| --- | --- | --- |\n| Source for \"4 hours a week\" | Customer-facing number with no stated basis | Find the survey or time study, or drop the number |\n| Marketplace listing and rating details | Core proof and the primary CTA destination | Confirm listing URL, rating platform, and review count |\n| Product screenshot | Shows the field-level claim, which the page depends on | Capture a real opportunity filled from a real call |\n| Supported recording sources | Solution steps depend on how recordings get in | Confirm the integrations and describe them accurately |\n| Trial or demo availability | Determines whether the CTA is an install or a demo request | Confirm the available next step |\n| Customer quote permission | Testimonials are not included without it | Request permission from 2 or 3 customers |\n| Problem statements | \"Notes slip\" and \"forecast is a best guess\" are not in product.md | Test with 2 to 3 account executives and 1 RevOps lead |\n\n## Claims check\n\n| Claim | Basis | Status |\n| --- | --- | --- |\n| Fills next steps, stakeholders, objections, close date | product.md | Supported. Confirm the fields write to standard HubSpot properties. |\n| 52 customers | Provided by the team | Supported. Confirm current count. |\n| Native HubSpot app, 4.7 rating | Provided by the team | Supported. Confirm platform and review count. |\n| Account executives save about 4 hours a week | Provided by the team, no method | Validation pending |\n| Forecast calls use real data | Provided by the team | Supported as an outcome. Keep it qualified. |\n| Notes slip, objections stay with one rep | Not in product.md | Assumption. Validation pending. |\n\n## Checklist result\n\n- Evidence integrity: pass. product.md is the only source, and assumptions are marked.\n- Reachable champion: pass on definition. Reach through the actual acquisition channel is pending.\n- Behavioral segment: pass. M2 is justified by manual CRM entry.\n- Correct comparator: pass. Typing notes and summary-only note-takers are addressed.\n- Honest outcomes: pass with the 4-hour claim flagged. The page promises clarity, not conversion or revenue impact.\n- Comprehension check: pending. Run 3 to 5 five-second tests with RevOps or sales leaders.\n- Workflow check: pending. Confirm the problem statements with account executives.\n\n**Status:** Provisional. The hero, value sections, and closing CTA are usable once the CTA destination and the 4-hour claim are confirmed.\n"}]}, "opus-1": {"cost_usd": 0.10943, "seconds": 34, "output_tokens": 2988, "files": [{"name": "landing-page.md", "text": "# Briefcraft \u2014 Landing Page Copy\n\n---\n\n## Hero\n\n**Headline:**\nYour sales calls, already in HubSpot.\n\n**Subheadline:**\nBriefcraft turns sales call recordings into a filled-in HubSpot opportunity: next steps, stakeholders, objections and close date. Your AEs stop typing notes, and your forecast runs on what buyers actually said.\n\n**Primary CTA:** Get a demo\n**Secondary CTA:** Install from the HubSpot Marketplace\n\n**Trust line (under CTAs):**\nNative HubSpot app \u00b7 Rated 4.7 \u00b7 Used by 52 B2B sales teams\n\n---\n\n## Problem\n\n**Headline:**\nYour forecast is only as good as the notes your AEs had time to type.\n\n**Body:**\nAfter a call, an AE has two choices: spend 20 minutes updating the CRM, or get to the next deal. Most pick the deal. So the opportunity record goes stale. The close date is a guess, nobody wrote down the stakeholders, and the objection that will kill the deal lives only in someone's head.\n\nThen, come forecast call, you're asking reps to fill in from memory.\n\n---\n\n## Solution / How it works\n\n**Headline:**\nFrom recording to updated opportunity, without anyone typing.\n\n**Step 1 \u2014 Record the call.**\nRun your sales calls as usual. Briefcraft works from the recording.\n\n**Step 2 \u2014 Briefcraft pulls out what matters.**\nIt finds the details a deal record needs: what was agreed as next steps, who's involved on the buyer side, what objections came up, and when the buyer expects to decide.\n\n**Step 3 \u2014 Your HubSpot opportunity fills itself in.**\nThe right fields are updated on the right deal, inside HubSpot, where your team already works.\n\n---\n\n## What gets captured\n\n**Headline:**\nThe fields your forecast depends on.\n\n- **Next steps:** What was agreed, and who owns it. Nothing falls through between calls.\n- **Stakeholders:** Who's in the deal, beyond the person on the call. Multi-threading you can see.\n- **Objections:** Pricing, timing, competitors, internal blockers, written down in the buyer's terms.\n- **Close date:** Based on what the buyer said, not on what the rep hopes.\n\n---\n\n## Outcomes\n\n**Headline:**\nLess admin for reps. Real data for leadership.\n\n**For account executives**\n**About 4 hours back every week.** That's how much time AEs save on CRM updates with Briefcraft. They spend it selling instead.\n\n**For VP Sales**\n**Forecast calls built on real data.** Stop asking reps \"how confident are you?\" Look at what the buyer said, captured from the call.\n\n**For RevOps**\n**A CRM you can trust.** Opportunity fields get filled in consistently on every deal, not just when a rep remembers.\n\n---\n\n## Comparison\n\n**Headline:**\nBuilt for sales teams of 20 to 200.\n\n**Gong and enterprise conversation intelligence**\nPowerful, but priced and designed for large sales orgs. Briefcraft does one job, keeping your HubSpot opportunities current, without an enterprise contract.\n\n**AEs typing notes**\nSlow, inconsistent, and the first thing dropped in a busy week. Briefcraft does it after every call.\n\n**Generic AI note-takers**\nThey give you a summary. Someone still has to copy it into the CRM. Briefcraft updates the deal fields directly.\n\n| | Briefcraft | Enterprise CI tools | Generic AI note-takers | Manual notes |\n|---|---|---|---|---|\n| Updates HubSpot opportunity fields | \u2713 | Varies | \u2717 | Only if someone does it |\n| Built for 20\u2013200 person teams | \u2713 | \u2717 | \u2713 | \u2713 |\n| No typing for AEs | \u2713 | \u2713 | Partly | \u2717 |\n| Native HubSpot app | \u2713 | Varies | Varies | n/a |\n\n---\n\n## Social proof\n\n**Headline:**\nTrusted by 52 B2B sales teams on HubSpot.\n\n**Stats bar:**\n- **52** customers\n- **~4 hrs/week** saved per AE on CRM updates\n- **4.7\u2605** on the HubSpot App Marketplace\n\n[PLACEHOLDER: Customer logos. Only use ones you have permission to show.]\n\n[PLACEHOLDER: 1\u20132 customer quotes, ideally one from a VP Sales on forecasting and one from an AE on time saved. Use real quotes with name, title and company.]\n\n---\n\n## FAQ\n\n**Does Briefcraft work with Salesforce or other CRMs?**\nBriefcraft is built for HubSpot, as a native HubSpot app.\n\n**Who is it for?**\nB2B sales teams of roughly 20\u2013200 people running their pipeline in HubSpot. Typically VP Sales or RevOps buys it, and account executives use it every day.\n\n**What does it update?**\nOpportunity next steps, stakeholders, objections and close date, pulled from your sales call recordings.\n\n**How is this different from an AI note-taker?**\nNote-takers produce a summary. Briefcraft fills in the CRM fields, so nobody has to copy and paste.\n\n**How is it different from Gong?**\nGong is a broad platform built and priced for large sales orgs. Briefcraft is focused on keeping HubSpot opportunities accurate for mid-sized teams.\n\n[PLACEHOLDER: Add answers for security/data handling, which call recording tools are supported, setup time, and pricing. These aren't in the product brief, so confirm them with the team before publishing.]\n\n---\n\n## Final CTA\n\n**Headline:**\nGet 4 hours a week back for every AE, and a forecast you can trust.\n\n**Body:**\nSee Briefcraft fill in a real HubSpot opportunity from a sales call.\n\n**Primary CTA:** Get a demo\n**Secondary CTA:** Install from the HubSpot Marketplace\n\n---\n\n### Alternate headline options (for testing)\n\n1. Your sales calls, already in HubSpot. *(used above)*\n2. Stop typing call notes into HubSpot.\n3. Every sales call, turned into an up-to-date HubSpot deal.\n4. Forecast on what buyers said, not what reps remember.\n5. The CRM update your AEs never have to do.\n"}]}, "opus-2": {"cost_usd": 0.10493, "seconds": 30, "output_tokens": 2805, "files": [{"name": "landing-page.md", "text": "# Briefcraft \u2014 Landing Page Copy\n\n---\n\n## Hero\n\n**Headline:**\nYour sales calls, already in HubSpot.\n\n**Subheadline:**\nBriefcraft turns every sales call recording into a filled-in HubSpot opportunity: next steps, stakeholders, objections and close date. Your AEs stop typing notes, and your forecast runs on what buyers actually said.\n\n**Primary CTA:** Book a demo\n**Secondary CTA:** See it on the HubSpot Marketplace\n\n**Trust line (under CTAs):**\nNative HubSpot app \u00b7 Rated 4.7 \u00b7 Used by 52 B2B sales teams\n\n---\n\n## Problem\n\n**Headline:**\nYour forecast is only as good as the last thing an AE typed.\n\n**Body:**\nAfter every call, someone has to update the deal: who was on it, what they pushed back on, what happens next, when it might close. Usually that's a tired AE at 6pm, rushing to the next call. Fields go stale or get guessed. Forecast calls turn into \"what's your gut on this one?\"\n\nThe details were all said out loud on the call. They just never make it into the CRM.\n\n---\n\n## How it works\n\n**Headline:**\nFrom recording to updated deal, without the typing.\n\n**1. Connect HubSpot and your call recordings**\nInstall the native HubSpot app. Briefcraft works with the deals and fields you already use.\n\n**2. Have your calls as usual**\nYour AEs keep running discovery, demos and negotiations the way they do now.\n\n**3. The opportunity fills itself in**\nAfter each call, Briefcraft writes the details into the HubSpot deal:\n- **Next steps:** what was agreed and who owns it\n- **Stakeholders:** who's involved and what role they play\n- **Objections:** what the buyer is worried about\n- **Close date:** based on the timeline the buyer described\n\n---\n\n## Benefits by role\n\n### For VPs of Sales\n**Forecast calls run on real data instead of guesses.**\nClose dates, objections and next steps come from what buyers said on the call, not from memory. You spend the meeting on deal strategy, not on asking for updates.\n\n### For RevOps\n**A CRM you can trust, without nagging anyone.**\nFields get filled in consistently on every deal, in HubSpot, where your reports and dashboards already live. No new system to manage.\n\n### For Account Executives\n**About 4 hours a week back.**\nThat's how much time AEs save on CRM updates with Briefcraft. They spend it selling instead.\n\n---\n\n## Proof\n\n**Headline:**\nBuilt for HubSpot teams. Trusted by 52 of them.\n\n- **~4 hours/week** saved per AE on CRM updates\n- **4.7 rating** on the HubSpot App Marketplace\n- **52 customers** running their pipeline with Briefcraft\n\n> [Customer quote: placeholder. Use a real quote from a VP Sales or RevOps lead, ideally about forecast accuracy or AE time saved. Include name, title, company.]\n\n---\n\n## Comparison\n\n**Headline:**\nThe call intelligence you need, at the size you are.\n\n| | **Briefcraft** | **Gong** | **Generic AI note-takers** | **AEs typing notes** |\n|---|---|---|---|---|\n| Fills in HubSpot deal fields | \u2705 | Partial / setup required | \u274c Notes only | Manually, when they remember |\n| Built for 20\u2013200 person teams | \u2705 | Built for large teams | \u2705 | \u2705 |\n| Cost | Fits a mid-size budget | Expensive | Low | AE selling time |\n| Native HubSpot app | \u2705 | \u2014 | Varies | \u2014 |\n\n*(Have someone check the Gong and note-taker cells before publishing. Keep claims about competitors accurate and fair.)*\n\n**Short version for a single line:**\nGong is built and priced for enterprise. Note-takers give you a summary, but someone still has to update the deal. Briefcraft updates the deal.\n\n---\n\n## FAQ\n\n**Does it work with our existing HubSpot setup?**\nYes. Briefcraft is a native HubSpot app and writes to your deal records. *[Confirm: does it support custom properties / field mapping?]*\n\n**Where do the call recordings come from?**\n*[Fill in: supported recording sources, e.g. Zoom, Google Meet, dialers.]*\n\n**Can AEs review what gets written?**\n*[Fill in: is there a review/approve step, or does it write directly?]*\n\n**How long does setup take?**\n*[Fill in: realistic setup time.]*\n\n**Is our call data secure?**\n*[Fill in: security/compliance details, e.g. SOC 2, data retention.]*\n\n**How is it priced?**\n*[Fill in, or: \"Pricing is per seat and built for teams of 20\u2013200. Book a demo for a quote.\"]*\n\n---\n\n## Final CTA\n\n**Headline:**\nStop asking AEs to update the CRM.\n\n**Body:**\nLet the calls do it. See Briefcraft fill in a real HubSpot deal in a 20-minute demo.\n\n**Primary CTA:** Book a demo\n**Secondary CTA:** Install from the HubSpot Marketplace\n\n---\n\n## Alternate headlines (for A/B testing)\n\n1. Your sales calls, already in HubSpot. *(current)*\n2. Every call fills in the deal.\n3. Forecast on what buyers said, not what AEs remember.\n4. Give every AE 4 hours a week back from CRM updates.\n5. Call notes don't update HubSpot. Briefcraft does.\n\n## Meta\n\n**Page title:** Briefcraft | Sales calls that update HubSpot automatically\n**Meta description:** Briefcraft turns sales call recordings into filled-in HubSpot deals: next steps, stakeholders, objections and close date. AEs save about 4 hours a week. Rated 4.7 on HubSpot.\n"}]}, "opus-3": {"cost_usd": 0.11102, "seconds": 32, "output_tokens": 3044, "files": [{"name": "landing-page.md", "text": "# Briefcraft \u2014 Landing Page Copy\n\n---\n\n## Hero\n\n**Headline:**\nYour sales calls fill in HubSpot. Your AEs get back to selling.\n\n**Subhead:**\nBriefcraft turns every sales call recording into a completed HubSpot opportunity, with next steps, stakeholders, objections and close date. Nobody has to type any of it.\n\n**Primary CTA:** Install from the HubSpot Marketplace\n**Secondary CTA:** See a sample deal record\n\n**Trust line (under CTAs):**\nNative HubSpot app \u00b7 Rated 4.7 on the HubSpot Marketplace \u00b7 Used by 52 B2B sales teams\n\n---\n\n## Problem\n\n**Headline:**\nYour forecast depends on notes nobody has time to write.\n\n**Body:**\nAfter every call, your AEs have a choice: spend 15 minutes updating HubSpot, or move on to the next deal. Most of them move on. The opportunity record ends up with a vague next step, one contact, and a close date someone guessed weeks ago.\n\nThen the forecast call comes around, and you're asking reps what's really happening instead of looking at the pipeline.\n\n---\n\n## How it works\n\n**Headline:**\nFrom recording to updated deal in three steps.\n\n**1. Connect HubSpot and your call recordings**\nInstall Briefcraft from the HubSpot Marketplace. Calls get matched to the right deal.\n\n**2. Briefcraft listens for what matters to the deal**\nIt pulls out what moves an opportunity forward: who was on the call, what they pushed back on, what was agreed, and when they expect to sign.\n\n**3. The opportunity fills itself in**\nNext steps, stakeholders, objections and close date are written straight into HubSpot's deal fields. You don't get a summary that someone still has to copy over.\n\n---\n\n## What gets filled in\n\n| Field | What Briefcraft captures |\n|---|---|\n| **Next steps** | Every commitment from the call, with owner and date |\n| **Stakeholders** | Who was in the room, their role, and who has buying power |\n| **Objections** | Pricing, timing, competitor and security concerns, in the buyer's own words |\n| **Close date** | Updated from what the buyer actually said, not from the rep's hopes |\n\n---\n\n## Benefits by role\n\n### For VP Sales: a forecast built on what buyers said\nWhen every deal is updated after every call, your forecast reviews use real data. You can see which deals have a confirmed next step and which have gone quiet, and you don't have to chase reps to find out.\n\n### For RevOps: clean CRM data without nagging anyone\nFields that used to sit empty get filled in the same way on every deal. Reporting, routing and pipeline hygiene finally have something to work with.\n\n### For Account Executives: about 4 hours a week back\nThat's roughly how much time AEs save on CRM updates with Briefcraft. They finish the call, check the deal, and move on.\n\n---\n\n## Comparison\n\n**Headline:**\nBuilt for teams of 20 to 200, not 2,000.\n\n| | **Briefcraft** | **Gong** | **Generic AI note-takers** | **AEs typing notes** |\n|---|---|---|---|---|\n| Fills in HubSpot deal fields | \u2705 Automatically | Partially, through setup | \u274c Notes only | \u2705 When they have time |\n| Built for mid-size B2B teams | \u2705 | Built for large enterprises | \u2705 | \u2014 |\n| Price fits a 20\u2013200 person company | \u2705 | \u274c Enterprise pricing | \u2705 | \"Free\" (costs ~4 hrs/week per AE) |\n| Native HubSpot app | \u2705 4.7 rating | Integration | Varies | \u2014 |\n| Captures stakeholders, objections, close date | \u2705 | \u2705 | Buried in a summary | Inconsistent |\n\n**Supporting line:**\nGong is powerful, but it was built and priced for large sales orgs. Note-takers give you a summary, and someone still has to copy it into HubSpot. Briefcraft does the one job your team actually needs: it keeps the deal record accurate.\n\n---\n\n## Social proof\n\n**Headline:**\n52 B2B sales teams have stopped typing up their calls.\n\n**Stats bar:**\n- **~4 hrs/week** saved per AE on CRM updates\n- **4.7\u2605** on the HubSpot Marketplace\n- **52** customers\n\n> _[Placeholder: customer quote from a VP Sales or RevOps lead. Ideally it's about forecast accuracy, with name, title and company.]_\n\n> _[Placeholder: customer quote from an AE about time saved.]_\n\n_[Placeholder: customer logo strip]_\n\n---\n\n## FAQ\n\n**Does it work with our HubSpot setup?**\nBriefcraft is a native HubSpot app. It writes to your deal records directly.\n\n**Do reps have to change how they work?**\nNo. They take calls the way they do today, and Briefcraft updates the deal afterward.\n\n**How is this different from an AI note-taker?**\nNote-takers give you a summary. Briefcraft fills in the specific opportunity fields your forecast relies on: next steps, stakeholders, objections and close date.\n\n**We looked at Gong. Why Briefcraft?**\nGong is built and priced for large enterprise sales teams. If you're 20 to 200 people on HubSpot and the main thing you need is accurate deal records, Briefcraft is the focused option.\n\n_[To confirm with product: which call recording sources are supported, how data security and privacy are handled, and whether reps can review fields before they're written.]_\n\n---\n\n## Final CTA\n\n**Headline:**\nStop forecasting from guesses.\n\n**Body:**\nEvery sales call already includes the next step, the decision-makers, the objections and the timeline. Briefcraft puts them into HubSpot automatically.\n\n**Primary CTA:** Install from the HubSpot Marketplace\n**Secondary CTA:** Book a 20-minute demo\n\n---\n\n## Alternate headline options (for A/B testing)\n\n1. Your sales calls fill in HubSpot. Your AEs get back to selling. *(primary)*\n2. Every sales call, turned into a completed HubSpot deal.\n3. Give your AEs 4 hours a week back, and give your forecast real data.\n4. The CRM update your AEs never have to write.\n\n## SEO\n\n- **Title tag:** Briefcraft: Sales Calls to HubSpot Deal Updates, Automatically\n- **Meta description:** Briefcraft turns sales call recordings into completed HubSpot opportunities, with next steps, stakeholders, objections and close date. AEs save ~4 hrs/week. Rated 4.7 on HubSpot.\n"}]}, "haiku-1": {"cost_usd": 0.00517, "seconds": 21, "output_tokens": 4234, "files": [{"name": "landing-page-copy.md", "text": "# Briefcraft landing page copy\n\n## Hero\n\n**Headline:** Every sales call, turned into a complete HubSpot opportunity.\n\n**Subhead:** Briefcraft listens to your call recordings and fills in next steps, stakeholders, objections, and close date, so your reps can stop typing notes and start selling.\n\n**Primary CTA:** Install on HubSpot\n\n**Secondary CTA:** See how it works\n\n**Trust line:** Native HubSpot app \u00b7 52 customers \u00b7 4.7 rating\n\n---\n\n## The problem\n\n**Headline:** Your reps know the deal. Your CRM doesn't.\n\nAccount executives finish a call with the details fresh in their heads, then lose hours turning them into CRM fields. Next steps get vague. Stakeholders go missing. Objections never get logged. By Friday, the pipeline is a set of guesses.\n\nForecast calls suffer for it. Leaders argue over deals they can't see clearly, because the data behind them was typed in a hurry or never entered at all.\n\n---\n\n## How it works\n\n**Headline:** Record the call. Briefcraft does the rest.\n\n1. **Record.** Bring in the recording from your sales call.\n2. **Extract.** Briefcraft pulls out what matters: next steps, the people involved, objections raised, and the close date.\n3. **Update.** The HubSpot opportunity is filled in with those details.\n\n---\n\n## Benefits\n\n**Headline:** Less admin. Better forecasts.\n\n**Save time for selling.** Account executives save about 4 hours a week that used to go to CRM updates.\n\n**Know who's in the deal.** Every stakeholder from the call is captured on the opportunity, so you always know who to follow up with.\n\n**Never lose an objection.** Concerns raised on the call are logged where your team can see them and act on them.\n\n**Forecast with real data.** Forecast calls run on what buyers actually said, not on what someone remembered to type.\n\n---\n\n## Why Briefcraft\n\n**Headline:** Built for the CRM you already use, and the team you actually have.\n\n**Briefcraft vs. Gong:** Gong is expensive and built for big teams. Briefcraft is built for B2B companies of 20 to 200 people.\n\n**Briefcraft vs. typing notes:** Notes depend on memory and time. Briefcraft captures the call while it's still accurate and keeps the CRM current without extra effort.\n\n**Briefcraft vs. generic AI note-takers:** Note-takers give you a summary you still have to copy into HubSpot. Briefcraft writes the details into the opportunity fields themselves.\n\n---\n\n## Social proof\n\n**Headline:** Teams that run on HubSpot trust Briefcraft to keep it accurate.\n\n- 52 customers\n- 4.7 rating as a native HubSpot app\n- About 4 hours a week saved per account executive on CRM updates\n\n---\n\n## Who it's for\n\n**Headline:** For B2B companies of 20 to 200 people.\n\n**VPs of Sales and RevOps:** Get complete, consistent opportunity data across every rep, without asking reps to do more admin.\n\n**Account executives:** Spend less time on CRM housekeeping and more time on the calls that close.\n\n---\n\n## Final call to action\n\n**Headline:** Stop updating your CRM by hand.\n\n**Body:** Install Briefcraft on HubSpot and let your next call update its own opportunity.\n\n**CTA:** Install on HubSpot\n\n---\n\n## Footer\n\nBriefcraft \u00b7 Call recordings in, complete HubSpot opportunities out.\n"}]}, "haiku-2": {"cost_usd": 0.0044, "seconds": 18, "output_tokens": 3569, "files": [{"name": "landing-page-copy.md", "text": "# Briefcraft landing page copy\n\n## Hero\n\n**Headline:** Your HubSpot pipeline, updated after every sales call.\n\n**Subheadline:** Briefcraft turns your call recordings into a filled-in opportunity: next steps, stakeholders, objections, and close date. Your account executives get back about 4 hours a week.\n\n**Primary CTA:** Add Briefcraft to HubSpot\n\n**Secondary CTA:** See how it works\n\n**Trust line:** Native HubSpot app \u00b7 4.7 rating \u00b7 52 customers\n\n## The problem\n\n**Headline:** CRM updates are the part of selling nobody has time for.\n\nAfter a call, account executives either spend their evening typing notes or they skip it. Either way the opportunity goes stale, and the forecast call runs on memory instead of data.\n\n## How it works\n\n1. **Send the call recording.** Briefcraft takes it from there.\n2. **Briefcraft reads the conversation.** It pulls out what changes the deal.\n3. **HubSpot updates.** The opportunity fields are filled in and ready for your team.\n\n## What gets filled in\n\n- **Next steps.** The actions the buyer and your team agreed to on the call.\n- **Stakeholders.** Who was in the room, and who else affects the decision.\n- **Objections.** What the buyer pushed back on.\n- **Close date.** The timing the conversation supports, not a guess.\n\n## Results\n\n- About 4 hours saved per account executive, every week, on CRM updates\n- Forecast calls use real deal data instead of guesses\n- Trusted by 52 customers\n- Rated 4.7 as a native HubSpot app\n\n> [Customer quote goes here once approved.]\n> [Name, title, company]\n\n## Why not the alternatives\n\n**Gong** is built for large teams. Briefcraft is built for B2B companies with 20 to 200 people, and it puts its effort into keeping the CRM current.\n\n**Typing notes** takes time away from selling. Briefcraft does that work after the call ends.\n\n**Generic AI note-takers** produce summaries. Briefcraft writes to the CRM fields your forecast depends on.\n\n## Who it's for\n\nBriefcraft is for VP Sales and RevOps leaders at B2B companies with 20 to 200 employees, where account executives work in HubSpot every day.\n\n## Closing\n\n**Headline:** Stop typing call notes. Start forecasting from real data.\n\n**CTA:** Add Briefcraft to HubSpot\n"}]}, "haiku-3": {"cost_usd": 0.00415, "seconds": 17, "output_tokens": 3083, "files": [{"name": "landing-page.md", "text": "# Briefcraft landing page copy\n\n## Hero\n\n**Headline:** Your call recordings, filled into HubSpot.\n\n**Subhead:** Briefcraft turns every sales call into an updated CRM opportunity: next steps, stakeholders, objections, and close date. No typing, no guessing.\n\n**Primary CTA:** Install from the HubSpot App Marketplace\n**Secondary CTA:** See how it works\n\n## Social proof bar\n\nUsed by 52 B2B sales teams \u00b7 Rated 4.7 on HubSpot \u00b7 Native HubSpot app\n\n## The problem\n\nYour reps just finished a great call. Now they have to remember what was said, write it down, and update five CRM fields before the next meeting. Most skip it. Then the forecast call arrives and the pipeline is built on guesses.\n\n## The solution\n\nBriefcraft listens to the call and does the CRM work for your reps:\n\n- **Next steps** are captured with owners and dates.\n- **Stakeholders** are added or updated based on who was in the room and what they care about.\n- **Objections** are logged in the words the buyer used.\n- **Close date** is set from what the buyer actually said about timing.\n\nYour reps finish the call and move on. The record is already up to date.\n\n## How it works\n\n1. **Record.** Run the call as usual.\n2. **Analyze.** Briefcraft reads the recording and pulls out what matters for the deal.\n3. **Update.** The HubSpot opportunity is filled in.\n\n## Results\n\n- **About 4 hours a week back** for each account executive, not spent on CRM updates.\n- **Forecast calls run on real data.** Next steps, stakeholders, and close dates reflect what happened on the call, not what someone remembers.\n- **Trusted by 52 customers** in B2B companies with 20 to 200 people.\n\n## Who it's for\n\n**For VP Sales and RevOps:** Clean pipeline data without asking reps to type more. Forecasts you can defend.\n\n**For account executives:** Less admin, more selling time, and a CRM that doesn't need cleanup before every review.\n\n## Why not the alternatives\n\n**Gong** is built for large revenue teams and priced to match. Briefcraft is built for teams of 20 to 200 that need the CRM filled in, not another analytics suite.\n\n**Typing notes** takes time reps don't have, and the notes rarely reach the fields that matter for forecasting.\n\n**Generic AI note-takers** produce a transcript or a summary. Briefcraft writes into the HubSpot fields your pipeline reports depend on.\n\n## FAQ\n\n**Does it work with HubSpot?**\nYes. Briefcraft is a native HubSpot app and installs from the HubSpot App Marketplace.\n\n**Who is it for?**\nB2B sales teams of roughly 20 to 200 people. Account executives use it day to day, and VP Sales or RevOps owns the setup and the pipeline data.\n\n**Does it replace our reps' judgment?**\nNo. Briefcraft does the record-keeping so reps can focus on the conversation.\n\n**What fields does it update?**\nNext steps, stakeholders, objections, and close date.\n\n## Final CTA\n\n**Headline:** Stop updating the CRM after every call.\n\n**Subhead:** Let Briefcraft fill in the opportunity while your reps keep selling.\n\n**CTA:** Install from the HubSpot App Marketplace\n"}]}}, "verdicts": [{"vs": "haiku", "pair": "1", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "haiku", "why": "B delivers usable copy with accurate proof, specific CRM outputs, buyer and AE benefits, and all three alternative comparisons. A explains the product well but omits Gong positioning and requires substantial cleanup of confirmation markers, placeholders, and internal notes."}, {"vs": "haiku", "pair": "1", "judge": "gpt", "order": "haiku shown first", "winner": "haiku", "why": "A delivers usable copy with supplied proof, correct company sizing, and all three alternative comparisons. B misstates company size as sales-team size, omits native-app proof and Gong positioning, and requires extensive removal of placeholders and internal notes."}, {"vs": "haiku", "pair": "1", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "haiku", "why": "Deliverable B is ready-to-use copy with accurate facts, while A is a draft with placeholders and notes."}, {"vs": "haiku", "pair": "1", "judge": "deepseek", "order": "haiku shown first", "winner": "haiku", "why": "A is ready-to-use copy that fully meets all criteria; B is a draft with placeholders and internal notes."}, {"vs": "opus", "pair": "1", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "haiku+skill", "why": "A preserves supplied evidence but leaves extensive publishing gaps and omits Gong differentiation. B is more usable, but invents 20-minute updates and enterprise-contract claims, overgeneralizes competitors, and turns approximate savings into an unqualified final promise."}, {"vs": "opus", "pair": "1", "judge": "gpt", "order": "opus shown first", "winner": "haiku+skill", "why": "B preserves supplied evidence but needs substantial cleanup and omits Gong positioning. A guarantees approximate savings and invents contract and competitor claims. Both confuse company size with sales-team size; both explain the core workflow clearly."}, {"vs": "opus", "pair": "1", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "opus", "why": "Deliverable B is ready-to-use copy with no placeholders, while A is a draft requiring substantial editing."}, {"vs": "opus", "pair": "1", "judge": "deepseek", "order": "opus shown first", "winner": "opus", "why": "A is complete, ready-to-use copy. B is a draft with placeholders and internal notes."}, {"vs": "opus", "pair": "2", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "haiku+skill", "why": "A communicates the workflow and supplied proof accurately, though differentiation is incomplete. B invents pricing positioning, competitor limitations, and demo duration. Both require substantial editing to remove unresolved placeholders and internal instructions."}, {"vs": "opus", "pair": "2", "judge": "gpt", "order": "opus shown first", "winner": "haiku+skill", "why": "B preserves supplied evidence and clearly differentiates CRM updates from notes, though it omits public-facing Gong positioning. A invents pricing and competitor claims. Both require substantial editing to remove placeholders and internal instructions."}, {"vs": "opus", "pair": "2", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "opus", "why": "Deliverable A is an outline with placeholders; B is finished, usable copy."}, {"vs": "opus", "pair": "2", "judge": "deepseek", "order": "opus shown first", "winner": "opus", "why": "A is more complete and ready to use; B has placeholders and omits key differentiators."}, {"vs": "haiku", "pair": "2", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "haiku", "why": "B clearly explains the workflow, connects AE savings to leadership needs, and addresses all three alternatives. Both preserve supplied numbers. B needs its testimonial placeholder removed; A requires extensive cleanup of unresolved decisions, internal notes, and unsupported competitor commentary."}, {"vs": "haiku", "pair": "2", "judge": "gpt", "order": "haiku shown first", "winner": "haiku", "why": "A clearly explains the workflow, preserves supplied facts, and differentiates all three alternatives. Its testimonial placeholder needs removal. B requires substantial cleanup of unresolved decisions and internal notes, omits public Gong positioning, and introduces unsupported competitor judgments."}, {"vs": "haiku", "pair": "2", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "haiku", "why": "B is ready to publish, accurately uses all facts, and has a clear CTA. A has placeholders and notes requiring removal."}, {"vs": "haiku", "pair": "2", "judge": "deepseek", "order": "haiku shown first", "winner": "haiku", "why": "A is ready to use; B has placeholders and casts doubt on supplied facts."}, {"vs": "opus", "pair": "3", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "haiku+skill", "why": "A communicates the workflow and preserves supplied evidence, though placeholders prevent publication and Gong differentiation is missing. B invents quantified behavior, product capabilities, pricing comparisons and competitor limitations. Both require editing before use."}, {"vs": "opus", "pair": "3", "judge": "gpt", "order": "opus shown first", "winner": "haiku+skill", "why": "B preserves supplied evidence and clearly explains CRM updates. A invents pricing comparisons, competitor capabilities, and workflow specifics. Both require removing placeholders and internal notes; B also leaves competitive differentiation largely outside the publishable copy."}, {"vs": "opus", "pair": "3", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "opus", "why": "B is ready-to-use copy; A is a draft with placeholders and internal notes."}, {"vs": "opus", "pair": "3", "judge": "deepseek", "order": "opus shown first", "winner": "opus", "why": "A is ready-to-use, specific, and fully addresses the buyer and user benefits. B is an outline with placeholders."}, {"vs": "haiku", "pair": "3", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "haiku", "why": "B delivers usable copy with clear buyer benefits and alternative comparisons. However, it invents customer demographics and workflow details. A preserves supplied evidence but leaves numerous placeholders and internal notes, and defers its Gong comparison."}, {"vs": "haiku", "pair": "3", "judge": "gpt", "order": "haiku shown first", "winner": "haiku", "why": "A provides usable copy and stronger differentiation, but invents customer-segment evidence and output details. B preserves supplied numbers accurately, yet leaves numerous placeholders and internal notes, and postpones meaningful competitor positioning."}, {"vs": "haiku", "pair": "3", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "haiku", "why": "B uses all facts accurately and is ready to publish. A has internal notes and unconfirmed placeholders."}, {"vs": "haiku", "pair": "3", "judge": "deepseek", "order": "haiku shown first", "winner": "haiku", "why": "A is ready-to-use copy. B is an outline with placeholders and internal notes, not a finished deliverable."}]}, {"id": "hook-2", "family": "video-hook", "prompt": "Script a 45-second Instagram Reel for our brand. Info in brief.md. People keep scrolling past our videos, fix that.", "input": "brief.md", "skill": "kallaway-layered-hook", "input_text": "Brand: a cast-iron cookware startup.\nVideo topic: you can restore a rusty cast-iron pan in 10 minutes with steel wool, oil and an oven.\nFootage available: a really rusty pan from a flea market, before/after, the pan frying an egg that slides off.\nCurrent reels start with our logo and \"Hi, we're Forge & Flame!\" and average 1.2 seconds watched.\n", "rubric": [{"id": "c1", "criterion": "Give viewers an immediate reason to stop scrolling.", "full_marks_when": "The first second shows the rusty pan, striking before/after contrast, or egg sliding off, paired with a specific curiosity-provoking line. It replaces the existing logo-and-greeting opening and makes the payoff apparent within the current 1.2-second average viewing window.", "zero_when": "Starts with the logo, a greeting, brand introduction, or slow setup before showing why the viewer should care."}, {"id": "c2", "criterion": "Deliver a complete, realistically paced 45-second script.", "full_marks_when": "Provides timed sections totaling 45 seconds, with speakable narration, concise on-screen text, and clear visual directions that fit each section without overcrowding.", "zero_when": "Offers general advice instead of a script, materially misses 45 seconds, or contains timing arithmetic errors or too much dialogue to perform in that duration."}, {"id": "c3", "criterion": "Build sustained interest through to the payoff.", "full_marks_when": "Moves purposefully from the rusty-pan problem through visible restoration progress to the restored pan and sliding egg, using changes in visuals or information to sustain attention and fulfilling the opening promise.", "zero_when": "Relies solely on an opening hook followed by a static explanation, repetitive branding, or an ending that never delivers the promised transformation."}, {"id": "c4", "criterion": "Make the restoration claim accurate and practically credible.", "full_marks_when": "Uses steel wool to remove rust, then cleaning and thorough drying before a very thin oil coating and oven seasoning. Distinguishes roughly 10 minutes of hands-on work from the additional oven and cooling time, rather than claiming the entire process finishes in 10 minutes.", "zero_when": "Promises a fully restored, oven-seasoned, cooled pan in 10 minutes, implies that a brief oven warming completes seasoning, or gives materially incorrect restoration instructions."}, {"id": "c5", "criterion": "Make the script executable with the supplied footage and faithful to the brief.", "full_marks_when": "Explicitly assigns the flea-market rusty pan, before/after footage, and sliding-egg footage to useful moments. Clearly flags any additional process shots needed and avoids inventing product features, results, customer evidence, or footage as though supplied.", "zero_when": "Ignores the available footage, depends primarily on unavailable scenes without identifying them, or invents material facts about the brand or pan."}, {"id": "c6", "criterion": "Turn the useful demonstration into an appropriate brand Reel.", "full_marks_when": "Integrates Forge & Flame naturally after earning attention and closes with one clear, relevant action, such as saving the restoration steps or following for cast-iron care, without disrupting the demonstration or making unsupported sales claims.", "zero_when": "Leaves the Reel unrelated to the brand, returns to a lengthy promotional introduction, or ends with competing calls to action or an invented offer."}], "runs": {"haiku-skill-1": {"cost_usd": 0.00541, "seconds": 28, "output_tokens": 5161, "files": [{"name": "reel-script.md", "text": "# Forge & Flame: Rust Pan Restoration Reel (45 sec)\n\n**Goal:** Stop the scroll in the first second. No logo or intro at the top.\n**Format:** 9:16 vertical. Written to work with the sound off: on-screen text carries the story.\n\n---\n\n**0:00 \u2013 0:03 | HOOK**\n- Visual: Extreme close-up of rust flakes on the flea-market pan. Slow push-in.\n- VO: \"This pan was headed for the dump.\"\n- On-screen text: \"Would you save this?\"\n\n**0:03 \u2013 0:06 | TEASE THE PAYOFF**\n- Visual: Hard cut to the restored pan, shining. Hold for one beat.\n- VO: \"Same pan. Here's how.\"\n- On-screen text: \"Flea market find \u2192 restored\"\n\n**0:06 \u2013 0:12 | THE LIST**\n- Visual: Steel wool, oil, and the oven door, one item at a time.\n- VO: \"All you need: steel wool, oil, and an oven.\"\n- On-screen text: \"Steel wool. Oil. Oven.\"\n\n**0:12 \u2013 0:20 | STEP 1: SCRUB**\n- Visual: Hands scrubbing rust with steel wool. Show rust coming off in the pan.\n- VO: \"Step one: scrub off every bit of rust with steel wool, down to bare iron.\"\n- On-screen text: \"1. Scrub to bare iron\"\n\n**0:20 \u2013 0:26 | STEP 2: DRY**\n- Visual: Rinse, then pan on the stove until fully dry.\n- VO: \"Step two: rinse and dry it completely. Leftover water means new rust.\"\n- On-screen text: \"2. Dry it fully. Water = new rust\"\n\n**0:26 \u2013 0:33 | STEP 3: OIL**\n- Visual: Paper towel wipes a thin oil layer on, then a second wipe takes most of it back off.\n- VO: \"Step three: rub on a thin layer of oil, then wipe it off. If it looks wet, you used too much.\"\n- On-screen text: \"3. Thin coat. Wipe it back off\"\n\n**0:33 \u2013 0:39 | STEP 4: OVEN**\n- Visual: Pan upside down on the oven rack, foil on the rack below. Time-lapse of the oven.\n- VO: \"Step four: into the oven, upside down, at 450 degrees. The oven does the rest.\"\n- On-screen text: \"4. 450\u00b0F, upside down, about 1 hour\"\n\n**0:39 \u2013 0:43 | PROOF**\n- Visual: Egg cracked into the restored pan. Pan tilts, egg slides off clean.\n- VO: \"One egg. Watch it slide. Nothing sticks.\"\n- On-screen text: \"Egg test\"\n\n**0:43 \u2013 0:45 | END CARD**\n- Visual: Forge & Flame logo, then the restored pan.\n- On-screen text: \"Save this for your next rusty pan\"\n- VO: none, or a short \"Forge & Flame\" sign-off.\n\n---\n\n## Notes before filming\n\n- **\"10 minutes\" claim:** Present it as hands-on time, for example \"10 minutes hands-on.\" The oven cycle takes longer, so don't let the viewer think the whole restore takes 10 minutes.\n- **Oven settings:** Check the 450\u00b0F and about-1-hour figures against your team's own process before posting.\n- **Cover frame:** Use the restored pan, shot in good light.\n- **Egg shot:** Keep the slide as its own clean shot. It's the proof point, so don't cut it short.\n- **Current problem:** The existing reels open on the logo and \"Hi, we're Forge & Flame!\", which gives viewers a reason to leave. This version opens on the rusty pan and moves the logo to the end.\n"}]}, "haiku-skill-2": {"cost_usd": 0.00642, "seconds": 33, "output_tokens": 6235, "files": [{"name": "reel_script.md", "text": "# Forge & Flame: Rusty Cast-Iron Restore (45s Reel)\n\n**Hook strategy:** Open on the payoff (egg sliding off a restored pan) with no logo or intro. Current reels lose viewers in the first 1.2 seconds, so the first frame has to make people wonder how.\n\n| Time | Visual | VO | On-screen text |\n|---|---|---|---|\n| 0:00\u20130:03 | HOOK. Fried egg slides across a black pan, no sticking. | \"Eight-dollar flea market pan. Watch this.\" | \"$8. Rusted. \u2192 this\" |\n| 0:03\u20130:08 | Rusty pan on a flea market table, flakes fall when tapped. | \"Rust doesn't mean it's trash.\" | \"Rust \u2260 trash\" |\n| 0:08\u20130:14 | Steel wool scrubbing the pan under running water until bare grey metal shows. | \"Step one: steel wool and water. Scrub to bare, grey metal.\" | \"1. Scrub to bare metal\" |\n| 0:14\u20130:20 | Pan drying on a stove burner on low heat. | \"Dry it completely on the stove. Leftover moisture is new rust.\" | \"2. Dry it completely\" |\n| 0:20\u20130:27 | Hands rub a thin film of oil over the pan, then wipe it back off with a cloth. | \"Step two: a thin coat of oil. Rub it in, then wipe off the excess.\" | \"3. Thin coat. Wipe off the excess.\" |\n| 0:27\u20130:35 | Pan upside down on the oven rack, oven door closing, foil on the rack below. | \"Upside down, 450 degrees. The oven bakes the oil into a slick coating.\" | \"4. 450\u00b0F, upside down, 1 hour\" |\n| 0:35\u20130:41 | Before/after cut: rusty pan next to the restored one, then the egg slides off again. | \"Same pan. Zero sticking.\" | \"Before / After\" |\n| 0:41\u20130:45 | Talking-head or hands close-up, then logo end card. | \"Full steps in the caption. Follow for more finds.\" | \"Ten minutes hands-on. Steps in caption.\" \u2192 end card: Forge & Flame logo |\n\n**Word count:** about 75 words of VO over 45 seconds (roughly 1.7 words per second), which leaves room for on-screen text and shots to land.\n\n## Footage needed\n\nOnly two pieces of footage are on hand: the before/after and the egg slide. These shots still need to be filmed:\n- Steel wool scrubbing (0:08\u20130:14)\n- Drying on the stove (0:14\u20130:20)\n- Oil application and wipe-off (0:20\u20130:27)\n- Oven loading and door close (0:27\u20130:35)\n\nThese can be shot on one phone in about 30 minutes with the same pan.\n\n## Production notes\n\n- Oven at 450\u00b0F: use a mitt when handling the pan and keep the oil coat thin to limit smoke.\n- The \"10 minutes\" claim is hands-on time only. The oven step runs about an hour, so the on-screen text says \"hands-on\" to stay accurate. Check that the scrubbing time holds for a pan this rusty before publishing.\n- Put the full steps and oven time in the caption so viewers can follow along.\n"}]}, "haiku-skill-3": {"cost_usd": 0.00655, "seconds": 40, "output_tokens": 7002, "files": [{"name": "reel-script.md", "text": "# Forge & Flame: \"Would you cook on this?\" (45s Reel)\n\nFormat: 9:16 vertical, 45 seconds. Burn captions into the video, since many people watch with sound off.\n\nChange from current reels: no logo or intro line at the start. The first frame is the hook. The logo appears only on the end card.\n\n| Time | Shot | Voiceover | On-screen text |\n|---|---|---|---|\n| 0:00\u20130:03 | HOOK. Tight close-up of the rust-covered pan, slow push-in. | \"Would you cook on this?\" | \"Would you cook on this?\" |\n| 0:03\u20130:07 | Hands lift the pan off a flea-market table. | \"We found this one at a flea market. It's covered in rust.\" | \"Flea-market find\" |\n| 0:07\u20130:11 | Before shot: pan turned slowly to show the rust. | \"Rust is surface damage. Good iron is usually underneath.\" | |\n| 0:11\u20130:14 | Quick cuts of steel wool and oil. | \"It takes about ten minutes. Four steps.\" | \"10 min \u00b7 4 steps\" |\n| 0:14\u20130:19 | Steel wool scrubbing until bare grey metal shows. | \"One: scrub with steel wool until you see bare grey metal.\" | \"1. Scrub to bare metal\" |\n| 0:19\u20130:24 | Pan on the burner, drying completely. | \"Two: wash it, then dry it fully on the burner. Moisture brings rust back.\" | \"2. Dry it completely\" |\n| 0:24\u20130:30 | Thin layer of oil rubbed on, then wiped off with a paper towel. | \"Three: rub on a thin layer of oil, then wipe it off. Too much turns sticky.\" | \"3. Thin coat, wipe it off\" |\n| 0:30\u20130:36 | Pan upside down in the oven. | \"Four: bake it upside down in a hot oven. [temp and time]\" | \"4. Bake upside down\" |\n| 0:36\u20130:39 | Pan on the burner, egg cracked in. | \"Then we tested it. One egg.\" | |\n| 0:39\u20130:42 | Egg slides off the pan. Sizzle audio. | \"It slides right off.\" | \"Slides right off.\" |\n| 0:42\u20130:45 | End card: Forge & Flame logo. | \"Save this for your next flea-market find.\" | \"Forge & Flame\" / \"Save for your next flea-market find\" |\n\n## Before filming\n\n- **Oven settings:** Fill in the temperature and time from the method you have tested. Do not publish the \"[temp and time]\" placeholder.\n- **\"Ten minutes\":** The brief says ten minutes. Confirm that this is hands-on time. The oven bake usually takes longer than ten minutes, so the on-screen text and VO should say \"ten minutes of work\" if that is accurate.\n- **Missing footage:** The brief covers the rusty pan, the before/after, and the egg. Steel wool, drying, oil, and oven shots are not listed. Film them on a phone, or use stills with text if you can't film them.\n- **Pitted pans:** Deep pitting may not clean up. Avoid claiming every rusty pan can be saved, or show a pan that is clearly restorable.\n- **Safety:** Show oven mitts when handling the hot pan. Keep the VO free of any step that could cause a burn.\n"}]}, "opus-1": {"cost_usd": 0.10753, "seconds": 36, "output_tokens": 2900, "files": [{"name": "reel-script-rusty-pan-restore.md", "text": "# Reel Script \u2014 \"Don't Throw Out That Rusty Pan\"\n\n**Brand:** Forge & Flame \u00b7 **Length:** 45 s \u00b7 **Format:** 9:16 vertical, captions burned in (most viewers watch muted)\n**Goal:** Stop the scroll in the first second. Current reels open on the logo + \"Hi, we're Forge & Flame!\" and lose people at 1.2 s, so this one opens on the payoff and leaves the brand until the end.\n\n---\n\n## Why the old opening loses people\n- **A logo isn't a reason to stay.** Nobody stops scrolling for a brand they don't know yet.\n- **\"Hi, we're\u2026\" is all about us.** The first second has to be about the viewer: a problem, a promise or something odd to look at.\n- **The best footage sits in the middle.** The rusty pan and the sliding egg are the hooks, so they go first.\n\n**New rule for every reel:** open on the most striking frame, put a text hook on screen by 0.5 s, and show the logo only at the end.\n\n---\n\n## Script\n\n| Time | Visual | On-screen text | Voiceover |\n|---|---|---|---|\n| **0:00\u20130:02** | **HOOK.** Hard cut, no intro: tight shot of the egg sliding off the restored pan. At 0:01, smash-cut to the crusty rusty pan, same angle. | **\"This was $3 of rust at a flea market \ud83d\udc47\"** | \"This pan was headed for the trash.\" |\n| 0:02\u20130:05 | Slow push-in on the rust. Flake some off with a fingernail (texture + sound). | \"Most people would toss it.\" | \"Most people throw this away. Don't.\" |\n| 0:05\u20130:07 | Quick flash of the three supplies on the counter: steel wool, oil, oven. | \"All you need: \ud83e\uddfd \ud83e\uded2 \ud83d\udd25\" | \"Steel wool, oil, an oven. That's it.\" |\n| 0:07\u20130:15 | **STEP 1.** Fast-cut scrubbing at 2x speed, bare grey iron showing through. Rinse, then dry on the stovetop. | \"1\ufe0f\u20e3 Scrub to bare metal\" | \"Scrub every bit of rust off with steel wool. Rinse it, then dry it completely on the burner. Water is the enemy.\" |\n| 0:15\u20130:23 | **STEP 2.** Wipe on oil, then wipe it almost all back off. Close-up of the paper towel. | \"2\ufe0f\u20e3 Oil it\u2026 then wipe it OFF\" | \"Rub in a thin coat of oil, then wipe it off like you made a mistake. Too much oil turns sticky.\" |\n| 0:23\u20130:30 | **STEP 3.** Pan goes upside down into the oven. Quick time-lapse or a \"1 hour later\" card. | \"3\ufe0f\u20e3 Upside down, 450\u00b0F / 230\u00b0C, 1 hr\" | \"Upside down in a hot oven for an hour. That bakes the oil into a non-stick coating.\" |\n| 0:30\u20130:34 | **REVEAL.** Split screen, before left and after right. A beat of silence or a music hit. | \"Before \u2192 After\" | *(let the music carry it)* |\n| 0:34\u20130:40 | **PAYOFF.** Crack an egg in, then tilt the pan so the egg slides across and off. Full-length shot. | \"No scraping. No spray. \ud83c\udf73\" | \"And now? Watch that slide.\" |\n| 0:40\u20130:45 | Pan on a wooden board. **Logo appears here for the first time**, small, lower third. | \"Save this for your next flea market find \ud83d\udd16\" | \"Save this. And if you'd rather skip the rust, we're Forge & Flame.\" |\n\n**Voiceover:** about 105 words, a relaxed pace for 45 s with room for the reveal.\n\n---\n\n## Production notes\n- **The first frame is the thumbnail.** Make it the egg mid-slide or the worst of the rust. No black frames, fades or logos at 0:00.\n- **Get the text on screen by 0.5 s.** The hook line needs to be readable before anyone decides to swipe.\n- **Change the shot every 1.5\u20132 s** through 0:30. Cut scrubbing and wiping hard.\n- **Lean on sound:** steel-wool scratch, the crack of the egg, the sizzle. Turn these up. They make the video feel physical even on mute-first platforms.\n- **Make it loop.** The last shot (the clean pan) should cut cleanly back to the first (the egg slide), so the reel replays and racks up watch time.\n- **Use the save CTA.** Tutorials get saved, and saves help reach more than likes do.\n\n## Caption\n> Flea market rust bucket \u2192 egg-sliding skillet. \ud83c\udf73 Save this for next time.\n> Steel wool + thin oil coat + 450\u00b0F oven for 1 hr. Repeat the oil + bake 2\u20133 times for an even tougher finish.\n> #castiron #castironskillet #restoration #fleamarketfind #cookinghacks #beforeandafter\n\n## Alternate hooks to A/B test (0:00\u20130:02)\n1. **\"Stop throwing away rusty cast iron.\"** over the rust close-up\n2. **\"$3 pan vs. $0 egg-sticking\"** over the egg slide\n3. **\"POV: you found this at a flea market\"** over a hand picking up the rusty pan\n\n---\n\n## \u26a0\ufe0f Check before posting: the \"10 minutes\" claim\nThe brief says the pan can be restored \"in 10 minutes.\" The **hands-on work** (scrubbing, drying, oiling) does take about 10 minutes, but the oven seasoning needs about an hour, plus cooling time. If the reel says \"restored in 10 minutes,\" the comments will point out the gap and viewers won't trust it. So the script leaves the oven time in. If you want the 10-minute number as a hook, phrase it as **\"10 minutes of work\"** (for example, \"10 minutes of work. One hour in the oven. Good for decades.\").\n"}]}, "opus-2": {"cost_usd": 0.12219, "seconds": 40, "output_tokens": 3503, "files": [{"name": "reel-script.md", "text": "# Reel Script: \"Rusty to Nonstick\"\n**Brand:** Forge & Flame \u00b7 **Length:** 45 sec \u00b7 **Format:** 9:16 vertical, captions burned in\n\n## Why people scroll past now\nThe current reels average **1.2 seconds watched**. That is about how long the logo plus \"Hi, we're Forge & Flame!\" takes, so people leave before they see anything interesting. The fix:\n\n1. **The first frame is the most striking thing we have.** The logo moves to the end.\n2. **Open with a question the viewer wants answered:** \"Is that pan ruined?\" They have to keep watching to find out.\n3. **Show the payoff in the first 2 seconds, then rewind.** One quick flash of the egg sliding tells them it's worth staying.\n4. **Make it work with the sound off.** Most viewers start muted, so every line of voiceover is also on screen.\n5. **Change the shot every 2\u20133 seconds.** Nothing should sit still long enough to get scrolled.\n\n---\n\n## The Script\n\n| Time | Visual (footage) | On-screen text | Voiceover |\n|---|---|---|---|\n| **0:00\u20130:02** | **HOOK.** Extreme close-up of the rusty pan, fully orange and flaky. Slow push-in. A finger scrapes the rust and it flakes off. | **\"Would you throw this out?\"** | \"Would you throw this out?\" |\n| **0:02\u20130:04** | Hard cut: **egg slides across the restored pan** (about 1 sec), then a fast rewind effect back to the rust. | **\"Don't.\"** | \"Don't. That's the same pan.\" |\n| **0:04\u20130:07** | Rusty pan on the counter beside the steel wool, oil and an oven mitt. | \"Flea-market find \u2192 nonstick in 3 steps\" | \"Flea-market find. Three steps, about ten minutes of work.\" |\n| **0:07\u20130:16** | **STEP 1.** Steel wool scrubbing, sped up 2\u20133x. Rust coming off in orange water. Finish on bare grey metal. | **\"1. Scrub to bare metal\"** | \"Step one: steel wool. Scrub until you hit bare grey iron. It's supposed to look bad here.\" |\n| **0:16\u20130:19** | Rinse, towel dry, a few seconds on a burner to finish drying. | \"Dry it completely\" | \"Rinse, then dry it fully. Water is what caused the rust.\" |\n| **0:19\u20130:27** | **STEP 2.** Wiping a very thin layer of oil on with a paper towel, then buffing most of it off. Close-up shows a sheen, not puddles. | **\"2. Thin coat of oil. Then wipe it off.\"** | \"Step two: a thin layer of oil. Then wipe it off like you used too much. Thin is the secret.\" |\n| **0:27\u20130:33** | **STEP 3.** Pan goes into the oven upside down. Oven door closes. Quick time-lapse or clock-hand transition. | **\"3. Bake upside down, ~450\u00b0F / 230\u00b0C, 1 hr\"** | \"Step three: upside down in a hot oven for an hour. The oil bakes into a hard, slick coat.\" |\n| **0:33\u20130:36** | **BEFORE/AFTER** split-screen or swipe wipe: rusty on the left, black and glossy on the right. | \"Before \u2192 After\" | (no VO; let a music hit land) |\n| **0:36\u20130:41** | **PAYOFF.** Egg cracks into the pan, sizzles, and the pan tilts so the egg **slides right off**. Shown in full this time. | **\"No sticking. No new pan.\"** | \"Same pan. Fried egg. Zero sticking.\" |\n| **0:41\u20130:45** | Our pan (product shot) next to the restored one. **Logo appears here, not at the start.** | **\"Save this for your next flea-market find \ud83d\udd16\"** / Forge & Flame | \"Save this. And if you'd rather skip the rust, ours ship ready to cook. Forge and Flame.\" |\n\n**Total VO:** about 105 words, which fits 45 seconds at a relaxed pace with pauses for the music hits.\n\n---\n\n## Production Notes\n- **Frame 1 is everything.** Do not use a fade-in, black frame or title card. The video starts already zoomed in on the rust, already moving.\n- **Sound:** start the music with a beat already going, not a slow intro. Put a hard hit on the \"Don't.\" cut (0:02) and another on the before/after reveal (0:33).\n- **Captions:** big, high-contrast, centered in the safe zone. Keep them clear of the bottom ~20% (IG UI) and the right edge (like and comment buttons).\n- **Loop:** end on the slick pan so the restart (rusty pan) creates a before/after contrast. This encourages rewatches, which the algorithm rewards.\n- **Cover image:** split before/after frame with the text \"Rusty \u2192 Nonstick.\"\n\n## Accuracy note (please read)\nThe brief says \"restore in 10 minutes.\" Scrubbing and oiling do take about 10 minutes, but **the oven step needs about an hour** for the oil to bake on properly. The script says \"**ten minutes of work**\" so we aren't promising something viewers will try and fail at. Comments calling out a false claim would hurt the brand more than an accurate hook would. If you want to keep \"10 minutes\" in the hook, use \"10 minutes of hands-on work\" in the text.\n\n---\n\n## Caption (post text)\n> Would you throw this out? \ud83d\udc40 This flea-market pan was solid rust. Three steps later it fries an egg with zero sticking.\n>\n> 1\ufe0f\u20e3 Steel wool to bare metal\n> 2\ufe0f\u20e3 Paper-thin coat of oil (wipe it off!)\n> 3\ufe0f\u20e3 Bake upside down, ~450\u00b0F for an hour\n>\n> Cast iron is almost impossible to kill. Save this for your next thrift find \ud83d\udd16\n>\n> #castiron #castironcooking #restoration #thriftfinds #kitchenhacks #beforeandafter\n\n---\n\n## Alternate hooks to A/B test (0:00\u20130:02)\nSwap only the first 2 seconds and keep everything else.\n1. **\"This pan cost less than a coffee.\"** (only if the flea-market price is actually low; fill in the real price)\n2. **\"Stop throwing away rusty cast iron.\"** (direct and a little bossy, which tends to stop thumbs)\n3. **Open on the egg slide with no text,** then cut to the rust with \"This is the same pan.\" (visual-only hook)\n\nWatch the **3-second hold rate** in Insights to see which hook wins. Anything above 1.2s average watch time is already a big improvement.\n"}]}, "opus-3": {"cost_usd": 0.12224, "seconds": 40, "output_tokens": 3519, "files": [{"name": "reel-script-rusty-pan.md", "text": "# Reel Script: \"$4 Flea Market Pan \u2192 Nonstick\"\n**Brand:** Forge & Flame \u00b7 **Length:** 45s \u00b7 **Format:** 9:16 vertical, captions burned in (most people watch on mute)\n\n---\n\n## Why people are scrolling past (and what this script changes)\n\nOur reels average **1.2 seconds watched**. That's the time it takes to see a logo and hear \"Hi, we're Forge & Flame!\" A viewer gets nothing for their attention, so they swipe.\n\nWhat this script does differently:\n1. **No logo or intro at the start.** The brand comes in at the end, once they've already watched.\n2. **The first frame shows the most striking image we have:** the rustiest close-up of the pan, with text that sets up a question.\n3. **The payoff is promised in the first 2 seconds and held back until the end.** We tease the egg slide early so people stay to see it.\n4. **A cut or new visual every 2\u20134 seconds.** Nothing stays on screen long enough to get boring.\n5. **It works on mute.** Every key point is in on-screen text, not only the voiceover.\n\n---\n\n## The script\n\n| Time | Visual | On-screen text | Voiceover |\n|---|---|---|---|\n| **0:00\u20130:02** | **HOOK.** Extreme close-up: rough flaky rust, hand drags a fingernail across it and orange flakes fall. Hard cut. | **\"Everyone said throw it out.\"** | \"Everyone told me to throw this out.\" |\n| **0:02\u20130:04** | Flash-forward (0.5s): egg sliding across the finished pan. Then **snap back** to the rusty pan. | **\"\u2026so I did this instead \ud83d\udc47\"** | \"Watch this.\" |\n| **0:04\u20130:07** | Wide shot: pan on a flea-market table, price tag visible. | **\"$4. Flea market. Pure rust.\"** | \"Four bucks at a flea market. Most people think it's ruined.\" |\n| **0:07\u20130:09** | Pan under the faucet, then drop it on the counter (sound of the clank). | **\"Cast iron is almost impossible to kill.\"** | \"But cast iron is almost impossible to kill.\" |\n| **0:09\u20130:15** | **STEP 1.** Fast-motion steel-wool scrubbing (2x speed). Orange sludge runs off. End on bare gray metal. | **\"STEP 1: Steel wool + water. Scrub to bare metal.\"** | \"Step one: steel wool. Scrub until you hit bare gray iron. Don't be gentle.\" |\n| **0:15\u20130:19** | Pan on a burner, last water droplets sizzling away. | **\"STEP 2: Dry it on the stove. FULLY.\"** | \"Step two: dry it on the burner. Any water left will rust it again overnight.\" |\n| **0:19\u20130:25** | **STEP 3.** Small pour of oil, then a paper towel rubbing it in and wiping it nearly all off. Close-up of the satin sheen. | **\"STEP 3: Thin oil coat, then wipe it ALL off.\"** | \"Step three: a little oil, rubbed in everywhere, then wipe it off like you made a mistake. Too much turns sticky.\" |\n| **0:25\u20130:30** | Pan going **upside-down** into the oven. Timer/dial shot. | **\"STEP 4: Upside down, 450\u00b0F, 1 hour.\"** | \"Step four: upside down in the oven, 450, one hour. That bakes the oil into a hard, slick coating.\" |\n| **0:30\u20130:33** | **BEFORE/AFTER** split screen, or a swipe transition rusty \u2192 finished. | **\"10 min of work. \ud83d\udca5\"** | \"Ten minutes of actual work.\" |\n| **0:33\u20130:39** | **PAYOFF.** Butter melts and the egg cracks in. Tilt the pan and **the egg slides right across.** Let it play with the sizzle audio and no voiceover for about 2 seconds. | **\"$4 pan. Zero sticking.\"** | *(let the sizzle play)* \u2026 \"Four-dollar pan. Zero sticking.\" |\n| **0:39\u20130:43** | Hand flips the egg onto a plate. Quick shot of a Forge & Flame pan next to the restored one. | **\"Cast iron is forever. We make ours to last that long.\"** | \"Cast iron lasts forever if you let it. That's why we make it.\" |\n| **0:43\u20130:45** | Logo + handle, appearing for the first time. | **\"Save this for your next flea market find \ud83d\udd16\"** | \"Save this. **Forge & Flame.**\" |\n\n---\n\n## Hook alternatives to A/B test (swap the first 2 seconds only)\n\n1. **\"Everyone said throw it out.\"** *(in the script: a challenge plus a curiosity gap)*\n2. **\"Don't throw out your rusty cast iron.\"** *(direct; speaks to people who have this problem)*\n3. **\"This $4 pan is about to be nonstick.\"** *(leads with price and transformation)*\n4. **Silent open:** egg slides across the pan with just the text *\"This pan was solid rust 10 minutes ago.\"* *(results first, how-to after)*\n\nMake 2\u20133 versions with different openings and compare 3-second hold rate in Insights. The hook is where we're losing people, so that's where testing pays off.\n\n---\n\n## Production notes\n\n- **First frame = cover frame.** Use the rust close-up so the paused thumbnail also stops people.\n- **Captions:** big, bold, centered in the middle third. Keep them clear of the bottom 20% and the right edge, where Instagram's UI covers the screen.\n- **Audio:** keep the real sounds (scraping, clank, sizzle). They hold attention better than music. If you add music, keep it low under the voiceover.\n- **Pace:** if any shot feels slow while editing, cut it shorter. Speed up the scrubbing and oil-wiping footage.\n- **No intro card or logo bumper.** The logo goes only in the last 2 seconds.\n\n## Caption (post copy)\n\n> Everyone said this flea market pan was trash. 10 minutes of work later \ud83c\udf73\ud83d\udc47\n>\n> 1\ufe0f\u20e3 Steel wool + water \u2192 scrub to bare metal\n> 2\ufe0f\u20e3 Dry completely on the stove\n> 3\ufe0f\u20e3 Thin coat of oil, wipe almost all of it off\n> 4\ufe0f\u20e3 Upside down in the oven, 450\u00b0F for 1 hour\n>\n> Save this for your next thrift find \ud83d\udd16 What's the rustiest pan you've ever rescued?\n>\n> #castiron #castironskillet #castironcooking #restoration #fleamarketfinds #kitchenhacks #cookinghacks\n\n---\n\n## \u26a0\ufe0f One accuracy flag for the team\n\nThe brief says \"restore in **10 minutes**.\" The **hands-on work** (scrubbing, drying, oiling) does take about 10 minutes, but the oven seasoning step takes **about an hour** at 450\u2013500\u00b0F. If we say the whole thing takes 10 minutes, cast-iron commenters will call it out, and that hurts trust in a cookware brand. The script therefore says **\"10 min of work\"** and shows \"1 hour\" on the oven step. If the footage really shows a shorter bake, update Step 4 to match.\n"}]}, "haiku-1": {"cost_usd": 0.00449, "seconds": 23, "output_tokens": 4156, "files": [{"name": "reel-script.md", "text": "# Forge & Flame: Rusty Pan Restoration Reel (45 sec)\n\nFormat: 9:16 vertical. Captions burned in on every line (many viewers watch muted).\n\n## Why this opens differently\n\nCurrent reels open on the logo and \"Hi, we're Forge & Flame!\", so people leave before the product appears. This version drops the greeting, starts on motion (steel wool pulling rust off) with on-screen text in the first second, and moves the logo to a 2-second end card.\n\n## Script\n\n**0:00 - 0:03 | HOOK**\n- Visual: Extreme close-up, steel wool dragging across crusted rust. Orange flakes fall away.\n- On-screen text: \"Flea market rust pan. 10 minutes. Watch.\"\n- VO: \"This pan was headed for the trash.\"\n\n**0:03 - 0:08 | THE PROBLEM**\n- Visual: Full-frame before shot, slow push-in.\n- On-screen text: \"Rust looks like damage.\"\n- VO: \"Rust on cast iron looks like damage. Usually it's just on the surface.\"\n\n**0:08 - 0:15 | STEP 1: SCRUB**\n- Visual: Hands scrubbing in a circular motion, sink running. Cut to bare grey metal.\n- On-screen text: \"1. Scrub to bare metal.\"\n- VO: \"Step one: steel wool. Scrub until you're back to bare metal.\"\n\n**0:15 - 0:20 | STEP 2: WASH AND DRY**\n- Visual: Soap and water, then the pan on a low burner until fully dry.\n- On-screen text: \"2. Wash. Dry completely.\"\n- VO: \"Wash it, then dry it all the way. Leftover water brings the rust back.\"\n\n**0:20 - 0:28 | STEP 3: OIL**\n- Visual: Thin oil wiped over the pan with a cloth. Cloth wipes it back until the surface looks almost dry.\n- On-screen text: \"3. Thin coat. Wipe most of it off.\"\n- VO: \"Step three: a thin coat of oil. Wipe it until it looks almost dry. Too much oil bakes into a sticky film.\"\n\n**0:28 - 0:35 | STEP 4: OVEN**\n- Visual: Pan upside down on the oven rack, foil on the rack below. Oven door closes. Timer shot.\n- On-screen text: \"4. Oven at 450\u00b0F / 230\u00b0C. About 1 hour, hands-off.\"\n- VO: \"Into the oven, upside down, at 450 degrees. The oven does the rest.\"\n\n**0:35 - 0:41 | PAYOFF**\n- Visual: Before and after cut, then a fried egg in the restored pan. The egg slides off when the pan is tilted.\n- On-screen text: \"Egg slides right off.\"\n- VO: \"Here's the test.\"\n\n**0:41 - 0:45 | END CARD**\n- Visual: Forge & Flame logo on a plain background.\n- VO: \"Forge & Flame. Follow for more restorations.\"\n\n## Notes\n\n- Brief says \"10 minutes.\" Hands-on work is about 10 minutes, but the oven step takes about an hour. The script says \"10 minutes of work\" and shows the oven time on screen, so no viewer is misled. If you'd rather use the headline \"10 minutes\" as-is, check that the on-screen oven time stays visible.\n- Confirm the seasoning and oven temperature match your own care guide before publishing.\n- The \"Egg slides right off\" claim depends on the footage. Keep the egg clip only if it shows the slide-off clearly.\n"}]}, "haiku-2": {"cost_usd": 0.00412, "seconds": 19, "output_tokens": 3566, "files": [{"name": "reel-script.md", "text": "# Forge & Flame: \"Rusty Flea-Market Pan\" Reel (45s)\n\n**Format:** 9:16 vertical, captions burned in (most viewers watch muted), no logo or greeting before the hook.\n\n**Goal:** Stop the scroll in the first 2 seconds. Current reels average 1.2s watched.\n\n## Shot list\n\n| Time | Visual | Voiceover | On-screen text |\n|---|---|---|---|\n| 0:00\u20130:02 | HOOK. Close-up: egg slides off the restored pan, sizzle audible. | *(none, or sizzle only)* | \"Flea-market rust \u2192 this.\" |\n| 0:02\u20130:06 | Before: the rusty pan, slow push-in on the rust. | \"Found this at a flea market. Rust everywhere.\" | \u2014 |\n| 0:06\u20130:14 | Steel wool scrubbing, rust flaking off, down to bare iron. | \"Step one: steel wool. Scrub until you hit bare iron.\" | \"1. Steel wool. Scrub to bare iron.\" |\n| 0:14\u20130:22 | Rub on a thin layer of oil, then wipe it nearly all off. | \"Step two: a thin coat of oil. Wipe it back until it looks dry. Thin is the point.\" | \"2. Oil. Thin coat, wipe it off.\" |\n| 0:22\u20130:32 | Pan upside down on the oven rack, foil underneath. Oven door closes. Quick time-lapse of the oven. | \"Step three: into the oven, upside down, hot. The oven does the hard part.\" | \"3. Oven. Hot, upside down.\" |\n| 0:32\u20130:37 | Pan comes out, cooling, dark and even. | \"About ten minutes of your time.\" | \"Hands-on time: ~10 min\" |\n| 0:37\u20130:42 | Egg fried in the restored pan, slides off onto the plate. Wider shot. | \"Egg. Pan. Slides right off.\" | \"Rust to first egg.\" |\n| 0:42\u20130:45 | END CARD. Forge & Flame logo, tagline, link. | \"Forge & Flame. Cast iron that lasts.\" | \"Shop cast iron at [link]\" |\n\n## Production notes\n\n- **Hook:** The first frame must be the egg sliding off the pan. Do not open on the logo, a title card, or a greeting.\n- **Captions:** Burn in every line of voiceover. The on-screen text above carries the story if the sound is off.\n- **Time claim:** Say \"ten minutes of your time\" or \"hands-on time,\" not \"restored in ten minutes.\" The oven bake runs much longer than ten minutes, so the wording has to be accurate. Check the real timing on camera before the spot runs.\n- **Pacing:** A cut every 3\u20136 seconds. Keep the time-lapse short so it doesn't drag.\n- **Logo:** Appears only on the end card, not at the start.\n- **Length:** About 45 seconds. The voiceover is short on purpose; the footage carries most of it.\n"}]}, "haiku-3": {"cost_usd": 0.0052, "seconds": 29, "output_tokens": 5285, "files": [{"name": "reel-script.md", "text": "# Forge & Flame: Rusty Pan Restoration Reel (45 sec)\n\nFormat: 9:16 vertical. Captions on every line (most viewers watch muted). No logo or intro card at the start.\n\n| Time | Shot | Voiceover | On-screen text |\n|---|---|---|---|\n| 0:00\u20130:03 | Hook. Steel wool scraping rust flakes off the pan, already in motion on frame one. | \"This rusted-solid pan cost five bucks. Watch it come back.\" | $5 flea market pan |\n| 0:03\u20130:08 | Before. Slow push-in on the rust. | \"Rust isn't the end. It's only on the surface.\" | Rust is surface damage |\n| 0:08\u20130:18 | Step 1, scrub. Close-up of steel wool and soap working the pan until dull grey metal shows. | \"Step one: steel wool and a little soap. Scrub until you see bare metal.\" | 1. SCRUB with steel wool + soap |\n| 0:18\u20130:26 | Step 2, oil. Thin oil wiped on, then wiped off until it looks nearly dry. | \"Step two: wipe on a thin layer of oil, then wipe off the excess.\" | 2. THIN OIL COAT |\n| 0:26\u20130:34 | Step 3, oven. Pan upside down on the rack, oven door closing. | \"Step three: bake it upside down at 450 degrees for an hour. You did about ten minutes of work. The oven does the rest.\" | 3. 450\u00b0F / 230\u00b0C, 1 HOUR, upside down |\n| 0:34\u20130:41 | Payoff. Egg fried in the restored pan, slides off on a tilt. | \"Then I fried an egg. It slid right off.\" | Zero sticking |\n| 0:41\u20130:45 | Outro. Logo card, then a short follow line. | \"Forge & Flame. Follow for the next restoration.\" | Forge & Flame \u00b7 Follow for the next one |\n\nVoiceover is about 100 words, which fits 45 seconds with room for the shots to breathe.\n\n## Footage status\n\n- Already shot: rusty pan (hook and before), before/after, egg slide-off (payoff).\n- Still needed: the scrub, oil wipe, and oven shots for steps 1\u20133. Without them, steps 1\u20133 play over the before shots.\n\n## Editing notes\n\n- Cut every 3\u20135 seconds. Keep motion in every shot.\n- The egg payoff comes after the process, so viewers who stay through the steps get the reward.\n- The oven bake takes about an hour, so the script says \"ten minutes of work\" rather than \"restored in 10 minutes.\" A \"10 minutes\" claim with no mention of the oven would likely draw comments saying it's wrong.\n"}]}}, "verdicts": [{"vs": "opus", "pair": "1", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "haiku+skill", "why": "A offers more workable pacing and fewer invented details. B invents a $3 price and a no-spray result, and crowds its opening narration. Both need explicit additional-shot requirements; A also needs cooling time."}, {"vs": "opus", "pair": "1", "judge": "gpt", "order": "opus shown first", "winner": "opus", "why": "A reveals the payoff immediately and explicitly distinguishes hands-on work, oven time, and cooling. Both total 45 seconds but crowd some beats and fail to flag additional footage. A invents the $3 price; B delays payoff and overclaims \u201cNothing sticks.\u201d"}, {"vs": "opus", "pair": "1", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "tie", "why": "Both meet all criteria: immediate hook, accurate 45s script, sustained interest, correct restoration, use of footage, and brand integration."}, {"vs": "opus", "pair": "1", "judge": "deepseek", "order": "opus shown first", "winner": "opus", "why": "A's first-second hook is stronger, using a striking before/after smash-cut with curiosity text, directly addressing the 1.2-second scroll problem."}, {"vs": "opus", "pair": "2", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "haiku+skill", "why": "A is more executable and avoids unsupported product promotion, though it invents the $8 price. B overcrowds timed narration and invents ready-to-cook products. Both sustain the transformation but omit cooling time and fully explicit cleaning."}, {"vs": "opus", "pair": "2", "judge": "gpt", "order": "opus shown first", "winner": "haiku+skill", "why": "B has more realistic pacing, flags needed process footage, and closes cleanly. Its $8 price is invented. A invents ready-to-cook product claims, leaves extra footage unflagged, and overcrowds its ending. Both omit cooling time."}, {"vs": "opus", "pair": "2", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "opus", "why": "B more fully utilizes the supplied footage and explicitly addresses the 1.2-second problem with a stronger, multi-part hook."}, {"vs": "opus", "pair": "2", "judge": "deepseek", "order": "opus shown first", "winner": "opus", "why": "A nails the hook, uses all supplied footage, and is accurate. B incorrectly states most footage is missing and has a weaker opening."}, {"vs": "haiku", "pair": "1", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "tie", "why": "Both deliver strong hooks, correctly timed scripts, transformation payoffs and clear branded CTAs. Neither includes cooling or explicitly flags additional footage. A overclaims \u201cNothing sticks\u201d; B\u2019s unqualified ten-minute hook contradicts its note claiming the script specifies hands-on time."}, {"vs": "haiku", "pair": "1", "judge": "gpt", "order": "haiku shown first", "winner": "haiku", "why": "A has more comfortable pacing and explicit washing. Both omit cooling and fail to flag required process footage. A\u2019s claimed hands-on clarification never appears in-script; B crowds narration and overstates the egg demonstration with \u201cNothing sticks.\u201d"}, {"vs": "haiku", "pair": "1", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "tie", "why": "Both scripts meet all criteria effectively, with strong hooks, accurate instructions, and proper use of footage."}, {"vs": "haiku", "pair": "1", "judge": "deepseek", "order": "haiku shown first", "winner": "tie", "why": "Both scripts meet all criteria effectively, with no material differences in quality or adherence."}, {"vs": "opus", "pair": "3", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "haiku+skill", "why": "A is more executable and faithful, though its timing claim and oven placeholder need fixing. B invents price, testimony, and longevity claims; leaves extra footage unflagged; rushes narration; and omits washing and cooling."}, {"vs": "opus", "pair": "3", "judge": "gpt", "order": "opus shown first", "winner": "haiku+skill", "why": "B avoids A\u2019s invented price, testimonials and longevity claims, flags missing footage, and integrates the brand cleanly. Both total 45 seconds, but B leaves oven instructions unfinished and misleadingly says ten minutes; A omits washing and crowds several narration segments."}, {"vs": "opus", "pair": "3", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "tie", "why": "Both meet all criteria perfectly, with strong hooks, accurate timing, and proper use of footage."}, {"vs": "opus", "pair": "3", "judge": "deepseek", "order": "opus shown first", "winner": "opus", "why": "A's hook is stronger, its restoration claim is more accurate, and it better utilizes the supplied footage."}, {"vs": "haiku", "pair": "3", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "haiku+skill", "why": "A needs finalized oven instructions and corrected ten-minute wording, but identifies those gaps. B invents the $5 price and directs oiling immediately after soap scrubbing without rinsing or drying, undermining practical credibility."}, {"vs": "haiku", "pair": "3", "judge": "gpt", "order": "haiku shown first", "winner": "haiku", "why": "A is more executable and correctly distinguishes hands-on work from baking. It omits washing, drying and cooling, and invents a $5 price. B leaves oven settings unresolved and retains an unqualified ten-minute claim. Neither makes the payoff immediate enough."}, {"vs": "haiku", "pair": "3", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "haiku+skill", "why": "A correctly flags missing footage needed for steps, while B incorrectly states the egg slide is already shot."}, {"vs": "haiku", "pair": "3", "judge": "deepseek", "order": "haiku shown first", "winner": "haiku", "why": "A is more accurate on timing, uses footage correctly, and fits 45 seconds precisely."}, {"vs": "haiku", "pair": "2", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "haiku+skill", "why": "A gives clearer seasoning instructions, includes thorough drying, flags process footage, and ends with a relevant follow action. Both omit cleaning and explicit cooling time. A invents an $8 price; B leaves additional footage unflagged and uses an unsupported sales tagline."}, {"vs": "haiku", "pair": "2", "judge": "gpt", "order": "haiku shown first", "winner": "haiku+skill", "why": "B provides more credible restoration instructions and flags process footage needed, but invents the $8 price and omits cooling. A skips cleaning and drying before oiling and fails to flag unavailable process shots. Both deliver strong hooks and correctly timed scripts."}, {"vs": "haiku", "pair": "2", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "haiku+skill", "why": "A's call to action is more appropriate; B's 'Shop cast iron' is unsupported by the brief."}, {"vs": "haiku", "pair": "2", "judge": "deepseek", "order": "haiku shown first", "winner": "haiku", "why": "A is fully executable with supplied footage, has correct timing, and accurately flags needed shots. B misstates footage availability and has timing errors."}]}, {"id": "hook-3", "family": "video-hook", "prompt": "Write a 30-second YouTube Short script on the topic in notes.md. I need people to actually watch it.", "input": "notes.md", "skill": "kallaway-layered-hook", "input_text": "Channel: a software engineer explaining tech careers, 12k subscribers.\nTopic: \"Why your resume gets rejected in 6 seconds\" \u2014 recruiters skim for 3 things: job titles, company names, and numbers in bullet points.\nAssets: a real (anonymised) before/after resume, a screen recording of a recruiter-style 6-second skim with a timer.\n", "rubric": [{"id": "c1", "criterion": "The opening gives job seekers an immediate reason to keep watching.", "full_marks_when": "The first 1\u20132 seconds introduce a concrete resume problem or challenge, such as whether viewers can spot the important information before the six-second timer runs out, without greetings or a lengthy setup.", "zero_when": "The script opens with channel introductions, generic career advice, or background explanation that delays the reason to watch."}, {"id": "c2", "criterion": "The script accurately represents the supplied notes and avoids unsupported claims.", "full_marks_when": "It identifies all three skim targets correctly: job titles, company names, and numbers in bullet points. It presents the six-second skim as the demonstration or premise, without claiming that every recruiter rejects resumes after exactly six seconds or inventing hiring statistics or results.", "zero_when": "It misstates the three targets, contradicts the input, or bases its explanation on fabricated statistics or universal claims about recruiter behavior."}, {"id": "c3", "criterion": "The deliverable is a complete, realistically paced 30-second script.", "full_marks_when": "It provides ready-to-record spoken lines and concise visual cues that fit approximately 30 seconds, allowing time for the six-second skim. If that skim is a separate segment, it leaves 24 seconds for the rest; if narration overlaps it, the timing makes that clear.", "zero_when": "It provides only advice or an outline, or its dialogue and required visual segments plainly cannot fit within 30 seconds."}, {"id": "c4", "criterion": "The available assets visibly prove the point.", "full_marks_when": "It specifies where to show the anonymised before/after resume and the recruiter-style screen recording with its six-second timer, using readable crops or highlights to connect the skim targets to concrete changes. It preserves anonymity and does not invent details about the unseen resume.", "zero_when": "It ignores the supplied assets, uses them only as unrelated background, exposes identifying information, or invents resume details as though they were supplied."}, {"id": "c5", "criterion": "Viewers receive a useful resume improvement they can apply immediately.", "full_marks_when": "The script shows how to make relevant titles, company names, and truthful quantified accomplishments easy to spot, with a concrete before/after contrast or clearly labelled illustrative example suited to tech job seekers. It avoids suggesting fabricated metrics or promising that formatting guarantees an interview.", "zero_when": "It offers only vague instructions such as 'make your resume better,' gives no actionable improvement, or encourages misleading credentials or invented numbers."}, {"id": "c6", "criterion": "The short sustains attention and delivers the promised payoff.", "full_marks_when": "The opening challenge or question leads into the timed skim, then a clear reveal or before/after payoff. Each beat advances the explanation, spoken language is accessible to the channel's tech-career audience, and any closing prompt is brief enough to preserve the payoff.", "zero_when": "It repeats the premise without a reveal, lets unrelated filler or engagement requests consume the runtime, or ends without explaining what viewers should learn from the skim."}], "runs": {"haiku-skill-1": {"cost_usd": 0.00358, "seconds": 16, "output_tokens": 2659, "files": [{"name": "short_script.md", "text": "# Short: \"Your resume gets rejected in 6 seconds\"\n\nRuntime: ~30s \u00b7 Format: 9:16 vertical \u00b7 Assets: before/after resume, 6-second skim recording with timer\n\n---\n\n**0:00\u20130:02 \u2014 HOOK**\nVISUAL: Screen recording starts. Timer at 0.0 over the *before* resume. Timer ticks visibly.\nVO: \"Recruiters spend six seconds on your resume. Here's what they actually read.\"\nON-SCREEN TEXT: \"6 SECONDS. WHAT DO THEY READ?\"\n\n**0:02\u20130:08 \u2014 THE THREE THINGS**\nVISUAL: Highlights flash on the resume in sequence: job titles, then company names, then numbers.\nVO: \"Three things. Job titles. Company names. And numbers in your bullet points.\"\nON-SCREEN TEXT: \"1. Job titles  2. Companies  3. Numbers\"\n\n**0:08\u20130:17 \u2014 THE PROBLEM**\nVISUAL: Timer at ~0:08. Cursor skims down the first bullet of the *before* resume. Bullet is dimmed as the cursor passes.\nVO: \"Look at this bullet. 'Responsible for improving the checkout flow.' No number. The eye slides right past it.\"\nON-SCREEN TEXT: \"No number = skipped\"\n\n**0:17\u20130:25 \u2014 THE FIX**\nVISUAL: Cut to the *after* resume. Same bullet, rewritten. The number glows.\nVO: \"Now the after. 'Cut checkout load time by 40%.' One number, and the skim stops.\"\nON-SCREEN TEXT: \"Cut checkout load time by 40%\"\n\n**0:25\u20130:30 \u2014 PAYOFF + LOOP**\nVISUAL: Timer hits 6.0 and freezes. Split screen: before (dimmed) and after (highlighted).\nVO: \"Put a number in your top three bullets. Check yours right now.\"\nON-SCREEN TEXT: \"Numbers > adjectives\"\nEND CARD: Follow for more career breakdowns.\n\n---\n\n## Word count\n~75 words of voiceover. Reads in about 28\u201330 seconds at a natural pace.\n\n## Production notes\n- Keep the timer visible from the first frame. It turns the hook into a challenge and gives viewers a reason to stay to the end.\n- Replace the example bullets with the real lines from your anonymised before/after resume so the on-screen text matches the footage.\n- Cut the \"Check yours right now\" line if you want a cleaner ending. Viewers who reach 0:30 are already looping back to the hook.\n- If you state \"six seconds\" as fact in the voiceover, cite the source in the description or drop the exact figure and say \"a few seconds.\"\n"}]}, "haiku-skill-2": {"cost_usd": 0.00434, "seconds": 20, "output_tokens": 3914, "files": [{"name": "resume-6-second-skim-short.md", "text": "# YouTube Short (30s): Why Your Resume Gets Rejected in 6 Seconds\n\n**Title:** Why your resume gets rejected in 6 seconds\n**Length:** ~30s (about 73 spoken words)\n\n## Script\n\n**[0:00\u20130:02] HOOK**\n- VISUAL: Full-screen resume. Timer overlay starts at 6.0s and counts down. Fast push-in.\n- ON-SCREEN TEXT: \"Rejected in 6 seconds.\"\n- VO: \"Your resume got rejected in six seconds.\"\n\n**[0:02\u20130:07] THE SKIM**\n- VISUAL: Screen recording from the skim asset. Cursor scrolls down the resume, timer visible.\n- VO: \"Recruiters don't read your resume. They skim it, looking for three things.\"\n\n**[0:07\u20130:12] THE THREE THINGS**\n- VISUAL: Cursor lands on each item in turn. Each one gets a highlight box.\n- ON-SCREEN TEXT: \"1. JOB TITLES\" / \"2. COMPANY NAMES\" / \"3. NUMBERS\"\n- VO: \"One: job titles. Two: company names. Three: numbers in your bullet points.\"\n\n**[0:12\u20130:20] BEFORE / AFTER**\n- VISUAL: Split screen from the anonymised before/after resume. Left: the original bullet. Right: the revised bullet. Hard cut between them at \"After.\"\n- ON-SCREEN TEXT: \"BEFORE\" / \"AFTER\"\n- VO: \"Before: 'Worked on the checkout flow.' After: 'Cut checkout latency 40%.' Same job. Only one of them gets a second look.\"\n- NOTE: Replace these two lines with the real bullets from the anonymised asset. The wording above is a stand-in.\n\n**[0:20\u20130:25] THE FIX**\n- VISUAL: Back to full resume. Title and company lines glow. Then a number inside a bullet glows.\n- VO: \"So: titles up top, companies named, a number in every bullet.\"\n\n**[0:25\u20130:30] CALL TO ACTION**\n- VISUAL: Creator on camera, or the resume with the timer frozen at 6.0s.\n- ON-SCREEN TEXT: \"Comment 'RESUME'\"\n- VO: \"Want the full breakdown? Comment 'resume' and I'll make it.\"\n\n## Production notes\n- **Hook in the first second.** The text \"Rejected in 6 seconds\" has to be on screen in frame one, because most viewers watch without sound.\n- **Captions on everything.** Burn in captions for the whole VO. Many Shorts viewers watch muted.\n- **Keep the timer on screen during the skim.** The on-screen countdown ties the visual to the title's promise and gives viewers a reason to stay.\n- **Use the real before/after.** Swap in the anonymised bullets. Recruiters react to specifics, so a real example works better than the stand-in.\n- **Pinned comment.** Pin a reply that asks what role viewers are applying for. That gives the call to action a place to go.\n"}]}, "haiku-skill-3": {"cost_usd": 0.00387, "seconds": 16, "output_tokens": 3162, "files": [{"name": "short-script.md", "text": "# Short: Why your resume gets rejected in 6 seconds\n\nRuntime: 30s | Format: 9:16 vertical | Spoken words: ~67 (about 27s at a brisk pace)\n\n| Time | Voiceover | Visual | On-screen text |\n|------|-----------|--------|----------------|\n| 0:00-0:03 | \"Recruiters give your resume six seconds. Here's what they're looking at.\" | Skim screen recording starts, timer visible. Cold open, no intro. | 6 SECONDS |\n| 0:03-0:10 | \"First, job titles. Does yours match the role they're hiring for?\" | Anonymised resume, job title line highlighted. | 1. JOB TITLES |\n| 0:10-0:17 | \"Second, company names. A recognizable one earns a second look.\" | Company name circled on the same resume. | 2. COMPANY NAMES |\n| 0:17-0:25 | \"Third, numbers. 'Managed a team' gets skipped. 'Cut deploy time by 40%' stops the scroll.\" | Cut to before/after bullets, vague line fades to the quantified one. | 3. NUMBERS |\n| 0:25-0:30 | \"Numbers buried in paragraph two? They never got read. Follow for the rewrite.\" | Side-by-side before/after, end card. | FOLLOW FOR THE REWRITE |\n\n## Notes\n- The hook is the first three seconds. Start on the timer, not on a face or a logo.\n- Captions are required; most Shorts are watched with sound off.\n- The loop back to \"six seconds\" in the final line encourages a replay.\n- The CTA promises a rewrite video. Make that video, or change the last line to match what you publish.\n"}]}, "opus-1": {"cost_usd": 0.09687, "seconds": 29, "output_tokens": 2494, "files": [{"name": "script.md", "text": "# YouTube Short: \"Why your resume gets rejected in 6 seconds\"\n\n**Length:** ~30 s (about 80 spoken words, at a brisk Shorts pace)\n**Format:** Vertical 9:16, captions burned in, on-screen timer for most of the video\n**Assets used:** anonymised before/after resume, screen recording of the 6-second skim with a timer\n\n---\n\n## Script\n\n| Time | Visual | Voiceover / on-screen text |\n|---|---|---|\n| **0:00\u20130:02** (HOOK) | Open straight on the \"before\" resume. A red **6.0** timer starts counting down right away. No intro or logo. | **VO:** \"Your resume just got rejected. Here's what the recruiter actually saw.\" <br>**Text:** `REJECTED IN 6 SECONDS` |\n| **0:02\u20130:08** | Play the skim recording on the \"before\" resume. Eye-path highlights jump around and land on nothing. Timer hits 0, then a red \u2715 stamp. | **VO:** \"That was the whole read. Six seconds. They aren't reading your resume, they're scanning it for three things.\" |\n| **0:08\u20130:12** | Hard cut. Zoom in on the job title line and circle it. | **VO:** \"One: job titles.\" <br>**Text:** `1. TITLES` |\n| **0:12\u20130:16** | Zoom in on company names and circle them. | **VO:** \"Two: company names.\" <br>**Text:** `2. COMPANIES` |\n| **0:16\u20130:22** | Zoom in on a \"before\" bullet: *\"Worked on backend services.\"* It morphs into the \"after\" bullet: *\"Cut API latency **40%** for **2M** users.\"* The numbers glow. | **VO:** \"Three, and this is where most people lose: numbers in your bullets. 'Worked on backend' is invisible. 'Cut latency 40%' gets read.\" <br>**Text:** `3. NUMBERS` |\n| **0:22\u20130:27** | Split screen: the \"before\" skim next to the \"after\" skim, with the timer restarting. On the \"after\" side, the eye path snaps cleanly from title to company to number. A green \u2713. | **VO:** \"Same person, same experience. One gets a call back.\" |\n| **0:27\u20130:30** (LOOP + CTA) | Cut back to the 6.0 timer on a blank resume. | **VO:** \"Pause this. Read your own resume for six seconds. What did you see?\" <br>**Text:** `PAUSE. 6 SECONDS. GO.` |\n\n---\n\n## Why it's built this way (retention notes)\n\n- **The first 2 seconds carry the most weight.** The hook says \"you\" and \"rejected\" and shows the timer in the first frame, so viewers feel the 6 seconds instead of hearing about them.\n- **The timer is the visual hook.** A countdown on screen keeps people watching to see it run out. Keep it visible through 0:08 and bring it back at 0:22.\n- **There's a cut every 2\u20134 seconds.** Each of the three points gets its own zoom and caption, so nothing on screen stays still.\n- **Point 3 has the payoff.** The bullet changing from vague to specific is the moment people will screenshot or share, so it gets the most screen time.\n- **The ending loops.** The last frame is the 6.0 timer and the first frame is the 6.0 timer, so the replay feels seamless and pushes watch time past 100%. The \"pause this\" line also gives people a reason to scrub back.\n- **The CTA is an action, not \"like and subscribe.\"** \"Read your own resume for six seconds\" sparks comments (\"I only saw my name lol\"), and comments help reach.\n\n## Production checklist\n\n- [ ] Captions on every line, centred, high contrast, kept out of the bottom 20% where the Shorts UI covers them\n- [ ] Double-check that names, emails, and company identifiers are blurred in both resume versions\n- [ ] Swap the example bullet for a real one from the \"after\" resume if it has a stronger number\n- [ ] Add a subtle tick sound under the timer and a \"thud\" on the \u2715 stamp\n- [ ] Read the VO aloud with a stopwatch: if it's over 30 s, cut \"Same person, same experience\" first\n\n## Title / caption options\n\n1. Why your resume gets rejected in 6 seconds\n2. Recruiters only look for 3 things on your resume\n3. I timed a recruiter reading a resume. 6 seconds.\n\n**Pinned comment idea:** \"Drop the first thing you noticed on your own resume after 6 seconds \ud83d\udc47\"\n"}]}, "opus-2": {"cost_usd": 0.10603, "seconds": 32, "output_tokens": 2874, "files": [{"name": "script.md", "text": "# YouTube Short: \"Why your resume gets rejected in 6 seconds\"\n\n**Length:** ~30s (about 80 spoken words at a brisk Shorts pace)\n**Format:** 9:16 vertical. Burned-in captions throughout. Fast cuts, nothing on screen longer than 3s without movement.\n**Assets used:** the before/after resume (anonymised) and the 6-second skim screen recording with a timer.\n\n---\n\n## Script\n\n| Time | Visual | Voiceover | On-screen text |\n|---|---|---|---|\n| **0:00\u20130:02** | **HOOK.** Open straight on the skim recording. Timer already running, \"BEFORE\" resume on screen. No intro and no face cam yet. | \"This is how long a recruiter looks at your resume.\" | **6 SECONDS** (big, pulsing with the timer) |\n| 0:02\u20130:05 | Timer hits 0:06. Hard cut, red \u2715 stamps over the resume, plus a \"reject\" sound. | \"Six seconds. And they're only looking for three things.\" | \u274c REJECTED |\n| 0:05\u20130:09 | Face cam, then a punch-in zoom on the **job titles** in the BEFORE resume, highlighted in yellow. | \"One: your job titles. Can they tell what you actually did?\" | **1. JOB TITLES** |\n| 0:09\u20130:13 | Zoom on the **company names**, highlighted. | \"Two: company names. Are they easy to spot, or buried in a paragraph?\" | **2. COMPANIES** |\n| 0:13\u20130:18 | Zoom on the bullet points. The BEFORE bullets have no numbers (\"Worked on backend services\u2026\"). Slow pan, everything grey. | \"Three, and this is where most people lose: numbers. 'Worked on backend services' tells them nothing.\" | **3. NUMBERS** |\n| 0:18\u20130:24 | **REVEAL.** Swipe transition to the AFTER resume. The skim recording replays with the timer and the eye path lands on titles, companies and numbers (\"Cut API latency 40%\"). Each one gets a \u2713 as it lands. | \"Same person. Same experience. Now titles pop, companies are bolded, and every bullet has a number.\" | \u2713 \u2713 \u2713 |\n| 0:24\u20130:28 | Timer stops at 0:06. Green \"INTERVIEW\" stamp. Split screen with BEFORE on the left and AFTER on the right. | \"Same six seconds. Completely different result.\" | BEFORE \u279c AFTER |\n| 0:28\u20130:30 | Face cam, pointing at the camera. | \"Go count the numbers on yours. I'll wait.\" | **How many numbers are on YOUR resume? \ud83d\udc47** |\n\n---\n\n## Voiceover (clean read, about 80 words)\n\n> This is how long a recruiter looks at your resume.\n> Six seconds. And they're only looking for three things.\n> One: your job titles. Can they tell what you actually did?\n> Two: company names. Are they easy to spot, or buried in a paragraph?\n> Three, and this is where most people lose: numbers. \"Worked on backend services\" tells them nothing.\n> Same person. Same experience. Now titles pop, companies are bolded, and every bullet has a number.\n> Same six seconds. Completely different result.\n> Go count the numbers on yours. I'll wait.\n\n---\n\n## Why it's built this way (retention notes)\n\n- **Hook in the first frame, not the first sentence.** A ticking timer on a real resume is the hook, so the viewer sees the stakes before they hear anything. No \"Hey guys\" and no channel intro, because those lose a large share of Shorts viewers in the first 2 seconds.\n- **Open loop.** \"Only three things\" makes people stay to hear all three. Numbering them on screen (\"1. / 2. / 3.\") shows how far along they are.\n- **The before/after is the payoff.** It goes at about 0:18, roughly the point where Shorts viewers start dropping off, so there's a fresh visual reward right when attention dips.\n- **Loopable ending.** The closing line and the opening line both involve the timer and \"your resume\", so the cut back to 0:00 feels seamless and boosts replays.\n- **The CTA asks for a comment, not a subscribe.** \"How many numbers are on YOUR resume?\" is easy to answer with a single digit, which drives comments, and comments drive Shorts distribution.\n\n## Title / caption options\n\n1. Why your resume gets rejected in 6 seconds\n2. Recruiters only look at 3 things on your resume\n3. I tested my resume with a 6-second timer \ud83d\ude2c\n\n**Description:** Recruiters skim, they don't read. Fix these 3 things first. #resume #techcareers #softwareengineer #jobsearch\n\n## Production checklist\n\n- [ ] Check the anonymisation on both resumes **at full zoom**. The punch-ins will make any leftover names, emails or company logos readable.\n- [ ] Make sure the AFTER resume's numbers are real or clearly illustrative. Commenters on career content will call out made-up metrics.\n- [ ] Sync the on-screen timer to the VO so \"six seconds\" lands exactly when the timer hits 0:06.\n- [ ] **Fact note on the \"6 seconds\" stat:** this comes from the Ladders eye-tracking study (2012). Their 2018 follow-up measured about 7.4 seconds. The script says \"how long a recruiter looks\", which fits your own demo recording. If you want to cite the study, a pinned comment like \"Based on Ladders' eye-tracking research (6\u20137s average)\" heads off \"well actually\" replies.\n"}]}, "opus-3": {"cost_usd": 0.10949, "seconds": 33, "output_tokens": 2999, "files": [{"name": "script.md", "text": "# YouTube Short: \"Why your resume gets rejected in 6 seconds\"\n\n**Length:** ~30 sec \u00b7 **Spoken words:** ~80 (fast, energetic delivery)\n**Format:** 9:16 vertical, burned-in captions throughout (most Shorts are watched muted)\n**Assets used:** recruiter-skim screen recording with timer, anonymised before/after resume\n\n---\n\n## Title & cover options\n- **Title:** Recruiters look at 3 things on your resume. That's it.\n- **Alt title:** Your resume has 6 seconds. Here's what they read.\n- **Cover text (big, 3\u20134 words):** \"6 SECONDS. 3 THINGS.\"\n\n---\n\n## Script\n\n| Time | Visual | Voiceover | On-screen text |\n|---|---|---|---|\n| **0\u20132s** (HOOK) | Open cold on the skim recording. 6-second timer already running, big, top of frame. The \"before\" resume fills the screen. | \"This resume just got rejected. Watch the clock.\" | \u23f1 6.0 \u2192 counting down |\n| **2\u20138s** | Same recording. As the recruiter's eye lands on each item, drop a bright box on it: title \u2192 company \u2192 a number. Dim everything else to ~30%. | \"A recruiter reads three things. Job titles. Company names. Numbers.\" | **1. TITLE** \u00b7 **2. COMPANY** \u00b7 **3. NUMBERS** (pop in one at a time, synced to VO) |\n| **8\u201311s** | Timer hits 0. Hard cut to the dimmed summary / skills section with a red \u2715. | \"Your summary? Skipped. That skills list? Skipped.\" | SKIPPED \u2715 |\n| **11\u201319s** | Zoom in on one \"before\" bullet, then swipe to the matching \"after\" bullet. | \"Before: 'Responsible for improving checkout.' Nothing to grab. After: 'Cut checkout load time 40%.' That's what stops the eye.\" | \u274c before line \u2192 \u2705 after line, with the number highlighted |\n| **19\u201326s** | Split screen: before vs after, side by side, boxes on titles/companies/numbers on the \"after.\" | \"So: title where the eye lands first. Company easy to spot. A number in every bullet you can.\" | \u2705 Title up top \u00b7 \u2705 Company visible \u00b7 \u2705 Numbers |\n| **26\u201330s** | Restart the 6-second timer over a blank \"your resume\" frame. Face-to-camera if you have it. | \"Now give your own resume six seconds. What did you see?\" | \u23f1 6.0 \u2014 YOUR TURN |\n\nThe last line hands viewers a task (checking their own resume) and leads back to the timer from the opening, so the Short loops cleanly when it replays.\n\n---\n\n## Full voiceover (for recording in one take)\n\n> This resume just got rejected. Watch the clock.\n> A recruiter reads three things. Job titles. Company names. Numbers.\n> Your summary? Skipped. That skills list? Skipped.\n> Before: \"Responsible for improving checkout.\" Nothing to grab.\n> After: \"Cut checkout load time 40%.\" That's what stops the eye.\n> So: title where the eye lands first. Company easy to spot. A number in every bullet you can.\n> Now give your own resume six seconds. What did you see?\n\n---\n\n## Pinned comment\n> Drop your worst \"Responsible for\u2026\" bullet below and I'll rewrite it with a number. \ud83d\udc47\n\n(This brings in comments, and the replies give you material for follow-up Shorts.)\n\n---\n\n## Production notes (check these before you film)\n1. **Swap in your real bullets.** The checkout before/after lines are placeholders. Use an actual before/after pair from your anonymised resume so what you say matches what's on screen. Keep the \"after\" under ~8 words so people can read it in 2 seconds.\n2. **Sync the boxes to the recording.** The 2\u20138s and 8\u201311s beats assume your skim recording lands on titles, companies, and numbers and skips the summary and skills. If it doesn't, trim or reorder the clip so it does, or drop the \"Skipped\" line.\n3. **Cite the \"6 seconds\" claim.** It comes from the Ladders eye-tracking study (2012). Their 2018 follow-up measured about 7.4 seconds. Someone in the comments will bring this up, so consider a small on-screen source note (\"Source: Ladders eye-tracking study\") or a line in the description.\n4. **Pacing:** no cut longer than ~3 seconds. Keep the timer on screen for the first 8 seconds; the countdown is what holds attention.\n5. **Don't open with an intro.** Skip \"Hey guys\" and channel branding. The first frame should be the timer and resume.\n"}]}, "haiku-1": {"cost_usd": 0.00439, "seconds": 20, "output_tokens": 3547, "files": [{"name": "shorts_script.md", "text": "# Why Your Resume Gets Rejected in 6 Seconds (30s Short)\n\n**Target length:** ~30s (about 66 spoken words)\n**Assets:** anonymised before/after resume, 6-second skim screen recording with timer\n\n---\n\n**[0:00\u20130:03] HOOK**\n- **VO:** \"A recruiter spends six seconds on your resume. Here's what they're actually looking at.\"\n- **Visual:** Resume fills the frame. Timer starts at 6.00 and counts down.\n- **On-screen text:** \"6 SECONDS. THEN NO.\"\n\n**[0:03\u20130:10] THE THREE THINGS**\n- **VO:** \"Three things: job titles, company names, and numbers in your bullets.\"\n- **Visual:** Highlighter sweeps over each one in turn.\n- **On-screen text:** \"1. TITLES  2. COMPANIES  3. NUMBERS\"\n\n**[0:10\u20130:20] BEFORE / AFTER**\n- **VO:** \"Same job, two versions. Before: 'Responsible for the checkout flow.' After: '[X]% faster checkout.' Same person. Different six seconds.\"\n- **Visual:** Split screen. The \"before\" bullet is grey and vague. The \"after\" bullet gets highlighted and the timer stops on it.\n- **Note:** Replace the bracketed text with the exact bullet from the anonymised asset. Don't use invented numbers as if they were real results.\n\n**[0:20\u20130:26] THE FIX**\n- **VO:** \"So lead with your strongest number, and make your titles and companies easy to spot.\"\n- **Visual:** Screen recording of the skim again, now with the numbers caught in the first pass.\n- **On-screen text:** \"LEAD WITH THE NUMBER\"\n\n**[0:26\u20130:30] LOOP + COMMENT BAIT**\n- **VO:** \"Comment 'roast' and I'll look at one of yours.\"\n- **Visual:** Cut back to the first frame of the resume, so the video loops into the hook.\n- **On-screen text:** \"COMMENT 'ROAST'\"\n\n---\n\n## Retention notes\n\n- **Hook:** The first 3 seconds promise a specific reveal (\"here's what they're actually looking at\"). The on-screen timer gives viewers a reason to stay until it hits zero.\n- **Pattern interrupts:** Each beat changes the visual (resume, highlighter, split screen, skim replay), so the screen never sits still for long.\n- **Loop:** The final frame matches the first, so people who are still watching may replay it.\n- **Accuracy:** The \"six seconds\" figure is widely cited, but sources vary (a 2018 eye-tracking study reported about 7.4 seconds). Keep the spoken claim as \"spends six seconds\" or \"about six seconds,\" and cite the source in the description.\n"}]}, "haiku-2": {"cost_usd": 0.00393, "seconds": 18, "output_tokens": 3263, "files": [{"name": "short_script.md", "text": "# Short: Why your resume gets rejected in 6 seconds\n\n**Runtime:** ~30s | **Format:** vertical 9:16, captions on (most people watch muted)\n\n**Title options:**\n- Recruiters spend 6 seconds on your resume\n- Why your resume gets rejected in 6 seconds\n\n---\n\n## Shot list\n\n**0:00\u20130:04 | Hook**\n- **VO:** \"Your resume gets rejected in six seconds. Not because you're underqualified. Because nobody saw the good part.\"\n- **Visual:** Anonymised before-resume fills the frame. Timer overlay starts at 0:00 and runs to 0:06.\n- **On-screen text:** \"6 SECONDS\"\n\n**0:04\u20130:09 | The skim**\n- **VO:** \"Recruiters skim for three things: job titles, company names, and numbers.\"\n- **Visual:** Screen recording of the skim. Three highlight boxes appear in sequence: title, company, bullet numbers.\n- **On-screen text:** \"1. Job titles  2. Company names  3. Numbers\"\n\n**0:09\u20130:15 | Before**\n- **VO:** \"This is the before. Nothing jumps out.\"\n- **Visual:** Hold on the before bullet from the real resume, [insert exact bullet text]. Timer runs in the corner.\n- **On-screen text:** \"No numbers. No hook.\"\n\n**0:15\u20130:23 | After**\n- **VO:** \"This is the after. Same job. Now the title, the company, and the number all land before the timer hits six.\"\n- **Visual:** Swap to the after-resume with the same layout. Highlight boxes land on the title, company, and number. Timer stops at 0:06.\n- **On-screen text:** \"Same job. Same person.\"\n\n**0:23\u20130:30 | Call to action and loop**\n- **VO:** \"Put a number in every bullet. Comment yours and I'll rewrite it. Six seconds. Make them count.\"\n- **Visual:** Back to the first frame of the after-resume so the video loops cleanly into the hook.\n- **On-screen text:** \"Comment your bullet\"\n\n---\n\n## Production notes\n\n- Replace the bracketed placeholder with the exact before-bullet from the anonymised resume. Keep the before and after bullets the same job, so the comparison is honest.\n- Keep the first frame readable. The hook has to land before the swipe.\n- The last frame matches the first so the Short loops. Replays raise watch time.\n- Pin a comment asking viewers to post a bullet. Reply to every one in the first day.\n"}]}, "haiku-3": {"cost_usd": 0.00375, "seconds": 16, "output_tokens": 3021, "files": [{"name": "short-script.md", "text": "# YouTube Short: \"Why your resume gets rejected in 6 seconds\"\n\nRuntime: ~30 seconds. Vertical 9:16. Assets: anonymised before/after resume, 6-second skim screen recording with timer.\n\n| Time | Visual | Voiceover | On-screen text |\n|---|---|---|---|\n| 0:00\u20130:03 | Resume fills frame, timer starts at 6.0s and ticks down | \"Your resume gets six seconds.\" | **YOUR RESUME GETS 6 SECONDS** |\n| 0:03\u20130:07 | Screen recording of a recruiter skimming, cursor jumping between sections | \"Here's what the recruiter actually looks at.\" | \u2014 |\n| 0:07\u20130:12 | Three labels slide in over the resume | \"They skim for three things: job titles, company names, and numbers.\" | **1. TITLES  2. COMPANIES  3. NUMBERS** |\n| 0:12\u20130:15 | Split screen: before (left) / after (right) | \"Here's a real resume I rewrote.\" | \u2014 |\n| 0:15\u20130:21 | Before bullet highlighted in red, then after bullet highlighted in green | \"Before: 'Responsible for managing client projects.' After: 'Managed 14 client projects, worth 2.3 million dollars a year.'\" | Red: *Responsible for managing client projects* / Green: *Managed 14 client projects, $2.3M/yr* |\n| 0:21\u20130:25 | Timer runs again on the \"after\" bullet, stops before 6s | \"Same job. Only one of these survives six seconds.\" | **SAME JOB** |\n| 0:25\u20130:30 | Creator on camera or a text card | \"Check your first bullet point. Does it have a number? Tell me in the comments and I'll rewrite it.\" | **COMMENT YOUR FIRST BULLET** |\n\n## Why it's built this way\n\n- **Hook in the first 3 seconds.** The claim and the timer are on screen before anyone decides to scroll.\n- **Text on screen, not only in voiceover.** Most Shorts are watched muted at first. The three labels carry the structure on their own.\n- **The bullet-point comparison is the payoff.** It shows the lesson instead of just stating it.\n- **Ending is a question with a clear action.** Asking viewers to post their own bullet gives a reason to comment, and it ends on something concrete instead of a generic \"subscribe.\"\n- **Loop-friendly.** The last line sends viewers back to the start to check a resume, which makes a rewatch likely.\n\n## Production notes\n\n- Blur or replace the candidate's name, employer logos, and contact details in every frame, including the timer recording.\n- Keep the real numbers from the resume if they're anonymised and accurate. Don't invent figures for the \"after\" example.\n- The \"6 seconds\" figure is widely repeated but its sourcing is weak. If you want to be safe, say \"about six seconds\" in the voiceover or keep the claim tied to the screen recording you made.\n- Read the voiceover aloud against the timings. It should land at about 75\u201385 words for a 30-second Short.\n- Pin a comment asking viewers to post a bullet point, then reply to the first few with rewrites. Replies keep the video in circulation.\n"}]}}, "verdicts": [{"vs": "opus", "pair": "1", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "haiku+skill", "why": "A is closer to recordable length and instructs replacing invented bullets. Both overstate six-second behavior. A\u2019s timer is inconsistent; B has roughly 110 spoken words, invents resume details and a callback outcome, and blurs company identifiers viewers need to see."}, {"vs": "opus", "pair": "1", "judge": "gpt", "order": "opus shown first", "winner": "tie", "why": "A has stronger reveal choreography but invents rejection and callback outcomes. B overstates recruiter behavior and mishandles its timer. Both present invented resume metrics as actual asset content, omit truthfulness guidance, and overcrowd their two-second hooks."}, {"vs": "opus", "pair": "1", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "tie", "why": "Both meet all rubric criteria effectively, with strong hooks, accurate content, and clear use of assets."}, {"vs": "opus", "pair": "1", "judge": "deepseek", "order": "opus shown first", "winner": "opus", "why": "A uses the skim recording directly to prove the point, while B's visual plan for 0:02-0:08 doesn't match the supplied asset."}, {"vs": "opus", "pair": "2", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "haiku+skill", "why": "A labels invented bullets as placeholders and uses the assets more responsibly. Both overstate recruiter behavior and undercount dialogue. B invents resume details and implies formatting produces interviews; neither adequately emphasizes truthful, relevant metrics."}, {"vs": "opus", "pair": "2", "judge": "gpt", "order": "opus shown first", "winner": "haiku+skill", "why": "B labels its example as a stand-in and directs use of real assets. A invents resume details and an interview outcome. Both overstate recruiter behavior, exceed their stated word counts, strain timing, and recommend numbers in every bullet without requiring truthfulness."}, {"vs": "opus", "pair": "2", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "tie", "why": "Both scripts meet all rubric criteria effectively, with strong hooks, accurate content, and clear use of assets."}, {"vs": "opus", "pair": "2", "judge": "deepseek", "order": "opus shown first", "winner": "opus", "why": "A uses assets precisely, shows a clear before/after payoff, and gives a more actionable, immediate improvement."}, {"vs": "haiku", "pair": "1", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "haiku", "why": "B gives actionable guidance for all three targets and explicitly guards against invented metrics. Both assert unsupported six-second recruiter behavior and leave timer sequencing unclear. A additionally scripts an unverified 40% result and contradicts its timer."}, {"vs": "haiku", "pair": "1", "judge": "gpt", "order": "haiku shown first", "winner": "haiku", "why": "A better connects the timed skim to a payoff and cautions against invented metrics. Both overstate recruiter behavior and overcrowd their opening timing. B invents a 40% result and gives contradictory timer instructions."}, {"vs": "haiku", "pair": "1", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "haiku+skill", "why": "A uses assets more precisely, provides a clearer payoff, and avoids unsupported engagement bait."}, {"vs": "haiku", "pair": "1", "judge": "deepseek", "order": "haiku shown first", "winner": "tie", "why": "Both scripts meet all criteria equally well, offering a strong hook, accurate content, proper timing, asset use, actionable advice, and sustained payoff."}, {"vs": "haiku", "pair": "2", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "haiku+skill", "why": "A delivers a clearer reveal and labels its example as a stand-in. Both assert unsupported six-second rejection claims and prescribe numbers in every bullet. B additionally assumes unseen resume flaws, overcrowds its opening, and muddles the timer."}, {"vs": "haiku", "pair": "2", "judge": "gpt", "order": "haiku shown first", "winner": "haiku+skill", "why": "B provides a clearer skim-to-reveal sequence and labels invented bullets as stand-ins. Both overstate six-second rejection and prescribe numbers in every bullet. A adds unsupported resume observations and confusing timer continuity; B\u2019s spoken-word estimate is inaccurate."}, {"vs": "haiku", "pair": "2", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "haiku+skill", "why": "A's 'fix' is more actionable and avoids promising to rewrite every comment."}, {"vs": "haiku", "pair": "2", "judge": "deepseek", "order": "haiku shown first", "winner": "tie", "why": "Both scripts meet all rubric criteria effectively, with no significant flaws."}, {"vs": "opus", "pair": "3", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "opus", "why": "B provides clearer asset highlights, explicitly labels invented bullets as placeholders, and delivers an actionable reveal. Both make unsupported recruiter claims. B also stretches its six-second countdown across eight seconds and should require truthful metrics."}, {"vs": "opus", "pair": "3", "judge": "gpt", "order": "opus shown first", "winner": "opus", "why": "A better connects assets, highlights and actionable edits, explicitly labels placeholder bullets, and delivers a payoff. Its six-second timer incorrectly lasts eight seconds. Both overclaim recruiter behavior; B adds unsupported prestige claims and presents invented resume details without qualification."}, {"vs": "opus", "pair": "3", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "haiku+skill", "why": "B adds unsupported claims (e.g., citing a study) and invents resume details, violating c2."}, {"vs": "opus", "pair": "3", "judge": "deepseek", "order": "opus shown first", "winner": "opus", "why": "A uses assets fully, times beats precisely, and delivers a concrete payoff. B is less specific and underutilizes the assets."}, {"vs": "haiku", "pair": "3", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "haiku+skill", "why": "A is more realistically paced. Both provide immediate hooks and visual payoffs, but universalize the six-second premise and present invented resume metrics without illustrative labels. Neither clearly teaches improvements across all three skim targets; B\u2019s dialogue crowds its assigned timings."}, {"vs": "haiku", "pair": "3", "judge": "gpt", "order": "haiku shown first", "winner": "haiku", "why": "A delivers a clearer timed demonstration and before/after payoff. Both universalize the six-second premise and invent details from an unseen resume. A's caveats do not repair its spoken claims; B adds unsupported recruiter preferences and defers the rewrite."}, {"vs": "haiku", "pair": "3", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "haiku+skill", "why": "B invents a specific resume detail ('$2.3M/yr') not in the input, violating c2 and c5."}, {"vs": "haiku", "pair": "3", "judge": "deepseek", "order": "haiku shown first", "winner": "haiku", "why": "A uses assets more concretely, provides a clearer before/after payoff, and offers a more actionable viewer takeaway."}]}, {"id": "pov-2", "family": "pov-post", "prompt": "Turn my notes in notes.md into a LinkedIn post that stands out.", "input": "notes.md", "skill": "wes-kao-spiky-pov", "input_text": "Me: founder of a 15-person agency doing paid ads for DTC brands.\nObservation: across 60 client accounts, the brands that cut their ad testing to 3 creatives a week (instead of 20+) got better results after 3 months, because each creative got enough spend to learn from.\nMost agencies sell \"high-volume creative testing\". I think it mostly helps agencies bill more.\n", "rubric": [{"id": "c1", "criterion": "Preserves the supplied facts and avoids invented evidence.", "full_marks_when": "Accurately represents the observation across 60 client accounts: brands reduced testing from 20+ creatives to 3 per week and saw better results after 3 months. Any personal context correctly identifies a founder of a 15-person agency serving DTC brands with paid ads. Invents no performance metrics, quotes, or client stories. If calculating the reduction in creative volume, states at least 85%.", "zero_when": "Misstates the numbers, timing, business context, or direction of results, or fabricates supporting evidence."}, {"id": "c2", "criterion": "Makes the counterintuitive insight immediately compelling.", "full_marks_when": "Opens with a specific, attention-catching tension: testing fewer ad creatives produced better results, challenging the industry's emphasis on volume. The opening gives readers a reason to continue without generic clickbait.", "zero_when": "Uses a generic introduction or promotes high-volume testing, burying or reversing the central insight."}, {"id": "c3", "criterion": "Explains why fewer creatives helped in this situation.", "full_marks_when": "Clearly connects lower testing volume to enough spend per creative to learn from its performance, giving readers a practical reason to reconsider how they allocate testing budgets.", "zero_when": "Omits the spending-and-learning explanation or substitutes an unsupported explanation such as better creative quality or algorithm changes."}, {"id": "c4", "criterion": "Keeps the claim proportional to the evidence.", "full_marks_when": "Presents the results as an observation from these client accounts, without claiming a controlled experiment, guaranteed improvement, or a universal rule that every brand should test exactly 3 creatives weekly.", "zero_when": "Treats the observation as definitive causal proof or prescribes 3 creatives per week as universally optimal."}, {"id": "c5", "criterion": "Expresses the founder's criticism with credibility and edge.", "full_marks_when": "Retains the pointed view that selling creative-testing volume can serve agency billing more than client outcomes, while making clear this is the founder's judgment. Sounds like an agency insider challenging an incentive in their own industry.", "zero_when": "Removes the criticism entirely, turns the post into agency promotion, or asserts as established fact that all agencies deliberately exploit clients."}, {"id": "c6", "criterion": "Delivers a distinctive LinkedIn post ready to publish.", "full_marks_when": "Provides a coherent, concise post in a believable first-person founder voice, with readable paragraph breaks and a memorable ending or relevant discussion prompt. Requires no editing to remove placeholders, drafting instructions, excessive hashtags, or generic motivational filler.", "zero_when": "Provides advice about writing the post instead of the post itself, or delivers an unfinished, cluttered, or generic draft requiring substantial rewriting."}], "runs": {"haiku-skill-1": {"cost_usd": 0.00777, "seconds": 30, "output_tokens": 5672, "files": [{"name": "linkedin-post.md", "text": "Most agencies sell high-volume creative testing as the way to win on paid social.\n\nI think it mostly helps the agency bill more.\n\nAcross 60 client accounts, the brands that cut testing to about 3 new creatives a week, instead of 20+, got better results after three months.\n\nMy read on why: a budget split 20 ways gives each creative too little spend to separate a real winner from noise. So the account keeps testing, keeps reading noise, and keeps paying for the test. At 3 creatives a week, each one gets enough spend to actually teach you something.\n\nThe case for volume is real. Wide testing can surface a breakout creative you'd never have picked. But that only pays off if the winner gets enough spend to be proven. Otherwise, volume is just a more expensive way to guess.\n\nSo the number that matters isn't creatives per week. It's spend per creative before you judge it.\n\nIf you're reviewing your agency, ask two questions:\n1. How many new creatives go live each week?\n2. How much budget does each one get before anyone calls it a winner or a loser?\n\nIf the second answer is vague, you're paying for activity, not learning.\n\nWhat's the minimum spend you'd want on a creative before you judge it?\n"}]}, "haiku-skill-2": {"cost_usd": 0.00353, "seconds": 14, "output_tokens": 2620, "files": [{"name": "linkedin-post.md", "text": "Brands that tested fewer ads got better results.\n\nI run a 15-person agency doing paid ads for DTC brands. Across 60 client accounts, the brands that cut creative testing from 20+ ads a week down to 3 were ahead after three months.\n\nThe reason is simple. Each creative got enough spend to teach us something. At 20+, most ads never got enough budget to show what they could do, so decisions were made on noise.\n\nMost agencies sell \"high-volume creative testing.\" I think it mostly helps the agency bill more.\n\nWhat I'd rather do:\n\n\u2192 Test fewer creatives, each with enough spend to learn from\n\u2192 Make calls on what to cut only once there's real signal\n\nVolume is easy to sell. Learning is harder to show.\n\nIf you run paid social: how many creatives are you testing a week, and how do you know each one got a fair shot?\n\n#PerformanceMarketing #DTC #MarketingStrategy\n"}]}, "haiku-skill-3": {"cost_usd": 0.00703, "seconds": 23, "output_tokens": 4499, "files": [{"name": "linkedin-post.md", "text": "Most agencies sell high-volume creative testing.\n\nMore creatives. More shots on goal. Faster learning.\n\nWe looked at 60 client accounts. The brands that cut testing to 3 new creatives a week (instead of 20+) got better results after 3 months.\n\nThe reason is spend. Split a budget across 20 creatives and none of them gets enough delivery to teach you anything. Concentrate it on 3 and each one can actually be read.\n\nI think high-volume testing sells so well for a simple reason: it's easy to bill for. More creatives means more production, more reporting, more retainer. Whether it grows the brand's revenue is a separate question, and the industry rarely asks it.\n\nFair pushback: this is 60 accounts of observation, not a controlled experiment. Budget, product, and season all matter. Test it on your own account before you change anything.\n\nThe rule I'd start with: before you kill a creative, check whether it got enough spend to give a readable answer. If it didn't, it hasn't failed. It hasn't been tested.\n\nIf you've seen the opposite in your accounts, I want to hear it.\n\n#PerformanceMarketing #DTC\n"}]}, "opus-1": {"cost_usd": 0.073, "seconds": 19, "output_tokens": 1508, "files": [{"name": "linkedin-post.md", "text": "My agency would make more money if I told you to test 20 creatives a week.\n\nI'm going to tell you to test 3.\n\nWe run paid ads for DTC brands. 15 people, 60 client accounts.\n\nAcross those accounts, one pattern keeps showing up:\n\nThe brands that cut creative testing from 20+ a week down to 3 had better results three months later.\n\nNot because those 3 ads were brilliant.\nBecause each one got enough spend to actually teach us something.\n\nSpread a weekly budget across 20 creatives and every ad gets a little money. Most never get enough data to be judged fairly. You end up picking \"winners\" from noise, killing ads that never had a chance, and starting over every Monday.\n\nWith 3, each creative gets real budget. You find out what works, and you find out why. Then the next 3 are better.\n\nSo why does almost every agency pitch \"high-volume creative testing\"?\n\nBecause it sells. More concepts mean more production, more briefs, more reports, more hours on the invoice. It looks like progress.\n\nI think it mostly helps agencies bill more.\n\nI say that as someone who runs one.\n\nIf your agency's answer to flat performance is always \"we need more creatives,\" ask them one question:\n\nHow much spend did each of last month's ads get before you called it?\n\nBrands and marketers: how many creatives are you testing a week right now, and is it working?\n\n#PaidSocial #DTC #PerformanceMarketing #CreativeTesting\n"}]}, "opus-2": {"cost_usd": 0.08243, "seconds": 21, "output_tokens": 1890, "files": [{"name": "linkedin-post.md", "text": "My agency would make more money telling you to test 20 ads a week.\n\nI'm going to tell you to test 3.\n\nWe run paid ads for DTC brands. 15 people, 60 client accounts.\n\nSome of those brands cut their creative testing from 20+ new ads a week down to 3.\n\nThree months later, they were getting better results.\n\n3 isn't a magic number. It comes down to budget.\n\nSplit a fixed budget across 20 creatives and each one gets scraps. Not enough spend to learn anything. Not enough data to tell a winner from noise. You end up \"killing losers\" that never had a fair shot.\n\nSplit it across 3 and each creative gets enough spend to actually tell you something.\n\nNow the uncomfortable part.\n\n\"High-volume creative testing\" is what most agencies sell.\n\nI think it mostly helps agencies bill more.\n\n20 creatives a week is a production retainer. It looks like hard work and it fills a report. But often it doesn't help the brand learn anything.\n\nIf you're a DTC founder, ask your agency one question:\n\n\"How much spend does each creative get before we decide it doesn't work?\"\n\nIf the answer is vague, you're paying for volume, not learning.\n\nFewer ads. More spend behind each one. Real answers.\n\nIf you've seen high-volume testing really work, I'd like to hear what made it work. Tell me in the comments.\n\n#PaidSocial #DTC #PerformanceMarketing\n\n\n---\n\nAlternate opening lines (swap in for the first two lines if you prefer):\n\n1. \"We looked at 60 ad accounts. The brands testing fewer creatives won.\"\n2. \"Testing 20 ads a week is great for your agency's invoice. Not always for your results.\"\n3. \"Most agencies sell volume. Our best-performing clients did the opposite.\"\n"}]}, "opus-3": {"cost_usd": 0.07367, "seconds": 19, "output_tokens": 1548, "files": [{"name": "linkedin-post.md", "text": "# LinkedIn Post\n\n---\n\nMy industry sells a lie, and I've billed for it.\n\n\"High-volume creative testing.\"\n20, 30, 50 new ads a week. More shots on goal. More data. More winners.\n\nIt sounds rigorous. It looks great in a monthly report.\n\nHere's what we actually saw across 60 DTC client accounts:\n\nThe brands that cut testing down to 3 creatives a week, instead of 20+, got better results after 3 months.\n\nNot because 3 is a magic number.\n\nBecause when you split a budget 20 ways, no single ad gets enough spend to tell you anything. You're not testing. You're guessing at scale and calling it data.\n\nGive 3 creatives real money and the platform can actually learn. So can you. You find out *why* something works, not just that it got lucky for four days.\n\nSo why does almost every agency push volume?\n\nMy honest take: it mostly helps agencies bill more.\nMore creatives = more production hours = bigger retainers = a busier-looking dashboard.\n\nI run a 15-person agency. Volume would be easier to sell. It's just not what the numbers told us.\n\nIf you're a DTC brand paying for \"high-volume testing,\" ask your agency one question:\n\nHow much spend did each creative get before you called it a loser?\n\nIf the answer makes you wince, you're paying for activity, not learning.\n\nFewer ads. More conviction. Better results.\n\nAgency folks: am I wrong? I'd genuinely like to hear the counter-argument.\n\n#PaidSocial #DTC #PerformanceMarketing #CreativeStrategy\n\n---\n\n## Alternative opening lines\n\n1. \"We told 60 clients to make fewer ads. The ones who listened won.\"\n2. \"20 ads a week isn't a testing strategy. It's a billing strategy.\"\n3. \"The most profitable thing we did for clients this year was cut their creative output by 85%.\"\n\n## Notes before posting\n\n- The post only says \"better results\" because your notes don't specify. Adding one real number (e.g. ROAS or CPA change, how many of the 60 accounts made the switch) would make it a lot more credible.\n- \"I've billed for it\" in the first line assumes your agency used to sell high-volume testing. If that's not true, change the hook to \"My industry sells a lie.\" or use one of the alternatives above.\n- Alternative #3 assumes the drop was 20 \u2192 3. Only use it if that's accurate.\n"}]}, "haiku-1": {"cost_usd": 0.00389, "seconds": 15, "output_tokens": 2734, "files": [{"name": "linkedin_post.md", "text": "Most agencies sell \"high-volume creative testing.\"\n\nI think it mostly helps agencies bill more.\n\nWe looked at 60 client accounts. The brands that cut their testing from 20+ creatives a week down to 3 got better results after three months.\n\nWhy? Each creative got enough spend to actually teach them something. At 20+ a week, the budget gets sliced so thin that most ads never get a real read. You end up with a lot of activity and very little signal.\n\nMore tests felt like more learning. It was mostly noise.\n\nIf you run paid ads for a DTC brand, try this before your next testing sprint: fewer creatives, with enough budget behind each one to learn from.\n\nHow many creatives is your team testing a week, and how many of them have you actually learned from?\n\n#PaidAds #DTC\n"}]}, "haiku-2": {"cost_usd": 0.00346, "seconds": 12, "output_tokens": 2067, "files": [{"name": "linkedin_post.md", "text": "More creative testing isn't the answer for most DTC brands.\n\nWe looked at 60 client ad accounts. The brands that cut their testing from 20+ creatives a week down to 3 got better results after three months.\n\nWhy? Fewer creatives meant each one got enough spend to actually teach us something.\n\nMost agencies sell \"high-volume creative testing.\" It sounds rigorous. I think it mostly helps agencies bill more.\n\nI'm not saying testing doesn't matter. I'm saying the number of tests matters less than whether each one gets enough budget to produce a real answer.\n\nTwo questions for anyone running paid social:\n\n- How many new creatives did you launch last week?\n- How many of them got enough spend to call a winner?\n\nIf the second number is much smaller than the first, you may be paying for activity, not learning.\n\n#PerformanceMarketing #DTC #PaidSocial\n"}]}, "haiku-3": {"cost_usd": 0.00359, "seconds": 13, "output_tokens": 2259, "files": [{"name": "linkedin_post.md", "text": "We cut our clients' creative testing from 20+ ads a week to 3.\n\nTheir results got better.\n\nAcross 60 client accounts, the DTC brands that made this change were stronger after three months. The reason is simple: when you spread spend across 20+ creatives, none of them gets enough budget to teach you anything. At 3, each one does.\n\nMost agencies sell high-volume creative testing. It's an easy pitch: more concepts, more shots on goal, a longer deliverables list. I think it mostly helps agencies bill more.\n\nOur job isn't to produce the most ads. It's to give each idea enough spend to prove itself or fail clearly.\n\nIf you run paid social for a DTC brand, check this before your next creative sprint: how many of last month's ads got enough spend to give you a real answer?\n\nI'm the founder of a 15-person paid ads agency. I'd like to hear whether others have seen the same pattern, or whether I'm missing something.\n\n#PaidSocial #DTC #PerformanceMarketing\n"}]}}, "verdicts": [{"vs": "opus", "pair": "1", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "haiku+skill", "why": "A preserves the observation and makes budget allocation the takeaway. B has a stronger hook, but prescribes three creatives broadly and adds unsupported claims about agency revenue, creative quality, and subsequent improvements."}, {"vs": "opus", "pair": "1", "judge": "gpt", "order": "opus shown first", "winner": "haiku+skill", "why": "B preserves the observation and makes budget per creative the decision criterion. A has a stronger hook and founder voice, but broadly prescribes three creatives and adds unsupported claims about repeated patterns and subsequent creatives improving."}, {"vs": "opus", "pair": "1", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "tie", "why": "Both accurately capture the facts, insight, and criticism, and are ready-to-publish posts."}, {"vs": "opus", "pair": "1", "judge": "deepseek", "order": "opus shown first", "winner": "tie", "why": "Both accurately capture all facts, the counterintuitive insight, the explanation, and the critical edge. Both are polished, publishable posts."}, {"vs": "haiku", "pair": "1", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "haiku", "why": "B preserves the exact testing numbers and delivers a tighter, actionable founder post. Both explain budget dilution and retain credible criticism, but neither opens with the specific counterintuitive result. A adds repetitive explanation without strengthening the observation."}, {"vs": "haiku", "pair": "1", "judge": "gpt", "order": "haiku shown first", "winner": "haiku", "why": "A preserves the exact three-creatives observation; B unnecessarily changes it to \u201cabout 3.\u201d Both explain spending and learning, retain credible insider criticism, and avoid universal prescriptions. Both could improve their openings by immediately foregrounding fewer creatives producing better results."}, {"vs": "haiku", "pair": "1", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "tie", "why": "Both accurately capture the facts, insight, and criticism, and are polished, compelling posts."}, {"vs": "haiku", "pair": "1", "judge": "deepseek", "order": "haiku shown first", "winner": "tie", "why": "Both accurately capture the facts, insight, and criticism, and are polished, ready posts."}, {"vs": "opus", "pair": "2", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "haiku+skill", "why": "A preserves the observation and delivers a publishable, proportionate post. B adds unsupported claims about retainers and best-performing clients, initially prescribes three creatives, and includes alternate openings that require editing."}, {"vs": "opus", "pair": "2", "judge": "gpt", "order": "opus shown first", "winner": "haiku+skill", "why": "B preserves the facts, explains the budget mechanism, and delivers a publishable insider critique. A initially prescribes three creatives too broadly despite its later caveat, and includes alternate openings and editing instructions that must be removed before publishing."}, {"vs": "opus", "pair": "2", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "haiku+skill", "why": "Both are strong, but A is a cleaner, ready-to-publish post. B includes drafting notes, which violates C6."}, {"vs": "opus", "pair": "2", "judge": "deepseek", "order": "opus shown first", "winner": "tie", "why": "Both accurately capture all facts, the insight, explanation, and criticism in a polished, ready-to-publish post."}, {"vs": "haiku", "pair": "2", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "haiku+skill", "why": "A opens with the specific counterintuitive result and keeps its scope tied to observed accounts, though its noise-based decisions claim is unsupported. B preserves the facts more cleanly but overgeneralizes to most DTC brands and has a less specific hook."}, {"vs": "haiku", "pair": "2", "judge": "gpt", "order": "haiku shown first", "winner": "tie", "why": "Both deliver compelling, publishable posts with the core numbers, mechanism, and billing critique. A overgeneralizes to most DTC brands; B adds unsupported claims that most ads lacked budget and decisions were based on noise."}, {"vs": "haiku", "pair": "2", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "tie", "why": "Both accurately convey the facts, insight, and criticism in a compelling, ready-to-publish post."}, {"vs": "haiku", "pair": "2", "judge": "deepseek", "order": "haiku shown first", "winner": "tie", "why": "Both accurately capture all facts, the insight, and the critical edge, delivering polished, ready-to-publish posts."}, {"vs": "opus", "pair": "3", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "haiku+skill", "why": "A preserves the evidence, bounds the claim, and delivers a publishable insider critique. B invents personal billing history and unsupported claims in alternative hooks, overstates conclusions, and includes drafting material requiring removal. Both could foreground the fewer-creatives paradox sooner."}, {"vs": "opus", "pair": "3", "judge": "gpt", "order": "opus shown first", "winner": "haiku+skill", "why": "B preserves the evidence, limits its claims, and is publishable, though its opening is generic. A invents a billing confession and unsupported claims in alternative hooks, overgeneralizes the learning mechanism, and includes drafting material requiring removal."}, {"vs": "opus", "pair": "3", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "tie", "why": "Both accurately reflect all facts, compellingly frame the insight, and are ready to publish."}, {"vs": "opus", "pair": "3", "judge": "deepseek", "order": "opus shown first", "winner": "opus", "why": "A's opening is more compelling, and its criticism of agency incentives is sharper and more credible."}, {"vs": "haiku", "pair": "3", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "haiku+skill", "why": "A preserves the observed subgroup and explicitly limits the evidence. B has a stronger hook, but invents agency-led action with \u201cWe cut\u201d and implies broader results and greater causal certainty than the notes support."}, {"vs": "haiku", "pair": "3", "judge": "gpt", "order": "haiku shown first", "winner": "haiku+skill", "why": "B better distinguishes observation from proof and offers a practical spending check. A has the stronger hook, but its categorical claim that three creatives each receive enough spend overstates what the notes establish."}, {"vs": "haiku", "pair": "3", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "tie", "why": "Both posts accurately capture the facts, insight, and criticism, and are polished, compelling, and ready to publish."}, {"vs": "haiku", "pair": "3", "judge": "deepseek", "order": "haiku shown first", "winner": "tie", "why": "Both accurately reflect all facts, present the insight compellingly, and are ready-to-publish posts with appropriate edge."}]}, {"id": "pov-3", "family": "pov-post", "prompt": "Write a LinkedIn post about the topic in topic.md. Make it something people argue about in the comments.", "input": "topic.md", "skill": "wes-kao-spiky-pov", "input_text": "Me: CTO at a 60-person fintech.\nTopic: we removed all internal \"AI coding tool\" mandates and productivity dashboards. Instead each team picks its own tools and we measure lead time from ticket to production.\nResult over 6 months: lead time down 18%, two teams use almost no AI tools and are among the fastest, one team that used AI the most had the most rollbacks.\n", "rubric": [{"id": "c1", "criterion": "Accurately represents the supplied facts without inventing evidence.", "full_marks_when": "Preserves the context of a CTO at a 60-person fintech, removal of internal AI coding tool mandates and productivity dashboards, team choice of tools, ticket-to-production lead time measurement, and an 18% lead-time reduction over six months. Accurately describes two teams using almost no AI as among the fastest and one team using AI the most as having the most rollbacks. Any included numbers match the input.", "zero_when": "Misstates the numbers or results, equates an 18% lead-time reduction with an 18% productivity increase, or invents metrics, savings, tool names, or team rankings."}, {"id": "c2", "criterion": "Builds the post around a specific, disputable position.", "full_marks_when": "Makes a clear claim readers can reasonably challenge, such as judging engineering teams by delivery outcomes rather than mandated AI adoption, and grounds it in the company's experience.", "zero_when": "Offers a generic celebration of AI, a neutral recap, or vague advice with no identifiable position to argue against."}, {"id": "c3", "criterion": "Distinguishes observed results from causal proof.", "full_marks_when": "Uses the six-month results to challenge assumptions about AI adoption without claiming that removing mandates caused the improvement, that heavy AI use caused rollbacks, or that AI tools universally hurt performance.", "zero_when": "Presents the observations as proof that AI causes worse engineering outcomes or that removing mandates necessarily improves delivery."}, {"id": "c4", "criterion": "Sounds like a credible CTO sharing an actual management decision.", "full_marks_when": "Uses a natural first-person voice, explains the shift from enforcing tool usage to giving teams choice and measuring delivery, and conveys accountability for the decision without blaming individual teams.", "zero_when": "Reads like an outsider's essay, vendor promotion, or fabricated personal story, or mocks the team with the most rollbacks."}, {"id": "c5", "criterion": "Invites substantive disagreement in the comments.", "full_marks_when": "Creates a clear tension around mandates, team autonomy, or what engineering leaders should measure, and ends with a pointed question or challenge that invites readers to defend opposing positions.", "zero_when": "Relies on insults, outrage bait, or a generic 'Thoughts?' prompt without giving readers a concrete issue to debate."}, {"id": "c6", "criterion": "Delivers a publishable LinkedIn post.", "full_marks_when": "Provides one cohesive, ready-to-paste post with an arresting opening, readable short paragraphs, concrete supporting observations, and a focused ending, without prefatory explanation or drafting instructions.", "zero_when": "Provides an outline, advice about writing the post, multiple unfinished alternatives, or a dense essay requiring substantial editing before publication."}], "runs": {"haiku-skill-1": {"cost_usd": 0.00689, "seconds": 26, "output_tokens": 4758, "files": [{"name": "linkedin-post.md", "text": "Stop measuring how much your engineers use AI. Measure how long a ticket takes to reach production.\n\nAs CTO of a 60-person fintech, I removed our internal AI coding tool mandates and our productivity dashboards. Each team now picks its own tools. The one thing we track is lead time from ticket to production.\n\nSix months later, lead time is down 18%.\n\nThe results people will argue about:\n\n- The two teams using almost no AI tools are among our fastest.\n- The team that used AI the most had the most rollbacks.\n\nI can't tell you AI caused those rollbacks. That team's work may have been riskier. I also can't attribute the 18% to this one decision, because plenty else changed in six months. What I can tell you is what the mandate was measuring: activity. Adoption dashboards reward looking modern. They say nothing about whether the change reached production and stayed there.\n\nMy position: usage targets for AI tools are a vanity metric. If you're a leader, replace them with the delivery outcomes you already trust, like lead time and rollbacks. Then let those outcomes tell you which tools are worth using.\n\nThe strongest objection is that without mandates, teams fall behind on tooling. That's a real risk, and six months isn't long enough to rule it out. I'd want to see the numbers at a year before calling it settled.\n\nWhich number is your org rewarding right now: the activity you can count, or the delivery you care about?\n"}]}, "haiku-skill-2": {"cost_usd": 0.00705, "seconds": 26, "output_tokens": 4986, "files": [{"name": "linkedin_post.md", "text": "Stop mandating AI coding tools. Measure what ships.\n\nWe removed every internal AI coding tool mandate and every productivity dashboard that tracked tool usage. Each team now picks its own tools. We measure one thing per team: lead time from ticket to production.\n\nSix months later, lead time is down 18%.\n\nTwo teams that use almost no AI tools are among our fastest. The team that used AI the most had the most rollbacks.\n\nMy read: adoption metrics measure activity, not delivery. A mandate tells people what to use. A usage dashboard tells them what to look busy doing. Neither tells you whether a change reached production cleanly, and that is what the business pays for.\n\nI'm not claiming AI slows engineers down. One company, six months, and a few teams can't prove that. The narrower claim is that we were tracking the wrong number. Our sample is small, but it points the same way.\n\nThe objection I expect: the fast teams are fast for other reasons, like seniority or a simpler codebase. Maybe. But that is the argument. Whatever drives lead time, tool usage was not the thing we were measuring.\n\nIf you want to try this, here is what we run:\n- Two numbers per team: ticket-to-production lead time and rollbacks.\n- Teams choose their own tools, with no adoption targets.\n- When rollbacks rise, we ask what changed in the process, not which tools people used.\n\nIs anyone running an AI mandate with an outcome metric attached? I'd like to see your numbers, especially if they contradict ours.\n"}]}, "haiku-skill-3": {"cost_usd": 0.00723, "seconds": 24, "output_tokens": 4761, "files": [{"name": "linkedin-post.md", "text": "Our AI coding tool mandate was measuring the wrong thing.\n\nSix months ago we removed every internal AI coding tool mandate and every productivity dashboard that tracked AI usage. Each team now picks its own tools. The metric we track is lead time from ticket to production.\n\nOver those six months, lead time dropped 18%.\n\nThe part that bothers people: two of our teams use almost no AI tools, and they are among our fastest. The team that used AI the most had the most rollbacks.\n\nMy read is that usage is an activity metric. Mandates and dashboards reward people for showing they use the tool, and that pressure can push effort toward looking productive instead of shipping safely. Lead time to production and rollbacks measure what the business actually depends on. Measure those, and let teams decide how they get there.\n\nHere is the objection I would raise against myself. Two fast teams and one high-usage team is a handful of data points, not a controlled experiment. The fast teams may have cleaner codebases or easier work. The high-usage team may have taken on riskier changes. The 18% also came alongside other changes, and I cannot cleanly separate them. So I am not claiming AI slows teams down. I am claiming that mandating usage is a poor way to find out whether it helps.\n\nIf you run an engineering org, here is what I would do:\n\n1. Remove AI-usage targets from team goals.\n2. Put the outcome metrics you already collect, lead time and rollbacks, on the dashboard instead.\n3. When a team wants a new tool, let them try it, then check the outcome metrics after a quarter.\n\nWhere am I wrong? If you ran this experiment the other way and mandates worked, tell me what you measured.\n"}]}, "opus-1": {"cost_usd": 0.07219, "seconds": 20, "output_tokens": 1475, "files": [{"name": "linkedin-post.md", "text": "Six months ago we deleted our AI productivity dashboard.\n\nNo more \"% of code AI-assisted.\" No more seat-utilization charts. No more mandate that every engineer use the approved coding assistant.\n\nI'm CTO at a 60-person fintech. Here's what we did instead:\n\n\u2192 Each team picks its own tools. AI, no AI, whatever.\n\u2192 We measure one thing: lead time from ticket to production.\n\nSix months later:\n\n\ud83d\udcc9 Lead time is down 18% across engineering.\n\ud83d\udc22 Two teams barely use AI tools. Both are among our fastest.\n\ud83d\udd01 The team that used AI the most had the most rollbacks.\n\nBefore anyone screenshots that last line for a \"see, AI is overhyped\" post, let me be clear about what I'm NOT saying.\n\nI'm not saying AI tools make teams slower. Our sample is tiny. The heavy-AI team also owns some of our messiest legacy code. Correlation, six months, one company.\n\nHere's what I AM saying:\n\n1\ufe0f\u20e3 We were measuring the input instead of the outcome. \"How much AI did you use\" is a vanity metric. Nobody's customer has ever cared how a line of code was written.\n\n2\ufe0f\u20e3 Mandates made people perform adoption. Once engineers stopped being watched on usage, they used AI where it actually helped and dropped it where it didn't.\n\n3\ufe0f\u20e3 Typing was never our bottleneck. Reviews, unclear tickets, flaky tests, and waiting on other teams were. Generating code faster into a slow pipeline just makes a bigger queue \u2014 and, apparently, more rollbacks.\n\n4\ufe0f\u20e3 Trusting teams to choose their tools is cheaper than any dashboard we could build.\n\nThe uncomfortable part for leaders: if you need a usage dashboard to prove your AI investment is working, maybe it isn't. Or maybe it is \u2014 and you're measuring the wrong thing to see it.\n\nSo I'll put it to you:\n\nIs mandating AI coding tools good leadership \u2014 making sure your team doesn't fall behind? Or is it a manager's way of looking busy with someone else's keyboard?\n\nAnd if you track AI usage at your company: what decision has that number ever actually changed?\n\n#EngineeringLeadership #AI #SoftwareEngineering #DevOps #Fintech #CTO\n"}]}, "opus-2": {"cost_usd": 0.07341, "seconds": 19, "output_tokens": 1527, "files": [{"name": "linkedin-post.md", "text": "We stopped measuring AI adoption. Our engineering got faster.\n\nSix months ago I killed two things at our company (60 people, fintech):\n\n1. Every internal mandate to use \"AI coding tools\"\n2. Every dashboard tracking AI usage, suggestions accepted, or lines generated\n\nWe replaced them with one rule: each team picks its own tools.\nAnd one metric: lead time from ticket to production.\n\nWhat happened over six months:\n\n\u2192 Lead time dropped 18% across engineering.\n\u2192 Two teams use almost no AI tools. Both are among our fastest.\n\u2192 The team that used AI the most had the most rollbacks.\n\nBefore anyone says it: no, this doesn't prove AI makes teams worse. It's one company and six months of data, and there are a dozen confounders I can't rule out. The heavy-AI team also owns some of our gnarliest code.\n\nBut it does prove something I'm now willing to say out loud:\n\n**\"How much AI are your engineers using?\" is the wrong question.**\n\nIt's an input metric. We'd never judge a team by how many hours they sit in an IDE or how many Jira tickets they open. Yet half the industry is now judging teams by prompts, completions, and \"% of code written by AI.\"\n\nMeasure activity and you get activity. You get engineers who accept suggestions to make a chart go up. You get code that ships fast and comes back faster. In fintech, a rollback isn't a rounding error. It's a customer's money.\n\nWhat I actually care about:\n\u2022 How fast does a real change reach a real customer?\n\u2022 Does it stay there?\n\nIf a team gets there with Claude, Copilot, Cursor, vim and a coffee, or a whiteboard, I genuinely don't care.\n\nSome of my CTO peers think this is naive. Their view: if you don't push adoption, you fall behind, and the teams opting out are just slower to learn. Maybe they're right. Maybe in 12 months my low-AI teams hit a wall.\n\nBut I'd rather find that out from lead time and rollback rate than from a usage dashboard that can only ever tell me the number went up.\n\nSo, honest question for engineering leaders:\n\nAre AI usage mandates a smart bet on the future, or a way for leadership to feel like it's doing something?\n\nTell me where I'm wrong. \ud83d\udc47\n\n#EngineeringLeadership #AI #DeveloperProductivity #Fintech #CTO\n"}]}, "opus-3": {"cost_usd": 0.07723, "seconds": 21, "output_tokens": 1679, "files": [{"name": "linkedin-post.md", "text": "Six months ago we deleted our AI coding mandate.\n\nNo more \"every engineer must use [tool].\"\nNo more dashboards tracking AI suggestions accepted, prompts per day, or % of code AI-generated.\n\nWe replaced all of it with one rule and one number:\n\n\u2192 Each team picks its own tools.\n\u2192 We measure lead time from ticket to production. That's it.\n\nI'm CTO at a 60-person fintech. Here's what happened:\n\n\ud83d\udcc9 Lead time across the org dropped 18%.\n\n\ud83d\udc22\u27a1\ufe0f\ud83d\udc07 Two teams use almost no AI tools. They're among our fastest.\n\n\ud83d\udd01 The team that used AI the most had the most rollbacks.\n\nBefore the comments section catches fire, let me be precise about what I'm NOT saying:\n\n\u2022 I'm not saying AI tools make engineers slower.\n\u2022 I'm not saying the rollbacks were caused by AI. One team, six months, lots of variables.\n\u2022 I'm not saying the low-AI teams would be fast at any company. They're fast here, on their codebases, with their habits.\n\nWhat I AM saying:\n\n1\ufe0f\u20e3 \"AI adoption\" is an input metric. We were managing it like an outcome. Nobody's customer cares how many lines Copilot wrote.\n\n2\ufe0f\u20e3 Mandates measure compliance, not productivity. When you track usage, you get usage. We got plenty of it. We just couldn't show it was making anything ship faster.\n\n3\ufe0f\u20e3 Rollbacks are part of the cost. A metric that counts \"code produced\" but ignores \"code reverted\" isn't measuring productivity. It's measuring typing.\n\n4\ufe0f\u20e3 Senior engineers know their bottlenecks better than a vendor dashboard does. For one of our fast teams, the bottleneck was never writing code. It was review, test environments, and waiting on compliance sign-off. No autocomplete fixes that.\n\nHere's the uncomfortable part for a lot of leadership teams right now:\n\nIf you're forcing AI tools on engineers and reporting adoption % to your board, you might be measuring how well you sold the initiative, not whether it worked.\n\nAnd here's the uncomfortable part for the AI skeptics:\n\nOur org still got faster. Most teams kept using AI. They just use it where it actually helps, because nobody's grading them on how often they open it.\n\nSo, genuine question for the engineering leaders here:\n\nIs mandating AI tools the right move to force a learning curve that pays off later?\n\nOr is it the 2026 version of measuring developers by lines of code?\n\nI have a strong opinion. I want to hear yours. \ud83d\udc47\n\n#EngineeringLeadership #AI #DeveloperProductivity #DORA #Fintech\n"}]}, "haiku-1": {"cost_usd": 0.0031, "seconds": 12, "output_tokens": 1906, "files": [{"name": "linkedin_post.md", "text": "We removed every internal AI coding tool mandate at our 60-person fintech. No usage targets. No productivity dashboards. No \"AI-first\" memo.\n\nEach team picks its own tools. The only thing we measure is lead time: how long a ticket takes to reach production.\n\nSix months in:\n\n- Lead time is down 18%.\n- Two teams use almost no AI tooling. They are among our fastest.\n- The team that used AI the most had the most rollbacks.\n\nI'm not claiming AI slows people down. Six months and a handful of teams is not a study, and those teams work on different systems with different risk profiles. Confounders everywhere.\n\nBut it changed what I think we should be measuring.\n\nMost AI rollouts measure adoption: seats, prompts, percentage of code generated. Those numbers climb almost by definition. They tell you people are using the tool. They don't tell you whether customers get working software faster and more safely.\n\nThe rollback result is the one that stuck with me. Producing code is now cheap. Knowing whether that code is safe to ship is still expensive, and it's the part a usage dashboard can't see.\n\nSo here's my question for other engineering leaders: what is your AI rollout actually measured on? If the answer is usage, you may be optimizing for the wrong thing.\n\nCurious where people think I'm wrong.\n"}]}, "haiku-2": {"cost_usd": 0.00306, "seconds": 11, "output_tokens": 1863, "files": [{"name": "linkedin_post.md", "text": "We removed every internal \"AI coding tool\" mandate. No adoption targets. No productivity dashboards.\n\nEach team picks its own tools. We measure one thing: lead time from ticket to production.\n\nSix months later:\n\n- Lead time is down 18%.\n- Two of our fastest teams use almost no AI tools.\n- The team that used AI the most had the most rollbacks.\n\nI'm not claiming AI tools are bad. This is one company, six months, and I don't yet know why the rollbacks happened. It could be the tool, the kind of work that team took on, or the team itself.\n\nBut I do think \"how much AI are we using?\" is the wrong thing to measure. Mandates and dashboards measure activity. Activity is easy to inflate with more generated code, more PRs, and more lines changed. None of that is the same as shipping working software faster.\n\nDashboards also teach people what to optimize for. Once engineers see their AI usage on a chart, that becomes the goal, and the goal stops being \"ship well.\"\n\nAs a CTO, the uncomfortable part is this: the fastest teams were the ones with the least reason to perform for me.\n\nSo my question, and I'd like the honest answer rather than the vendor one:\n\nIf your company mandated AI usage tomorrow, what would it be optimizing for, and how would you know if it was making you slower?\n"}]}, "haiku-3": {"cost_usd": 0.00293, "seconds": 9, "output_tokens": 1668, "files": [{"name": "linkedin_post.md", "text": "We removed every internal \"AI coding tool\" mandate and killed the productivity dashboards.\n\nSix months later, our lead time from ticket to production is down 18%.\n\nHere's the setup. Each team picks its own tools. No adoption targets. No AI usage scorecards. We measure one thing: how long it takes a ticket to reach production.\n\nHere's what surprised us:\n\n- Two of our fastest teams use almost no AI tools.\n- The team that used AI the most had the most rollbacks.\n\nI'm not claiming AI makes people slower. This is one company, a small sample, and plenty of confounders. But I am claiming that tracking AI usage was measuring the wrong thing. It's a number that looks like progress, and it told us nothing about whether we shipped faster or safer.\n\nLead time and rollbacks are what the business actually cares about. Tool usage is a vanity metric.\n\nMost engineering leaders I talk to are mandating AI adoption and tracking seats, prompts, and \"percent of code generated.\" Ask yourself what those numbers would look like if they were pointing you in the wrong direction.\n\nSo I'll ask you:\n\nIs the rollback pattern a tooling problem, a review problem, or just noise? I genuinely don't know yet. Tell me where I'm wrong.\n"}]}}, "verdicts": [{"vs": "opus", "pair": "1", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "haiku+skill", "why": "A grounds its argument in supplied results and separates observation from causation, though it invents other concurrent changes. B invents dashboard metrics, legacy-code context, bottlenecks, behavioral changes, and cost claims, while implying AI increased rollbacks."}, {"vs": "opus", "pair": "1", "judge": "gpt", "order": "opus shown first", "winner": "haiku+skill", "why": "B makes a clear, debatable management argument while separating observation from causation. Its claim that other things changed is unsupported. A invents dashboard metrics, legacy-code context, bottlenecks, behavioral changes, and cost benefits, then undermines its causal caveat."}, {"vs": "opus", "pair": "1", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "tie", "why": "Both posts accurately present the facts, take a disputable position, distinguish correlation from causation, sound credible, invite debate, and are publishable."}, {"vs": "opus", "pair": "1", "judge": "deepseek", "order": "opus shown first", "winner": "opus", "why": "A more effectively invites debate with a direct, challenging question, while B's prompt is slightly less provocative."}, {"vs": "haiku", "pair": "1", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "haiku+skill", "why": "A states a sharper disputable position and acknowledges the strongest opposing argument. Both preserve the numerical results and avoid causal claims, but invent context: A asserts other changes occurred; B asserts differing risk profiles and no AI-first memo."}, {"vs": "haiku", "pair": "1", "judge": "gpt", "order": "haiku shown first", "winner": "haiku+skill", "why": "B states the CTO context explicitly and offers a sharper position with a credible opposing argument. Both preserve the numbers and avoid causal overclaims, but invent context: A asserts differing risk profiles; B asserts other changes occurred."}, {"vs": "haiku", "pair": "1", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "tie", "why": "Both posts accurately present the facts, take a disputable position, distinguish results from proof, sound credible, invite debate, and are publishable."}, {"vs": "haiku", "pair": "1", "judge": "deepseek", "order": "haiku shown first", "winner": "tie", "why": "Both accurately represent facts, take a disputable position, distinguish results from proof, sound credible, invite disagreement, and are publishable."}, {"vs": "opus", "pair": "2", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "haiku+skill", "why": "A grounds its argument more faithfully and separates observation from causation. It omits company context and invents operational details, but B fabricates dashboard metrics, codebase complexity, peer opinions, and industry prevalence while implying unsupported causal mechanisms."}, {"vs": "opus", "pair": "2", "judge": "gpt", "order": "opus shown first", "winner": "haiku+skill", "why": "B is more focused and invents less. Both fabricate operational details: A adds codebase complexity and peer conversations; B adds rollback measurement and review procedures. B also omits the CTO and 60-person fintech context. Both invite substantive debate."}, {"vs": "opus", "pair": "2", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "tie", "why": "Both accurately present the facts, take a disputable position, distinguish results from proof, sound credible, invite debate, and are publishable."}, {"vs": "opus", "pair": "2", "judge": "deepseek", "order": "opus shown first", "winner": "opus", "why": "A's ending question is more pointed and disputable, directly challenging the rationale for mandates."}, {"vs": "haiku", "pair": "2", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "haiku", "why": "B offers a clearer CTO voice and sharper debate prompt. Both omit the 60-person fintech context. A invents operating practices and contradicts its one-metric claim with two metrics; B preserves the supplied results and acknowledges causal uncertainty."}, {"vs": "haiku", "pair": "2", "judge": "gpt", "order": "haiku shown first", "winner": "haiku", "why": "A stays closer to the evidence and clearly limits causal claims. Both omit the 60-person fintech context. B invents rollback measurement and a rollback-review policy, and contradicts its own claim that teams measure one thing."}, {"vs": "haiku", "pair": "2", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "tie", "why": "Both accurately present the facts, take a disputable position, distinguish results from proof, sound credible, invite debate, and are publishable."}, {"vs": "haiku", "pair": "2", "judge": "deepseek", "order": "haiku shown first", "winner": "tie", "why": "Both posts accurately represent the facts, take a disputable position, distinguish results from proof, sound credible, invite debate, and are publishable."}, {"vs": "haiku", "pair": "3", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "haiku+skill", "why": "A invites a clearer debate about mandates and opposing evidence. Both omit the CTO/60-person fintech context and invent details: A asserts concurrent changes; B fabricates conversations with engineering leaders. B\u2019s closing question focuses more on diagnosis than management disagreement."}, {"vs": "haiku", "pair": "3", "judge": "gpt", "order": "haiku shown first", "winner": "tie", "why": "Both offer credible, disputable posts with causal caveats and substantive debate prompts. Both omit the CTO/60-person fintech context and invent evidence: A adds conversations with other leaders; B asserts concurrent changes absent from the input."}, {"vs": "haiku", "pair": "3", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "tie", "why": "Both posts accurately represent the facts, take a disputable position, distinguish results from proof, sound credible, invite disagreement, and are publishable."}, {"vs": "haiku", "pair": "3", "judge": "deepseek", "order": "haiku shown first", "winner": "tie", "why": "Both posts flawlessly meet all criteria, presenting the same strong, disputable position with equal clarity and professionalism."}, {"vs": "opus", "pair": "3", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "haiku+skill", "why": "A omits the CTO/company context and invents concurrent changes, but B invents dashboard metrics, team bottlenecks, continued AI usage, and prior findings. Both invite substantive debate and avoid causal overclaims; A is more faithful."}, {"vs": "opus", "pair": "3", "judge": "gpt", "order": "opus shown first", "winner": "haiku+skill", "why": "B makes a grounded, disputable management argument with clear causal restraint. It omits the CTO/company context and invents concurrent changes. A invents dashboard metrics, team bottlenecks, and continued adoption patterns, undermining its credibility."}, {"vs": "opus", "pair": "3", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "tie", "why": "Both posts accurately present the facts, take a disputable position, distinguish results from proof, sound credible, invite debate, and are publishable."}, {"vs": "opus", "pair": "3", "judge": "deepseek", "order": "opus shown first", "winner": "opus", "why": "A more directly invites debate with a pointed, provocative question, while B's prompt is slightly less engaging."}]}, {"id": "neg-2", "family": "negotiation", "prompt": "Help me prepare for a salary negotiation. Situation in offer.md.", "input": "offer.md", "skill": "costa-negotiation-dynamics", "input_text": "I'm a senior data engineer. Offer from a Series C company: USD 175k base, 0.05% equity (4-year vest), USD 10k signing bonus. They want an answer by Friday.\nMy current job: USD 160k, I'm not desperate to leave. I have a second process at another company, final round next Wednesday.\nMarket data I found: senior data engineers in my city are around USD 170-200k base. The hiring manager said I was \"the strongest candidate they've seen\".\nI'd most like more base salary and a remote day per week.\n", "rubric": [{"id": "c1", "criterion": "Accurately uses the offer, current compensation, and supplied market data.", "full_marks_when": "Correctly identifies USD 175k base, 0.05% equity vesting over four years, and a USD 10k signing bonus versus USD 160k current base. States that the offered base increase is USD 15k, or 9.375%, and places USD 175k within the supplied USD 170\u2013200k market range. Keeps the signing bonus separate from recurring base pay.", "zero_when": "Misstates material inputs, makes incorrect compensation calculations, or treats the signing bonus as recurring salary."}, {"id": "c2", "criterion": "Turns the user's priorities into a concrete, defensible counteroffer.", "full_marks_when": "Recommends a specific base salary ask grounded in the supplied market range, with a brief rationale and room to negotiate, and explicitly requests one remote day per week. Prioritizes those two requests over equity or signing bonus.", "zero_when": "Provides no actionable salary ask, omits the remote day, or centers the negotiation on benefits the user did not prioritize."}, {"id": "c3", "criterion": "Uses the user's bargaining position credibly without overstating leverage.", "full_marks_when": "Uses the hiring manager's strong endorsement and the user's ability to remain in their current job to support a confident, collaborative negotiation. Accurately describes the second opportunity as an ongoing process, not an offer, and avoids claiming the company must meet the counteroffer.", "zero_when": "Encourages inventing a competing offer, exaggerating market evidence, bluffing about willingness to walk away, or issuing an unsupported ultimatum."}, {"id": "c4", "criterion": "Addresses the Friday deadline and next Wednesday interview with a workable sequence.", "full_marks_when": "Advises contacting the company before Friday to negotiate and request a specific extension until after next Wednesday, with enough time to assess the second process. Provides a fallback if the extension is refused and recognizes that a final interview does not guarantee an offer.", "zero_when": "Ignores the timing conflict, recommends waiting silently past Friday, assumes the second offer will arrive Wednesday, or recommends accepting merely to buy time."}, {"id": "c5", "criterion": "Provides language the user can use directly in the negotiation.", "full_marks_when": "Includes a concise email or speaking script that expresses interest, makes the proposed base and weekly remote-day requests explicitly, grounds them in the user's situation, and handles the deadline or supplies a separate extension message. The tone is professional and confident without fabricated achievements or commitments.", "zero_when": "Offers only abstract advice, leaves the user to invent the actual requests, or supplies an aggressive, misleading, or materially incomplete script."}, {"id": "c6", "criterion": "Helps the user evaluate concessions and secure a clear final package.", "full_marks_when": "Explains how to respond if base salary flexibility is limited while preserving the priority of recurring pay and the remote day. Encourages setting a private acceptance threshold based on staying in the current job, obtaining agreed compensation and remote arrangements in writing, and treating startup equity as uncertain rather than assigning a cash value from 0.05% alone.", "zero_when": "Treats signing bonus or uncertain equity as automatically equivalent to recurring salary, invents an equity valuation, or advises accepting vague promises about pay or remote work."}], "runs": {"haiku-skill-1": {"cost_usd": 0.01551, "seconds": 101, "output_tokens": 18028, "files": [{"name": "salary-negotiation-brief.md", "text": "# Salary Negotiation Brief: Series C Senior Data Engineer Offer\n\n**Mode:** Preparation. Round record pending.\n\n**Date assumptions:** Today is Thu 8 Oct 2026, so the offer deadline is Fri 9 Oct. \"Next Wednesday\" is read as Wed 14 Oct. If you meant 21 Oct, the timing gap is larger.\n\n## 1. Assignment and decision\n\n- **Decision:** Accept, negotiate or decline: USD 175k base, 0.05% equity over 4 years, USD 10k signing bonus.\n- **Your priorities, in order:** base salary, then one remote day per week. Equity and signing bonus were not named as priorities.\n- **Parties and authority:** Unresolved. You know the hiring manager. You do not know who approves comp exceptions.\n- **Timing problem:** The offer deadline (Fri 9 Oct) falls before your other final round (Wed 14 Oct). Your leverage and information are greatest after that round, so the first task is to extend the deadline.\n- **Situation:** A single offer with several linked terms, in a live two-process search. Base is the dominant issue.\n\n## 2. Relationship state\n\n| Category | Evidence | Consequence |\n| --- | --- | --- |\n| Prior agreement | None | No commitments yet |\n| Your unilateral decisions | None yet | Avoid saying yes before the package is in writing |\n| Company's decisions | Offer at 175k base (stated) | Opening position from the company; band unknown |\n| External | Market range 170\u2013200k, from your own research (source unverified). Other process ongoing. | Anchor is market, not your current salary |\n| Process and information | Hiring manager called you \"the strongest candidate they've seen\" (reported, not verified). Friday deadline (firmness unknown). | Signal of interest; not a promise of flexibility |\n\n## 3. Critical unknowns, ranked\n\n| Priority | Unknown | Question to ask | What the answer changes |\n| --- | --- | --- | --- |\n| High | Is 175k the top of the approved band for this level? | \"What is the band for this level, and where does 175k sit in it?\" | If 175k is the cap, base movement needs a level change or exception. Shift to other terms. |\n| High | Is the Friday deadline firm, and why? | \"Is Friday a hard deadline? What is driving it?\" | How hard to push for an extension |\n| High | Remote policy: company-wide, team norm or manager discretion? | \"Is one remote day a week a policy here, or something we can agree for this role?\" | Whether remote is a term you can write in or a request |\n| High | Your walk-away and your remote must-have | Your decision, not the company's | Sets the limit (section 4). Unresolved until you answer. |\n| High | The other process: expected pay range, and whether its final round or decision lands before Friday | Your own check with that recruiter | Whether you have real leverage or only a process |\n| Medium-high | Verified market range for senior data engineers at this stage in your city | Two or three independent sources (salary sites, recruiters, peers) | Opening and target. Your 170\u2013200k range is unverified. |\n| Medium-high | Equity: post-money valuation, last preferred price, fully diluted share count, option or RSU type, strike price, vesting cliff, post-departure exercise window | Ask for the figures and the option agreement | Whether equity is a usable trade chip. Check tax treatment with an advisor. |\n| Medium | Signing bonus: clawback period and payment timing | Ask directly | Real value of the 10k |\n\n## 4. Positions\n\n- **Limit:** **Unresolved. This is your call.** Inputs: the minimum base that makes leaving a 160k job worth the startup risk; a discounted value for illiquid, diluted equity; and what the other process could pay.\n- **What the offer is worth to you (arithmetic only):**\n  - Base over current: +15k (+9.4%).\n  - Year one including signing bonus: +25k (+15.6%). The bonus is one-time.\n  - Equity: paper value only, not included above.\n- **Target: 185\u2013190k base.** The midpoint of your range is 185k. 175k sits in the lower sixth of the range (about 17% of the way up from 170k to 200k). This target depends on the market range holding up under verification.\n- **Opening: 195k base, plus a written remote day, with the 10k signing bonus unchanged.** Defensible because 195k is near the top of the range you found, and it matches the seniority the hiring manager described. Present the hiring manager's comment as their view, not as leverage. If verified data tops out lower, reset the opening to the top of the verified range.\n- **Anchor warning:** Do not anchor on your current 160k. Anchor on the market range and the scope of the role.\n- **Likely company response:** \"175 is our number,\" or a small move to about 180k.\n- **Planned responses:**\n  - If 175k is the cap: ask for the package in section 6 (P2). Do not accept a base reduction-free offer without a reciprocal term.\n  - If they offer about 180k: counter at 185k, and accept only if the remote day is in writing.\n- **Credibility check:**\n  - Say \"I'm in a final round elsewhere,\" not \"I have another offer.\"\n  - Do not commit to a Friday decision unless you would accept on those terms.\n  - Do not threaten to walk away unless you would actually walk.\n\n## 5. Interests and issue map\n\n| Issue | Your interest and priority | Company interest (hypothesis) | Evidence | Possible difference | Feasible variable |\n| --- | --- | --- | --- | --- | --- |\n| Base | Highest; recurring | Budget discipline at Series C (unknown) | Your priority: stated. Company view: inferred. | Cash cost is recurring for them; you value it most | Base |\n| Remote day | Second priority; time and focus | Possibly team or in-office norms; low cash cost | Your priority: stated | They may give it more easily than base; the trade is cheap for them | One written remote day |\n| Signing bonus | Moderate; one-time | One-time cash, possibly easier to approve | Unknown | A one-time bonus does not replace recurring base; only useful as a supplement | Amount and clawback |\n| Equity | Not stated | Low cash cost to them | Unknown | Your valuation of illiquid equity is probably low | Exercise window, refresh grant |\n| Decision speed | Can offer a quick yes on good terms | Wants to close a candidate it rates highly | Hypothesis | Speed is a currency you can spend | Commitment date, once terms are in writing |\n\n## 6. Linked packages (illustrative numbers, to test against your limit)\n\n- **P1, base-led:** 188k base, one written remote day, 10k signing. Costs them +13k base. Why they might accept: remote is cheap for them and the manager wants to close. Hypothesis.\n- **P2, if base is truly capped at 175k:** 175k base, one written remote day, 20k signing, a base review at 6 months against stated criteria with a fixed date, and an extended post-departure exercise window if the equity is options. Why it could work for you: recurring growth is tied to a date. Weakness: a review is not a guarantee. Get the criteria in writing.\n- **Rule:** Do not accept one term while the linked term is still open. Confirm the whole package in writing before signing.\n\n## 7. Equity scenarios\n\nPaper value at exit = 0.05% \u00d7 company equity value, before dilution. The figure is illiquid and uncertain.\n\n- Illustrative only: at a 100M exit value, about 50k; at 500M, about 250k; at 1B, about 500k.\n- Future funding rounds will dilute the stake. Treat it as a lottery ticket unless the valuation evidence supports more.\n- Adverse case: no exit, or an exit below the preference stack. Equity is worth little. The decision should still hold on cash alone (+15k base over your current job, plus startup risk).\n\nI have no equity benchmark data, so I cannot say whether 0.05% is strong or weak for this stage and level.\n\n## 8. Timeline and actions\n\n1. **Today (Thu 8 Oct):** Send the extension request, and ask about the band and the remote policy. Draft below.\n2. **Thu\u2013Fri:** Verify the market range. Write down your floor and your remote must-have.\n3. **Fri 9 Oct (deadline):**\n   - If you got an extension: wait for the other process's outcome before final terms.\n   - If not: present P1, or give a conditional yes.\n4. **Wed 14 Oct (other final round):** Re-run this brief with the new information.\n\n**Draft extension request:**\n\n> Thank you for the offer. I'm excited about the role and the team, and I want to make this decision with full information. I'm also in late-stage interviews elsewhere, with a final round on [date]. Could we extend my decision date to [date]? I'd also like to understand the band for this level and whether one remote day a week is a policy or something we can agree for this role.\n\n**Draft base ask (after the extension):**\n\n> Based on what I see for senior data engineers here, roughly 170\u2013200k, and the scope I'd own, I'd like to be at 195k base. What's the most you can do on base?\n\n**If base is capped at 175k:**\n\n> If base is fixed, I'd like the rest of the package to move: one remote day written into the offer, signing bonus at 20k, and a base review at six months against criteria we agree now.\n\n## 9. Rehearsal: likely objections\n\n- **\"175k is our max.\"** Ask about level and band. Move to P2. Do not concede base alone.\n- **\"We need an answer Friday.\"** Ask for the extension. If refused: \"I can't decide on these terms by Friday without knowing the band and the remote policy.\"\n- **\"Other candidates are waiting.\"** Acknowledge it. Restate your interest and ask for a specific date.\n- **A quick verbal yes with a higher number.** Check the full package (signing clawback, written remote day, equity terms) before accepting.\n- **The other process gives an offer before Friday.** That is new information. Re-run the brief and compare the two packages on the same basis.\n\n**Largest plausible mistake:** Deciding on Friday on this offer alone, before the other process resolves. Or ending a call with an unconditional yes at 175k because the deadline feels hard. Prevention: request the extension today, write down your floor and remote must-have before any call, and do not accept until the package is in writing.\n\n## 10. Round record\n\nPending. Complete after the real round.\n\n## Final review note\n\n- **Status:** Conditional. Open gates: limit (needs your input), market range (unverified), deadline firmness and extension (unknown), equity details (missing), other process timing (unknown).\n- **Most damaging plausible error:** Accepting or rejecting by Friday without the other process's result.\n- **Prevention:** Request the extension today. Set your floor and remote must-have first.\n- **Next action:** Send the extension request. Tell me your floor and whether remote is a must-have, and I'll finalize the opening and the package.\n"}]}, "haiku-skill-2": {"cost_usd": 0.01116, "seconds": 56, "output_tokens": 10402, "files": [{"name": "salary-negotiation-brief.md", "text": "# Salary Negotiation Brief: Series C Senior Data Engineer Offer\n\n**Mode:** Preparation\n**Status:** Conditional. The limit, the equity details and the remote policy are unresolved (see sections 3 and 6).\n\n## 1. Assignment and decision\n\n- **Decision needed:** Accept, negotiate or decline the offer. Offer terms: USD 175k base, 0.05% equity (4-year vest), USD 10k signing bonus.\n- **Deadline:** \"By Friday\" is most likely **Fri 2026-10-09, which is tomorrow**. Your other final round is **Wed 2026-10-14**, after the offer deadline. The deadline is the main pressure in this decision, so ask for an extension early (section 5).\n- **Authority:** Unknown. Confirm who can change base, equity and remote terms (recruiter, hiring manager, or a comp committee) before you assume a number is final.\n- **Your priorities, in order:** (1) more base, (2) one remote day per week. Equity and signing bonus are secondary levers.\n- **Diagnosis:** A single offer with several linked terms, not an auction. Your second process is real leverage only once it produces an offer. Today it has not, so do not present it as one.\n\n## 2. Facts, hypotheses and unknowns\n\n**Observed**\n- Current base: USD 160k. The offer is a 9.4% increase on base.\n- Your market range: USD 170k to 200k base for senior data engineers in your city. Source and date unverified.\n- The hiring manager called you \"the strongest candidate they've seen.\" This is praise, not a term. Use it to justify the ask, not as a promise.\n\n**Hypothesized**\n- USD 175k sits at the bottom of your market range, so there is room to move base.\n- The company has more flexibility on equity and signing bonus than on base, because base is usually banded.\n\n**Unknown (ranked by how much the answer would change the plan)**\n\n| Priority | Unknown | Question to ask | What the answer changes |\n| --- | --- | --- | --- |\n| High | Approved base band for this role | \"What is the band for this role, and where does 175k sit in it?\" | Sets the realistic target. If 175k is the band top, shift the ask to remote, signing and equity. |\n| High | Remote policy | \"Is one remote day per week possible, and can it go in the offer letter?\" | If no, it is a hard constraint and base must carry more of the package. |\n| High | Your limit (walk-away) | Not from the offer. You must set it. | Without it, you cannot decide Friday. See section 3. |\n| Medium | Equity details | \"Options or RSUs? Strike price? Latest 409A or preferred price? Fully diluted share count? Post-termination exercise window?\" | Determines whether 0.05% is worth anything (section 4). |\n| Medium | Total comp of current job | Is the USD 160k base only? Is there a bonus or benefit that the new offer lacks? | Your true baseline. |\n| Medium | Bonus target | \"Is there an annual bonus, and what is the target?\" | Changes the real comparison. |\n| Low | Signing bonus clawback | \"Is the 10k repayable if I leave within 12 months?\" | Matters if you might leave early. |\n| Medium | Market data quality | Where did the 170 to 200k range come from, and does it match \"senior\" and your stack? | If weak, use two or three sources before quoting it to the company. |\n\n## 3. Positions\n\n- **Limit:** **Unresolved. You must set this.** It is a whole-package boundary, for example \"base of at least X, or base of Y plus remote day.\" The offer's 175k is the only figure on the table. Decide your floor before the call.\n- **Target:** Base 185k to 190k, plus one remote day per week. The market range supports this, and the hiring manager's praise supports asking for the upper half.\n- **Opening:** Base 195k, one remote day per week, and the 10k signing bonus as offered. 195k is inside the market range you found, so it is defensible. It is not an outrageous number.\n- **Likely response:** A counter near 180k, or \"base is fixed, but we can add equity or signing.\"\n- **Planned response:**\n  - If base is capped near 175k to 180k: ask for a larger signing bonus (for example 20k) and the remote day in writing. Equity can be a third lever, but it is worth little to you as cash.\n  - If the company refuses remote: base must reach the target. Do not trade base for remote unless you have decided you would accept that trade.\n- **Credibility check:**\n  - Do not claim a competing offer. You have an interview, not an offer.\n  - Do not give a number you would not accept. Decide your limit first.\n  - Do not resign until you have a signed offer.\n\n## 4. Numbers\n\n**Base increase over your current 160k (before tax, excluding equity and bonus):**\n\n| Base offered | Increase | Percent |\n| --- | --- | --- |\n| 175k (current offer) | +15k | +9% |\n| 180k | +20k | +12.5% |\n| 185k | +25k | +16% |\n| 190k | +30k | +19% |\n| 195k (opening) | +35k | +22% |\n\n**Equity (illustrative only):** 0.05% of company value, before future dilution, liquidation preferences and option strike cost. These figures are not forecasts.\n\n| Company value at exit | Gross value of 0.05% |\n| --- | --- |\n| 100M | 50k |\n| 500M | 250k |\n| 2B | 1M |\n| Company fails | 0 |\n\nEquity is a lottery, not salary. Do not count it toward the decision as if it were cash. Get the real figures (section 2) before valuing it.\n\n## 5. Package and timeline\n\n**Preferred package:** Base 190k, one remote day per week, 10k signing bonus.\n- Why they might accept: base moves inside the band they probably have, and the remote day costs them little. You are asking for something they praised you for, not something new.\n\n**Fallback package:** Base 180k, one remote day per week written into the offer, signing bonus 20k.\n- Use this if the band tops out near 180k. The signing bonus is a one-time cost, so it is easier for them to approve than base.\n\n**Package to avoid:** Base at 175k with only more equity. Equity does not meet your stated top priority.\n\n**Timeline:**\n1. **Today (Thu 10-08):** Set your limit. Gather the equity details and the market sources.\n2. **Thu or Fri morning:** Call the recruiter. Ask for the band, the remote policy, and an extension to **Wed 10-14 or Thu 10-15**. Frame the extension as \"I'm in a late-stage process and want to decide well.\" Do not mention the other company's terms.\n3. **Same day:** Send the counter in writing (see script below).\n4. **If the extension is refused:** Decide by Friday on the terms you have. This is a real decision. Decide now whether you would accept the fallback package by Friday, so you are not making that choice under pressure.\n\n**Call script (short):**\n> \"Thanks, I'm excited about the role. Before I answer, can you tell me the band for this position and whether one remote day a week is possible? Based on my market research for senior data engineers here, which is 170 to 200k, and the fact that [hiring manager] called me the strongest candidate, I'm looking for base of 190k and one remote day a week. Could we also have until Wednesday the 14th? I'm finishing a late-stage process, and I want to make the right decision for both of us.\"\n\nFollow up with an email that states the package you proposed. Do not accept verbally.\n\n## 6. Review\n\n- **Situation diagnosis:** PASS. Single offer, linked terms, real deadline identified.\n- **Relationship state:** PASS. Current base and offer recorded. No prior history with this company.\n- **Critical information:** CONDITIONAL. Band, remote policy and equity details are unknown.\n- **Preparation positions:** CONDITIONAL. Limit is unresolved; the user must supply it.\n- **Credibility:** PASS. No competing-offer claim, no unsupported number, no resignation before signing.\n- **Linked exchanges:** PASS. Base, remote and signing are linked in the packages above.\n- **Uncertainty:** PASS. Equity is shown as scenarios, not as a forecast.\n- **Process:** PASS. Courteous reset available if the recruiter pushes back.\n- **Round record:** Pending until the call.\n\n**Most damaging plausible error:** Accepting 175k on Friday because of the deadline, without asking for the band or the remote policy.\n**Prevention:** Set your limit today and ask the band question before any number is discussed. Request the extension before the call ends.\n**Next action:** Decide your limit and gather equity details. Those two answers unblock the call.\n\n## 7. Round record (pending)\n\nComplete after the call: agreed terms, unilateral decisions, open commitments, and what the company revealed about the band and remote policy.\n"}]}, "haiku-skill-3": {"cost_usd": 0.01331, "seconds": 74, "output_tokens": 14453, "files": [{"name": "negotiation-brief.md", "text": "# Negotiation Brief: Senior Data Engineer Offer (Series C)\n\n**Mode:** Preparation and proposal review. **Date assumed:** Thu 8 Oct 2026.\n**Status:** Conditional. Limit and several facts are unresolved (see section 3).\n\n---\n\n## 1. Assignment and decision\n\n- **Decision needed:** Accept, counter, or decline the offer of USD 175k base, 0.05% equity (4-year vest), and USD 10k signing bonus. The unit of agreement is the whole package, not base alone.\n- **Parties and authority:** Your contact is the hiring manager. They called you \"the strongest candidate they've seen,\" which is good evidence of value but does not show they can approve more base. Ask who sets the band and who signs off on exceptions.\n- **Deadline:** The offer answer is due **Fri 9 Oct (tomorrow)**. Your other final round is **Wed 14 Oct** (assuming \"next Wednesday\" means 14 Oct). The offer deadline falls five days before your next decision point. This timing gap is the central problem.\n- **Next action:** Today, contact the hiring manager or recruiter to (a) request a call, (b) ask for a short extension, and (c) open the counter. Do not wait until Friday.\n- **Diagnosis:** Mainly a single-issue base negotiation, with linked non-price items (remote day, signing bonus, equity). It is also the start of an ongoing relationship: the terms you agree to will shape your first-year review, raises, and remote arrangement. Get non-cash items in writing.\n\n## 2. Current relationship state\n\n| Category | Evidence | Immediate effect | Later effect | Confidence |\n| --- | --- | --- | --- | --- |\n| Prior agreement | None. Nothing accepted. Current job: USD 160k base. | Current job is your fallback. | None. | Known |\n| Your unilateral decision | Second process; final round on 14 Oct. | You have no second offer. Do not imply one. | Gives you an option only if it reaches an offer. | Known |\n| Their unilateral decision | Offer at 175k with a Friday deadline. | Sets their opening position. | They may expect a counter. | Known |\n| External change | Market data: USD 170-200k base for senior data engineers in your city (your research; source unverified). | Gives a basis for asking above 175k. | Strengthens or weakens your case depending on the source. | Unverified |\n| Process | Hiring manager's praise. | Good rapport; a counter now signals interest, not hostility. | Trust at the start of the job. | Observed, one comment |\n\n## 3. Critical unknowns (ranked by effect on the decision)\n\n1. **Base band and where 175k sits in it.** Ask: \"What is the approved range for this role, and where does 175 sit?\" If 175 is near the top, base is the wrong ask. Push signing, equity, and remote instead.\n2. **Equity details.** Ask for: fully diluted share count, latest preferred or 409A price per share, option or RSU type, exercise cost, vesting cliff, refresh policy. Arithmetic only: 0.05% of a company valued at USD 100M is USD 50k on paper, before dilution. Without the valuation, the grant could be worth very little or a lot.\n3. **Remote day.** Is one day per week already company policy for this role or team? Is it in writing? If it is informal, it is easy to lose.\n4. **Annual bonus.** Is there one, and what is the target? This changes the comparison with your current USD 160k, which may include bonus.\n5. **Signing bonus terms.** Clawback period, payment timing, and whether it can move.\n6. **Your limit.** Unresolved. You need to define it (see section 4).\n7. **Second process.** Likely offer range and timing. Unknown. Do not present it as an offer.\n8. **Source of the market range.** Where it came from, sample size, and date. An unsourced figure is weak in negotiation.\n\n## 4. Limit, target, opening, and concession responses\n\n- **Limit:** **UNRESOLVED. You must set this.** It should be the lowest total package at which you would leave your current job, given the risk and the 14 Oct process. Do not use USD 160k as the floor by default; it is only your current pay.\n- **Target:** Base USD 182-188k plus a written remote day. Basis: the midpoint of your market range is 185k, and \"strongest candidate\" supports asking. Feasibility is hypothesized, because Series C bands are often rigid.\n- **Opening:** Base USD 190k; remote one day per week written into the offer; signing bonus USD 20k; and a request for equity documentation before you agree to the equity figure. 190k is near the top of your range, which makes it defensible. Opening at 200k with no source would be harder to justify.\n- **Likely response:** \"Base is capped at [X]\" or \"We can do [X] on base but not 190.\"\n- **Planned response (conditional):**\n  - **If base can go to 180k or higher:** Accept only with the remote day written, signing at 15k or more, and a six-month review with stated criteria.\n  - **If base is firm at 175k:** Ask for signing at 20k, the remote day in writing, a six-month review, and equity documentation. If the total still falls below your limit, decline.\n  - **If they grant a short extension:** Take it.\n- **Credibility check:**\n  - Do not say you have a competing offer. You have a final-round interview. Say exactly that.\n  - Do not state a deadline you will not keep.\n  - Do not present the market figure as fact unless you can cite it.\n  - Do not accept before you know the remote and equity terms.\n  - If you revise your ask later, tie the change to a new fact (for example, updated market data), not to pressure.\n\n## 5. Interests and issue map\n\n| Issue | Your interest and priority | Their likely interest (hypothesis) | Evidence | Possible difference | Feasible variable |\n| --- | --- | --- | --- | --- | --- |\n| Base | Highest (your own statement) | Budget and band discipline | Your statement; their band unknown | Series C bands may be fixed; equity and signing are often more flexible (hypothesis) | Base within band, or a six-month review |\n| Remote day | Second (your own statement) | Team and hub policy | Your statement; policy unknown | Low cost to them, high value to you (hypothesis) | One day per week, in writing |\n| Signing bonus | Useful | One-time cash, easier to approve than base (hypothesis) | Offer terms | Cheap for them to move | 10k to 15-20k |\n| Equity | Unknown; you have not said | Low cash cost to them | Offer terms only | Value depends on valuation (unknown) | Grant size, after documentation |\n| Decision timing | You want time to compare | They want to fill the role quickly | \"Answer by Friday\" | Compatible: a short extension costs them little if they hire you | Decision date, e.g. 14-16 Oct |\n\n## 6. Linked package and recommendation\n\n**Opening package:** Base 190k, remote one day per week in writing, signing 20k, equity documentation provided. **Give:** a quick signature once terms are agreed, and a flexible start date.\n\n**Fallback package (if base is capped at 175-180k):** Base at their maximum, remote in writing, signing 15-20k, six-month review with criteria. **Give:** a decision by Friday or Monday.\n\n**Dependency:** If they refuse the remote day, reopen base rather than accepting base alone at 175k.\n\n- **Why you might accept:** Base is +9% over your current pay, plus the remote day. Equity upside is real but unquantified.\n- **Why they might accept:** A base ask within their market range is reasonable if the data holds. The remote day and signing bonus are cheaper for them to give (hypothesis). They have signaled high regard for you.\n- **Feasibility:** Unknown. The hiring manager may not be able to approve exceptions alone (hypothesis).\n- **Recommendation:** Do not accept 175k as-is. Ask for an extension to 14-16 Oct. If refused, counter by Friday with the fallback package. Decline only if the final package is below the limit you set.\n\n## 7. Uncertainty and agreement architecture\n\n| Condition | Immediate result | Later consequence | Trigger or evidence | Response |\n| --- | --- | --- | --- | --- |\n| Band is tight (175k near top) | Base cannot move | You rely on remote, signing, equity | Recruiter states band | Lock the six-month review in writing |\n| Second process moves fast and makes an offer | Better option | You need to compare packages before 14 Oct | Their written offer | Ask for extension early; decide with both in hand |\n| They refuse extension and withdraw the offer | You lose this offer | Current job remains the fallback | Refusal or rescission | Decide Friday only on a package you can accept |\n| Second process delays past 14 Oct | You decide with only this offer | Limit becomes more important | Their timeline | Set the limit before the call |\n\n**Unresolved mechanics:** Whether 0.05% is fully diluted; vesting cliff and refresh; whether the remote day is written; whether there is an annual bonus; clawback on the signing bonus.\n\n## 8. Process and rehearsal\n\n- **Escalation signal:** \"That's our final number,\" an unexplained refusal to discuss terms, or a withdrawn offer after you ask questions.\n- **Reset words:** \"I'm glad this role is a good fit, and I want to make it work. What is flexible here: base, signing, or the remote arrangement?\"\n- **Agenda:**\n  1. Confirm the call. Ask about the band and equity documentation.\n  2. Present the opening, with the market basis stated plainly.\n  3. Ask what can move.\n  4. Respond with the package from section 6.\n  5. Request the extension in writing.\n- **Rehearsal:**\n  - *\"175 is in line with our band.\"* Reply: \"I understand the band. What else can move: signing, remote, equity, or a review in six months?\"\n  - *\"We can't go beyond 175.\"* Reply: \"Thanks for being clear. If base can't move, could we put the remote day in writing, raise signing to 20k, and set a review date?\"\n  - *\"Your other process: are you close to an offer?\"* Reply only with fact: \"I'm in a final round with another company next Wednesday.\"\n  - *They refuse the extension.* Decide by Friday on the fallback package. Do not invent urgency.\n- **Largest plausible mistake:** Implying a competing offer you do not have. If they test it, or the second process falls through, you lose the offer and credibility. Prevention: use exact facts only.\n- **Second mistake:** Accepting base at 175k without the remote day in writing. Prevention: get it in the offer letter before you sign.\n\n## 9. Round record\n\n**Pending.** Complete after the round.\n\n---\n\n## Review note\n\n- **Gates failed or unresolved:** Limit (not set), band (unknown), equity valuation (unknown), remote policy (unknown), bonus (unknown), market data source (unverified), second-process status (unknown).\n- **Most damaging plausible error:** Bluffing about a competing offer.\n- **Prevention:** Use the exact sentence in section 8.\n- **Next actions, today:** (1) Set your limit. (2) Email or call the hiring manager to set a call and ask for an extension. (3) Request equity documentation and the band. (4) Find and save the source of the market data.\n\n*Structure adapted from the Costa negotiation-dynamics method. Not an official template.*\n"}]}, "opus-1": {"cost_usd": 0.13612, "seconds": 51, "output_tokens": 4052, "files": [{"name": "negotiation-prep.md", "text": "# Salary Negotiation Prep \u2014 Senior Data Engineer, Series C offer\n\n*Prepared Thu 2026-10-08. Their deadline: Fri 2026-10-09 (tomorrow). Other final round: Wed 2026-10-14.*\n\n---\n\n## 1. Where you stand\n\n| | |\n|---|---|\n| **Their offer** | $175k base \u00b7 0.05% equity, 4-yr vest \u00b7 $10k signing |\n| **Your current job** | $160k, and you're happy enough to stay |\n| **Market (your research)** | $170\u2013200k base. Their $175k is in the **bottom fifth** of that range |\n| **Their view of you** | \"The strongest candidate they've seen\" |\n| **Other option** | Final round Wed 10/14, no offer yet |\n\n**Your leverage is strong.** You don't need this job (staying at $160k is a real fallback), they've said you're their top pick, and their offer is low in the market range. Having the other process helps, but you don't need it to make your case.\n\n**The weak spot is time.** A Friday deadline lands before your other final round. Sort that out first.\n\n---\n\n## 2. Your numbers (decide these before you talk to them)\n\n| | Base | Why |\n|---|---|---|\n| **Ask (anchor)** | **$195k** | Near the top of the market range. Backed by \"strongest candidate\" and a low starting offer. Not so high it looks unserious. |\n| **Target** | **$185\u2013190k** | A realistic result. Mid-to-upper market. |\n| **Walk-away floor** | **~$180k** *(adjust to your situation)* | Below this, the move gives you little over a $160k job you like, plus the risk of a new role. |\n\nRemote day: **one fixed remote day per week, written into the offer letter.** Ask for it alongside salary, not after.\n\n**Things you can trade if they can't move base:**\n1. A bigger signing bonus (e.g. $10k \u2192 $20\u201325k). This is often a separate budget from salary.\n2. More equity (e.g. 0.05% \u2192 0.07\u20130.08%).\n3. A guaranteed compensation review at 6 months, written down.\n4. Title or level, if it maps to a higher pay band.\n\nDon't trade away the remote day. You've said it matters, and it usually costs them nothing.\n\n---\n\n## 3. Do this first: get the deadline moved (today)\n\nSend this **today** so they have time to respond before Friday. Be honest: you have a final round, **not an offer**. Don't hint that you have one.\n\n> **Subject: Re: Offer \u2014 timing**\n>\n> Hi [Name],\n>\n> Thank you again for the offer. I'm genuinely excited about [company] and the team, and I want to give it a proper answer.\n>\n> To be transparent: I'm finishing up one other process, with a final conversation next Wednesday. Would it be possible to extend my decision deadline to **Friday, October 16**? I'd also like to set up a short call this week to talk through a few parts of the offer, so we can use that time well.\n>\n> Best,\n> [You]\n\n- Companies extend deadlines for their top candidate all the time. Asking is normal and shows you take the decision seriously.\n- **If they say no:** negotiate on the call anyway (section 4). Then decide by Friday using your walk-away number. Your fallback is still your current job, not the other company.\n- **Once the deadline is extended:** tell the other company you have an offer with a deadline. Ask if they can move faster.\n\n---\n\n## 4. The negotiation call\n\nDo it by phone or video if you can, then confirm by email.\n\n### Opening script\n> \"Thanks for making time. I want to say up front that I'm excited about this role and I'd like to make it work.\n>\n> I have two things I'd like to talk about. First, base salary. From my research, senior data engineers here are paid roughly $170\u2013200k. Given the scope of the role and the feedback from the process, I was expecting to be toward the top of that range. **I'd be looking for $195k base.**\n>\n> Second, I'd like one remote day per week, written into the offer.\n>\n> If we can get there on those two, I'm ready to sign.\"\n\nThen **stop talking.** Let them answer first.\n\n### Likely pushback and how to respond\n\n| They say | You say |\n|---|---|\n| \"$175k is the top of the band.\" | \"I understand. Is there flexibility in the band for a strong candidate, or a higher level this role could map to? If base really is fixed, can we talk about the signing bonus and equity to close the gap?\" |\n| \"We can do $180k.\" | Don't accept straight away. \"That's a move in the right direction, thank you. Could we meet at $188k? And can we confirm the remote day?\" |\n| \"Remote days aren't our policy.\" | \"Is there room for an exception, or a trial period, say three months then review? It's important to me, and I'd take it into account against the base number.\" |\n| \"We need an answer Friday.\" | \"I understand. I want to give you a committed yes, not a rushed one. A few more days would let me do that.\" |\n| \"Why is $160k now not enough?\" | You don't have to share your current salary. If it's already known: \"I'm pricing this role at market, not against my current job. The role and scope are different.\" |\n| \"What's the other company offering?\" | \"That process isn't finished yet, so I don't have numbers to share. This role is the one I'm most excited about, and I'd rather get to a yes with you.\" *(Only say this if it's true.)* |\n\n### Rules of thumb\n- Sound excited and on their side the whole way through. You're working out a problem *with* them.\n- Ask for everything in **one round** (base and remote day). Coming back later with new asks wears down goodwill.\n- Don't accept on the call. Say \"Thank you, let me look at the updated offer in writing,\" then reply within 24 hours.\n- Get everything in writing: base, signing bonus, equity, and the remote day.\n\n---\n\n## 5. Questions to ask about the equity\n\n0.05% means nothing until you know what it's worth. Ask:\n\n1. How many shares, and what's the **total fully diluted share count**?\n2. What was the **valuation at the last round** (preferred price per share)? What's the current **409A / strike price**?\n3. Are these options (ISO/NSO) or RSUs? Is there a **1-year cliff**?\n4. How long do you have to **exercise after leaving** (90 days vs. extended)?\n5. Is there a **refresh grant** policy?\n\nRough idea of value: at a $500M valuation, 0.05% = $250k on paper \u2248 $62k/yr. At $1.5B it's \u2248 $187k/yr. Before any dilution or exit discount, and options are worth less than this because you pay the strike price. **Value it as a lottery ticket. Your base salary is what you can count on.**\n\nAlso confirm: annual bonus target (if any), 401k match, health insurance costs, PTO, and whether you'd have to **repay the signing bonus** if you leave within 12 months.\n\n---\n\n## 6. Follow-up email after the call (fill in)\n\n> Hi [Name],\n>\n> Thanks for the conversation today. To confirm what we discussed:\n> - Base salary: $[X]\n> - Signing bonus: $[X]\n> - Equity: [X shares / X%], 4-year vest, [cliff]\n> - One remote day per week [fixed day / flexible]\n>\n> I'm looking forward to the updated offer letter, and I'll get back to you by [date].\n>\n> Best,\n> [You]\n\n---\n\n## 7. Decision checklist (before you sign)\n\n- [ ] Base at or above my walk-away number\n- [ ] Remote day in writing\n- [ ] Equity details understood (shares, valuation, strike, cliff, exercise window)\n- [ ] Signing bonus clawback terms known\n- [ ] Compared against the other company's offer (if it arrives in time)\n- [ ] Compared honestly against staying at $160k: is the gain worth the switch?\n\n---\n\n*Note: the market range comes from your research and I haven't checked it. If you can, compare it with Levels.fyi or a recruiter for your city and level before the call.*\n"}]}, "opus-2": {"cost_usd": 0.1473, "seconds": 52, "output_tokens": 4375, "files": [{"name": "negotiation-prep.md", "text": "# Salary Negotiation Prep: Senior Data Engineer, Series C Offer\n\n*Prepared Thursday 2026-10-08. The offer deadline (\"Friday\") is assumed to be **tomorrow, Oct 9**. The other company's final round is **Wed Oct 14**.*\n\n---\n\n## 1. Where you stand\n\n| | |\n|---|---|\n| Offer | $175k base \u00b7 0.05% equity (4-yr vest) \u00b7 $10k signing |\n| Current | $160k, and you're happy to stay |\n| Market (your city) | $170\u2013200k base. The offer is at the **lower-middle** of that range |\n| Their view of you | \"Strongest candidate we've seen\" |\n| Alternatives | Keep your current job, or the second process (final round next Wed) |\n\n**Your leverage is strong.** You don't need this job, they've said you're their top pick, the offer is below the market midpoint (~$185k), and you have another process running. The deadline is the main thing working against you.\n\n---\n\n## 2. Your numbers (decide these before any call)\n\n| | Base | Notes |\n|---|---|---|\n| **Ask (anchor)** | **$195k** | Near the top of the market band; defensible given \"strongest candidate\" |\n| **Target** | **$185\u2013190k** + 1 remote day/week | A realistic place to land |\n| **Walk-away** | *Fill in yourself:* ______ | Suggestion: if they won't go above ~$178k and won't give a remote day, it's only a modest gain over $160k, and staying or waiting for the other offer is a real option |\n\nThings you can trade if base gets stuck (from most to least useful to you):\n1. **Remote day.** Often free for them to give. Ask for it in writing, in the offer letter.\n2. **Bigger signing bonus** ($20\u201325k) to cover part of a base gap.\n3. **More equity** (e.g. 0.07\u20130.08%).\n4. **Guaranteed comp review at 6 months**, written into the offer.\n\n> Signing bonus is one-time money. A $10k base increase compounds through every future raise and job offer. Push for base first.\n\n---\n\n## 3. Step one: get the deadline moved (do this today or first thing tomorrow)\n\nA deadline the day after you get the offer, with your other final round on Wednesday, is worth pushing back on. Asking for a few more days is normal, and a company that thinks you're its strongest candidate will almost always agree.\n\n**Script (call or email):**\n> \"Thanks again, I'm really excited about this role and the team. I want to give you a considered answer rather than a rushed one. Could I have until **Thursday Oct 15** [or Monday Oct 19]? I also have a few questions about the compensation package that I'd like to go over before then.\"\n\n- You can mention the other process if you want: *\"I'm finishing one other process that wraps up early next week, and I'd like to close everything out together.\"* That's true, and it's the strongest reason you can give.\n- **Don't** say you have another offer. You don't yet, and getting caught bluffing would cost you their trust.\n- If they won't move at all, treat that as a warning sign about the company, and negotiate tomorrow anyway (section 4).\n\n---\n\n## 4. The negotiation conversation\n\n**Do it live (phone or video) if you can, then confirm by email.** Open with enthusiasm, then make the ask.\n\n**Main script:**\n> \"I'm really excited about this. The team, the data platform work, and [specific thing you liked] all fit what I'm looking for. I want to make this work.\n>\n> Based on my research, senior data engineers in [city] are paid roughly $170\u2013200k base. Given the scope of the role and the feedback from the process, I was expecting the upper part of that range. **If you can get the base to $195k and include one remote day a week, I'm ready to move forward.**\"\n\nThen **stop talking.** Let them answer first.\n\n**Rules:**\n- Make one specific number your first ask. Don't give a range, or they'll take the bottom of it.\n- Bring up the remote day **together with** the base ask, not after you've settled on salary. A second request after agreement feels like re-opening the deal.\n- Don't justify your ask with personal needs (rent, etc.). Stick to market rate, the role, and your fit.\n- Don't accept on the call. Say: *\"Thank you, that's really helpful. Can you send the updated offer in writing? I'll get back to you by [date].\"*\n\n---\n\n## 5. Likely pushback and how to answer\n\n| They say | You say |\n|---|---|\n| \"$175k is the top of the band.\" | \"I understand. Is there flexibility in the level or the band, given the feedback from the interviews? If base really can't move, could we close the gap with signing bonus and equity?\" |\n| \"What are you making now?\" | \"I'd rather focus on what this role is worth. Based on market data, that's $185\u2013200k.\" *(Your $160k weakens your position, so don't volunteer it. If they press: \"I'm well paid where I am and not looking to leave just for a lateral move.\")* |\n| \"We can do $180k.\" | \"That's a step in the right direction, thank you. Can we meet at $190k with the remote day, and I'll sign this week?\" |\n| \"Remote days are against team policy.\" | \"Is it something we could formalize after 90 days, or put in writing as a trial?\" |\n| \"We need an answer now.\" | \"I'm very positive, and I want to say yes on terms I'll be happy with long term. I just need [X days].\" |\n| \"Do you have other offers?\" | \"I'm in final rounds with one other company. You're my top choice if we can get the package right.\" *(Only say this if it's true.)* |\n\n---\n\n## 6. Questions to ask about the equity\n\n\"0.05%\" means little without context. Ask for these before you decide:\n\n1. **Number of shares**, and the **fully diluted share count** they used to calculate 0.05%\n2. **Strike price** (409A value) vs. **latest preferred price per share**\n3. **Latest valuation** (the Series C post-money)\n4. Options or RSUs? ISO or NSO?\n5. **Vesting:** 1-year cliff? Monthly after that?\n6. **Exercise window after leaving** (90 days vs. a longer window like 5\u201310 years)\n7. **Liquidation preferences:** total preference stack, and whether any of it is participating\n8. **Refresh grant** policy\n\n**Rough paper value:** 0.05% \u00d7 valuation \u00f7 4 years. For example, at a $500M valuation that's $250k total, or about $62k/yr on paper. Before going public, assume it's worth **far less** (discount it by at least 50\u201375%), and future rounds will dilute it. Don't trade away guaranteed base for equity unless the numbers above look good.\n\n---\n\n## 7. Using the second process\n\n- **Before Wednesday:** if you can, let the second company know you have an offer with a deadline. They may be able to speed things up.\n- **If you get a second offer:** you're in a much stronger position. You can share the competing number (only if it's real) and ask the first company to match or beat it.\n- **If the Series C company won't extend past Friday:** you'll have to choose between taking the best terms you can get tomorrow and giving up the offer while you wait. Decide **now** which you'd pick, so you aren't deciding under pressure.\n\n---\n\n## 8. Checklist\n\n- [ ] **Today/tomorrow AM:** ask for the deadline to be extended (section 3)\n- [ ] Write down your walk-away number (section 2)\n- [ ] Send the equity questions (section 6)\n- [ ] Negotiation call: ask for $195k + 1 remote day (section 4)\n- [ ] Get everything in writing, **including the remote day**\n- [ ] Tell the second company about your timeline\n- [ ] Before signing: compare total package vs. $160k + staying put vs. the other offer\n\n---\n\n## Email template (after the call)\n\n> Subject: Re: Offer \u2013 [Your Name]\n>\n> Hi [Name],\n>\n> Thanks for taking the time to talk today. I'm excited about the role and joining [Company].\n>\n> As discussed, I'd be ready to accept with a base salary of $195,000 and one remote day per week, with the equity and signing bonus as offered. I'd also appreciate the equity details (share count, fully diluted shares, strike price, and latest preferred price) so I can fully evaluate the package.\n>\n> Would it be possible to confirm by [date]?\n>\n> Best,\n> [Your Name]\n"}]}, "opus-3": {"cost_usd": 0.12983, "seconds": 44, "output_tokens": 3791, "files": [{"name": "negotiation-prep.md", "text": "# Salary Negotiation Prep: Senior Data Engineer, Series C Offer\n\n_Prepared 2026-10-08 (Thursday). Offer deadline: **Friday 2026-10-09**. Other process: final round **Wednesday 2026-10-14**._\n\n---\n\n## 1. Where you stand\n\n| Factor | Effect on leverage |\n|---|---|\n| You're employed at $160k and not desperate | **Strong.** Walking away is a real option, and they can tell. |\n| \"Strongest candidate they've seen\" | **Strong.** Replacing you costs them weeks plus a weaker hire. |\n| Second process, final round next Wed | **Moderate.** It's real, but it isn't an offer yet. Describe it accurately. |\n| Market $170\u2013200k; offer is $175k | Offer sits in the **bottom third** of the band. Your profile supports the top half. |\n| Friday deadline | **This is their pressure tactic.** Deadlines like this usually have some give. |\n\n**Bottom line:** You have more leverage than they do. The main risk is accepting too fast, not pushing too hard.\n\n---\n\n## 2. Your numbers (decide these before the call)\n\n| | Base | Notes |\n|---|---|---|\n| **Ask (anchor)** | **$195k** | Top of the band but defensible. Leaves room to settle around $185\u2013190k. |\n| **Target** | **$188\u2013190k** | A realistic good outcome. |\n| **Walk-away / floor** | **$180k** _(adjust to your gut)_ | Below this, a +$15\u201320k raise for an early-stage risk plus switching costs isn't clearly worth it. |\n| Remote | 1 day/week, **in writing** | Ask for it explicitly in the offer letter. |\n\n> Fill in your real floor here: **$______**. Don't change it during the call.\n\n### Total first-year comparison\n- Current: $160k\n- Offer as-is: $175k + $10k signing = **$185k year 1**, then $175k\n- At target: $190k + $10k = **$200k year 1**\n\nThe base matters more than the signing bonus because it compounds into every future raise and every future offer.\n\n---\n\n## 3. Equity: get facts before you assign it value\n\n0.05% means nothing without context. **Ask for these in writing:**\n1. Number of options/RSUs (actual share count)\n2. Total fully diluted shares outstanding\n3. Latest preferred share price (Series C price) and current 409A / strike price\n4. Options or RSUs? Cliff (usually 1 yr)? Post-termination exercise window (90 days vs. 5\u201310 yrs is a big difference)?\n5. Liquidation preference stack: roughly how much was raised in total, and are any preferences above 1x?\n\n**Quick math example:** if the Series C post-money valuation was $500M, 0.05% = $250k on paper, about **$62.5k/yr** over 4 years, *before* dilution and the strike price, and assuming a good exit. For planning, treat it as a lottery ticket worth $0\u2013X. **Don't trade guaranteed base for equity** unless the numbers clearly justify it.\n\nEquity is also an extra lever. If base stalls, ask for 0.07\u20130.1%.\n\n---\n\n## 4. The deadline: ask for an extension first\n\nYou want to hear back from the other company before deciding. Ask for an extension **before** you negotiate, or in the same message.\n\n**Ask for:** decision by **Thursday 2026-10-15** (the day after your final round).\n\n> \"I'm really excited about this role and want to give it a proper decision. I have one other process wrapping up, with a final round next Wednesday, and I'd like to be able to commit fully once that's done. Could we move the deadline to Thursday the 15th?\"\n\n- Most companies agree, especially for a candidate they call the strongest they've seen.\n- If they refuse outright, that tells you something about the culture. You can still negotiate by Friday (see \u00a77).\n- Also email the other company today: *\"I have an offer with a deadline. Is there any way to speed up the timeline?\"* That's standard and often works.\n\n---\n\n## 5. Script: the negotiation call\n\n**Do it on the phone or video if you can.** Follow up in writing.\n\n**Open warm:**\n> \"Thanks again for the offer. I've really enjoyed meeting the team, and I'm excited about [specific thing: the data platform work / the problem space].\"\n\n**State the ask with reasons:**\n> \"There are two things I'd want to sort out before I can say yes. First, base. Based on market data for senior data engineers here, which runs about $170\u2013200k, and the scope of this role, I was expecting something closer to **$195k**. Second, I'd like one remote day a week written into the offer.\"\n\n**Then stop talking.** Let the silence sit. Don't fill it with justifications or a lower number.\n\n**If they ask \"what would it take for you to sign?\"**\n> \"If you can get base to $195k and include the remote day, I'm ready to sign.\" _(Only say this if it's true.)_\n\n**If they ask about other offers:**\n> \"I'm in the final stages with another company, final round next week. You're my first choice right now, and I'd like to make this work.\"\n> _(Only say \"first choice\" if it's true. Don't claim you have a competing offer.)_\n\n**If they ask your current salary:** deflect.\n> \"I'd rather focus on what this role is worth. Market for this level is $170\u2013200k, and that's where I'm anchoring.\"\n\n---\n\n## 6. Handling their responses\n\n| They say | You say / do |\n|---|---|\n| \"$175k is the top of the band.\" | \"I understand. Is there any flexibility on level, or could we close the gap with a higher signing bonus or more equity?\" Then ask for the remote day either way. |\n| \"We can do $182k.\" | \"I appreciate that. Could we meet at $190k? That would make this an easy yes for me.\" |\n| \"We can do $185\u2013190k.\" | You're at target. Close out the remote day and the extension, then accept or say you'll confirm by the agreed date. |\n| \"We can't do remote.\" | \"Could we do one day a week on a trial basis, reviewed after 3 months?\" or \"Could that be an option after onboarding?\" |\n| \"We need an answer Friday, no extension.\" | Negotiate anyway. Decide using your floor. Remember: you have a job you're fine in. |\n| Base is fixed but other things move | Trade for: signing bonus to $20\u201325k, equity to 0.07\u20130.1%, a guaranteed comp review at 6 months, extra PTO, a learning budget. |\n\n**Priority order of levers** (your stated preferences): base > remote day > equity > signing bonus > other.\n\n---\n\n## 7. Decision tree\n\n```\nAsk for extension to Thu 10/15\n\u251c\u2500\u2500 Granted\n\u2502   \u2514\u2500\u2500 Negotiate now \u2192 get final numbers in writing\n\u2502       \u2192 finish other final round Wed \u2192 compare \u2192 decide Thu\n\u2514\u2500\u2500 Refused\n    \u2514\u2500\u2500 Negotiate now (Thu/Fri)\n        \u251c\u2500\u2500 Final \u2265 floor + remote OK \u2192 accept, or decline the other process\n        \u251c\u2500\u2500 Final \u2265 floor, no remote \u2192 your call: how much is 1 day worth to you?\n        \u2514\u2500\u2500 Final < floor \u2192 decline politely, stay at current job / continue other process\n```\n\nIf the other company makes an offer, come back to this company before deciding:\n> \"I've received another offer at $X. I'd still prefer to join you. Can you match it?\"\n\n---\n\n## 8. Do / Don't\n\n**Do**\n- Be warm and enthusiastic. You're negotiating *with* your future manager.\n- Get everything (base, remote, equity details, deadline) **in writing** before signing.\n- Use specific numbers, not ranges. A range gives them the bottom of it.\n- Thank them whatever they say.\n\n**Don't**\n- Accept on the call. Say: \"That sounds great. Can you send the updated offer so I can review it?\"\n- Invent or inflate the other process.\n- Apologize for asking or over-explain.\n- Bring up personal finances as a reason. Stick to market and value.\n\n---\n\n## 9. Checklist before the call\n\n- [ ] Write down your real floor: $______\n- [ ] Decide: am I really willing to walk at the floor?\n- [ ] Email the other company asking to speed things up\n- [ ] Prepare 2\u20133 concrete examples of your impact (pipelines built, cost/time saved, scale)\n- [ ] List the equity questions (\u00a73)\n- [ ] Have this doc open during the call\n- [ ] After the call: send a written recap email of what was agreed\n"}]}, "haiku-1": {"cost_usd": 0.00778, "seconds": 51, "output_tokens": 9447, "files": [{"name": "negotiation-prep.md", "text": "# Salary Negotiation Prep: Series C Senior Data Engineer Offer\n\n## 1. Snapshot\n\n| Item | Offer | Current job | Difference |\n|---|---|---|---|\n| Base | $175k | $160k | +$15k (+9.4%) |\n| Equity | 0.05%, 4-year vest | unknown | cannot compare yet |\n| Signing bonus | $10k (one-time) | unknown | |\n| Annual bonus | unknown | unknown | ask both sides |\n| Remote | not offered | not stated | target: 1 day/week |\n\nMarket range from your research: $170k to $200k base, midpoint about $185k. The $175k offer sits in the bottom sixth of that range.\n\n## 2. Timing problem (act first)\n\n- Today is Thursday, 2026-10-08. The offer deadline is Friday, 2026-10-09, which is tomorrow.\n- Your other final round is Wednesday, 2026-10-14. That is after the deadline.\n- You do not have an offer from the other company. Do not imply that you do. You can truthfully say you are in late-stage interviews elsewhere.\n\nAsk for an extension today. A reasonable request is a decision by Monday, 2026-10-19, with Friday 2026-10-16 as the fallback. Asking for a few extra days is routine for senior roles. If the company refuses to move a one-day deadline on a senior hire, treat that as information about how they will handle pressure later.\n\n## 3. Your numbers\n\n| | Base | Notes |\n|---|---|---|\n| Opening ask | $195k | Top of your market range, justified by scope and their praise |\n| Target | $190k | Upper-middle of the range, about +19% over current |\n| Walk-away on base | $180k | Below this, you need strong offsets elsewhere |\n\nDo not go below $175k (their current number). Negotiating below the offer invites a reset.\n\n## 4. Priorities and what to ask for\n\nYour stated order is base first, remote second, equity and signing bonus as secondary levers. Ask in that order.\n\n1. **Base.** Hardest to get, most valuable (recurring, compounds, and drives future bonus and equity percentages). Ask once, clearly, then stop talking.\n2. **Remote day, in writing.** Often cheaper for the company to grant than more base, and it is a policy question as much as a money question. Ask whether the team has a hybrid policy and whether the day can be confirmed in the offer letter.\n3. **Signing bonus.** One-time and cheap for them to give. Realistic ask: $20k to $25k. Check the clawback period (commonly 12 months, often prorated) before you accept.\n4. **Equity.** Ask for a higher grant (for example 0.08% to 0.10%) only after base is settled. See section 5.\n5. **Level and review timing.** Ask what level the offer is at and the band for that level. Moving from senior to staff is worth more than most negotiating. Also ask for a comp review at 6 months rather than 12.\n\nIf you have a current annual bonus, ask for the target percentage on the new role. Compare total cash (base plus expected bonus) against your current package, not base alone.\n\n## 5. Equity: what to find out before valuing it\n\n0.05% is a paper number. Before you weigh it, get these facts in writing:\n\n- Post-money valuation and date of the Series C\n- Total fully diluted shares outstanding (so you can convert 0.05% to a share count)\n- Option strike price (current 409A value) and whether the grant is ISOs or NSOs\n- Vesting: is there a 1-year cliff? Monthly after that?\n- Exercise window after you leave (the default is often 90 days; ask for longer)\n- Whether there is an acceleration clause\n- Refresh grant policy\n- Preference stack (liquidation preferences) and expected future dilution\n- Runway and last-round date\n\nIllustrative value only, before any dilution or preference adjustments. The actual valuation is unknown:\n\n| Post-money valuation | 0.05% paper value | Per year over 4 years |\n|---|---|---|\n| $300M | $150k | $37.5k |\n| $1B | $500k | $125k |\n| $3B | $1.5M | $375k |\n\nHeuristics:\n- Treat equity as a possible bonus, not salary. Base is guaranteed.\n- Future rounds will dilute the grant. Discount heavily for dilution, preference stacks, and years until liquidity.\n- Do not let the equity number sway the base negotiation. A higher base is worth more to you than a larger illiquid grant.\n\n## 6. Scripts\n\n### Email today (timing and document request)\n\n> Subject: Re: [Role] offer, timing\n>\n> Hi [Name],\n>\n> Thank you again. I'm excited about the role and the team, and I want to make a decision I'm confident in. I'm in late-stage interviews with another company, with a final round on Wednesday, October 14. Would it be possible to extend the decision deadline to Monday, October 19?\n>\n> Could you also send the full offer in writing, including the equity details (number of shares, total fully diluted shares, strike price, vesting schedule, and cliff)?\n>\n> I'm happy to talk by phone tomorrow about compensation. Thank you.\n>\n> [Name]\n\n### Call script: base\n\n> \"I'm really excited about this role and I want to make it work. Based on my research for senior data engineers in [city] and the scope of the work, I was hoping for a base of $190,000. Is there room to get closer to that?\"\n\nThen stop talking. Let them respond.\n\n### If they say base is fixed\n\n> \"I understand base is constrained. Could you help close the gap another way? A signing bonus of $25,000 would help a lot. And I'd like one remote day a week confirmed in the offer letter.\"\n\n### Remote day\n\n> \"One more item: I'd like one remote day per week written into the offer. Is that something you can confirm?\"\n\n### Email after a verbal agreement\n\n> Hi [Name], thanks for the call. To confirm what we discussed: base of $[X], signing bonus of $[Y], equity of [Z] shares with [vesting terms], and one remote day per week. Could you send the revised offer letter? Thank you.\n\nAlways get the revised offer in writing before you resign from your current job.\n\n## 7. Likely pushbacks\n\n| They say | You say |\n|---|---|\n| \"$175k is the top of our band for this level.\" | \"Which level is this, and what is the band for it? If the scope fits the next level, can we discuss that?\" |\n| \"Our other candidates accepted at this number.\" | \"I understand. I'd still like to close the gap, and I'm open to other parts of the package.\" |\n| \"The decision has to be by Friday.\" | \"I want to make a sound decision. Can we do Monday?\" |\n| \"What is your current salary?\" | \"I'd prefer to base this on the market value of the role.\" (Some jurisdictions prohibit asking this at all.) |\n| \"We can only move on signing bonus / equity.\" | Take the offsets only if the total package is worth it. Base is still the priority to re-raise later. |\n\n## 8. Decision rules\n\n- Base $185k or more, remote day confirmed in writing: strong yes, subject to the equity and runway checks.\n- Base $180k to $185k: good if signing bonus is at least $20k and remote is confirmed. Compare against the other process.\n- Base $175k to $180k: needs real offsets (bonus, equity, or level change) to justify the risk of a Series C.\n- Base below $175k: do not accept.\n\nYou are not desperate to leave, which is real leverage. Use it by being willing to stay put, not by bluffing.\n\n## 9. Things to avoid\n\n- Do not claim you have another offer. You don't yet.\n- Do not cite the hiring manager's \"strongest candidate\" comment as a reason for more money. Use it as evidence that you are the level they want, e.g., \"Given what you said about my work, I'd like the comp to reflect the level I'm operating at.\"\n- Do not accept verbally before you have the written offer with equity details.\n- Do not negotiate in email alone for base. Call, then confirm by email.\n- Do not give your current salary number if you prefer not to. Redirect to market value.\n\n## 10. Timeline\n\n- **Thu 2026-10-08 (today):** Send the extension and document-request email. Confirm your market data source and the pay range for this level.\n- **Fri 2026-10-09:** Call the recruiter or hiring manager with the base ask (sections 6 and 7). If the extension is granted, use the time to get equity details and ask about level.\n- **Week of 2026-10-12:** Get the revised offer in writing. Complete the other final round on Wed 2026-10-14.\n- **By Mon 2026-10-19 (or Fri 2026-10-16 if the extension is shorter):** Make the decision with both processes in hand.\n\n## 11. Open questions for you\n\n- Does your current job have an annual bonus? What is the target percentage?\n- What is your city's market data source, and is it for the same level (senior vs staff)?\n- Is the remote day a must-have or a strong preference? How much base would you trade for it?\n- Would you accept lower base in exchange for equity, or the reverse? Be specific before the call.\n- What would make you walk away from this offer entirely?\n"}]}, "haiku-2": {"cost_usd": 0.00856, "seconds": 53, "output_tokens": 10140, "files": [{"name": "negotiation-prep.md", "text": "# Salary negotiation prep: Series C senior data engineer offer\n\n## 1. Timing: act today\n\n- Today is **Thursday 8 Oct**. \"By Friday\" means **tomorrow, 9 Oct**.\n- Your other final round is **next Wednesday, 14 Oct**, so you'd have to decide before you know the outcome there.\n- **Today:** send the extension email (section 5) and ask for a call. Don't wait until Friday to raise the timing.\n\n## 2. Your position\n\n**Leverage**\n- You're not desperate to leave. Your current job at $160k is a real fallback, so you can walk away. This is your strongest card.\n- The offer is at the bottom third of the market range you found ($170\u2013200k).\n- The hiring manager called you the strongest candidate they've seen. Mention it lightly, as context for why you're asking for the upper part of the range. It isn't a commitment.\n- The other process is a real alternative, but you don't have an offer from it yet. Say \"late-stage with another company.\" Don't claim an offer you don't have.\n\n**Weaknesses**\n- Only one offer in hand.\n- The raise is modest: $175k is +$15k (+9.4%) over your current pay. Moving to a Series C carries startup risk, so the package needs to justify that risk.\n\n**Check the market data.** Confirm whether your $170\u2013200k figure is base or total comp, and that it's for your city and years of experience. Good sources: levels.fyi, Pave, Glassdoor, and recruiters in your market.\n\n## 3. Numbers\n\n**Base salary (your priority)**\n\n| | Amount |\n|---|---|\n| Current | $160k |\n| Offer | $175k (+9.4%) |\n| Market midpoint | ~$185k |\n| **Opening ask** | **$195k** |\n| **Target** | **$185\u2013190k** |\n| Suggested floor | ~$180k (decide your own before the call) |\n\n**Signing bonus ($10k):** Ask for $20\u201325k. This is usually the easiest item to move if base is capped. Ask whether it's paid with the first paycheck and what the clawback period is.\n\n**Equity (0.05%, 4-year vest):** Treat this as a lottery ticket, not a substitute for base. Before you put any value on it, get:\n- Post-money valuation and fully diluted share count\n- Options or RSUs, and if options, the strike price and latest 409A valuation\n- Vesting cliff and refresh policy\n- Post-termination exercise window (often 90 days, which can cost a lot if you leave)\n- Liquidation preference stack\n\nRough paper value is 0.05% \u00d7 post-money valuation. For example, at a $500M post-money, that's $250k, or about $62.5k per year over four years. At $1B, it's about $125k per year. Discount heavily for dilution, illiquidity, preferences, and strike price. Ask for a modest grant increase, but expect limited flexibility.\n\n**Remote day:** Ask for one remote day per week in writing (offer letter or a manager email), and confirm it's standard policy, not \"for now.\" If they refuse, ask for a six-month review of the arrangement. Don't trade base for remote unless you value the remote day more than the difference. Estimate your savings: commute time and cost \u00d7 ~46 weeks.\n\n**Total comp:** Compare the full package, not just base. Get your current bonus target, 401k match, and benefits so the comparison is like-for-like.\n\n## 4. Questions to ask before countering\n\n- What's the level and salary band for this role (senior vs. staff)?\n- What are the bonus target, 401k match, and benefits?\n- Equity: post-money valuation, share count, options vs. RSUs, strike price, cliff, exercise window.\n- Is the remote day standard for the team, and can it go in writing?\n- When is the first compensation review?\n\n## 5. Scripts\n\n**Extension email (send today):**\n\n> Subject: Re: [Role] offer\n>\n> Hi [Name],\n>\n> Thank you for the offer. I'm excited about the team and the work. Before I give a final answer, I'd like to discuss base salary, the signing bonus, and a remote day per week. Could we talk [today / tomorrow morning] at [time]?\n>\n> I'm also in a late stage with another company, with a decision expected next week. Could we extend the decision date to [Friday, 16 Oct]? I want to make a decision I'm confident in.\n>\n> Thanks,\n> [Name]\n\n**Call opener (after thanks and enthusiasm):**\n\n> \"Based on the market for senior data engineers here, and the scope of this role, I was hoping for a base of $190,000. Is there room to get there?\"\n\nThen stop talking and let them respond.\n\n**If they push back on base:**\n- \"Is that a hard band, or is there flexibility for this level?\"\n- \"If base is at its limit, could we do the signing bonus at $25k, a written remote day, and a six-month base review?\"\n\n**If they say \"this is our final offer\":**\n- \"I understand. I'm excited about the role. Is there any part of the package you can move?\"\n- If the answer is no, don't bluff. Decide whether to accept, or ask for the extension.\n\n**If they won't extend the deadline:** Ask whether you can decide after your other final round, or whether they'd accept a decision by Monday. If neither works, decide based on the floor you set below.\n\n**After the call:** Confirm every number in writing. Don't accept verbally. Take 24\u201348 hours to review the revised offer.\n\n## 6. Ground rules\n\n- Don't claim a competing offer you don't have. \"Late-stage with another company\" is accurate and enough.\n- Don't disclose your current salary. If asked: \"I'd rather anchor on what the market pays for this role.\" In some places, employers aren't allowed to ask salary history, so check your local rules.\n- Don't quote the other company's numbers.\n- Don't ask your current employer for a counter unless you'd really stay. That changes your situation and can't be undone.\n\n## 7. Decide your rules before the call\n\n- **Accept if:** base \u2265 your floor AND the remote day is in writing.\n- **Hold for the other result if:** base is below your floor but the company is still engaged and will extend.\n- **Decline if:** base is below your floor, no remote day, and no extension.\n\nFill in your floor and your walk-away number before the call. The calls are short and it's easy to drift.\n\n## 8. Open items I couldn't determine from offer.md\n\n- Your city, years of experience, and whether the market data is base or total comp.\n- Your current bonus, 401k match, and benefits.\n- Whether the other company's compensation is comparable.\n"}]}, "haiku-3": {"cost_usd": 0.00691, "seconds": 43, "output_tokens": 8076, "files": [{"name": "negotiation-prep.md", "text": "# Salary negotiation prep\n\n## Timeline (assumes today is Thu 2026-10-08)\n\n- **Thu 10/8 (today):** Ask for an extension on the offer deadline. Send the counter in the same message or on the call.\n- **Fri 10/9:** Offer deadline as stated. Treat this as a soft date. Companies often extend when asked.\n- **Wed 10/14:** Your other final round. Ideally you decide after this, so you have both offers in hand.\n\nThe offer deadline is tomorrow, so the extension request needs to go out today.\n\n## The numbers\n\n| Item | Figure | Notes |\n|---|---|---|\n| Current base | $160k | Don't disclose this. Anchor on market value instead. |\n| Offer base | $175k | +$15k (+9.4%) over current |\n| Market range (you found) | $170k\u2013$200k | Midpoint $185k. Verify with a second source. |\n| Signing bonus | $10k | One-time. Check clawback terms. |\n| Equity | 0.05%, 4-year vest | See the equity section below |\n\n**Counter targets:**\n- **Open:** $195k (top of range; room to move)\n- **Target:** $185k\u2013$190k (market midpoint, supported by your market data)\n- **Floor:** about $180k base. Below this you're under market midpoint for a role where you've been told you're the strongest candidate. Adjust to your own judgment about what you'd accept.\n\n**Why base matters most:** Base compounds. A $15k difference is $15k every year, plus any raises calculated off it, plus any bonus percentage applied to it. Signing bonus and equity are one-time or uncertain.\n\n## Equity reality check\n\n0.05% is a lottery ticket at Series C. Don't count it toward the base comparison. Illustrative only, using hypothetical exit values (not estimates for this company):\n\n| Exit value | 0.05% gross | After ~40% future dilution (assumed) |\n|---|---|---|\n| $200M | $100k | ~$60k |\n| $500M | $250k | ~$150k |\n| $1B | $500k | ~$300k |\n\nAlso: options are worth something only above the strike price, and liquidation preferences can leave common holders with much less at moderate exits. Many outcomes are worth zero.\n\n**Questions to ask before deciding on equity:**\n- Fully diluted share count, and what 0.05% is measured against\n- Current preferred price per share and the common/409A strike price\n- ISO or NSO, and how much of the grant is ISO\n- Liquidation preference stack\n- Vesting cliff and schedule\n- Post-termination exercise window (90 days is standard; ask for longer)\n- Acceleration on change of control\n- Refresh grant policy\n\n## Asks, in priority order\n\n1. **Base:** $190k (open at $195k).\n2. **Remote day:** one day per week, in writing. Check whether the company's remote policy allows it for this role. Estimate the value to you: weekly commute time and cost \u00d7 ~46 weeks/year. Non-monetary asks are often easier for them to grant.\n3. **Signing bonus (fallback if base won't move):** raise from $10k to $20k\u2013$25k. One-time money is often easier to approve. Ask for a clawback period of 12 months or less, and payment in the first paycheck if possible.\n4. **Other things to ask about:**\n   - Annual bonus target percentage (not mentioned in the offer; if your current job has one, compare total comp)\n   - Title and level, confirmed as senior\n   - 401(k) match, health premium share, PTO\n   - Early review (e.g., 6 months) with a base adjustment\n   - Learning/conference budget, equipment\n5. **Equity:** ask for more only if base and bonus are settled. Less likely to move.\n\n## Scripts\n\n**Extension request (today, by email or call):**\n> Thank you, I'm excited about the role and the team. I want to make a decision I'm confident in. I'm in late stages with another company, with a final round on Wednesday, October 14. Could we extend the decision deadline to [Tuesday, October 20]? If that's not possible, please let me know what flexibility there is so I can plan.\n\nDon't claim another offer you don't have. Describing a process in late stages is accurate and enough.\n\n**Counter (on a call, then confirm in writing):**\n> Thanks for the offer. I'm excited to join. Based on the market for senior data engineers here, which I've seen at roughly $170k to $200k, and the scope of this role, I'd like to get base to $190,000. I'd also like one remote day per week in writing. Can we talk about what's possible?\n\nThen stop talking. Let them respond.\n\n**If base won't move:**\n> I understand base is constrained. Could we move the signing bonus to $20,000, put the remote day in writing, and revisit base at the six-month review?\n\n**On the \"strongest candidate\" comment:** Use it as context, not as your main argument. For example: \"I'm glad the team sees it that way, and it's why I'd like the offer to reflect the market.\" Market data and the scope of the role carry more weight than praise.\n\n## Things to avoid\n\n- Accepting or declining on the call. Thank them, then confirm in writing.\n- Revealing your current salary. Some places restrict salary-history questions. If asked, redirect to market value for the role.\n- Bluffing about other offers.\n- Making the counter with a long justification. One or two sentences of reasoning, then the ask.\n- Accepting the first revised number without asking about the other items (signing bonus, remote, review timing).\n\n## Decision rule\n\n- If base reaches $180k+ and the remote day is in writing, you're in a strong position to accept, subject to the other company's result on Wed.\n- If base stays at $175k, the signing bonus and remote day are worth pushing hard on. Compare with the other company once you have its offer.\n- If the company refuses to extend and you haven't heard from the other company, weigh the risk of losing this offer against the upside of waiting. You can ask for a limited extension or accept with a conditional clause. Decide this consciously, not by default.\n\n## Information I still need\n\n- Your current total comp (bonus %, benefits, equity, if any)\n- Whether the offer includes an annual bonus target\n- Whether the remote day is permitted under the company's policy\n- Status and expected timing of the other process\n- Your own floor (lowest package you'd accept)\n"}]}}, "verdicts": [{"vs": "opus", "pair": "1", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "opus", "why": "B provides a specific extension date and a workable refusal fallback. A leaves dates unresolved and suggests a conditional yes. Neither states the exact 9.375% increase; B\u2019s ready-to-sign commitment is unsupported by the user\u2019s stated intentions."}, {"vs": "opus", "pair": "1", "judge": "gpt", "order": "opus shown first", "winner": "opus", "why": "A gives a specific extension date and workable refusal fallback. B leaves extension dates blank and overstates the need to await the second process. A omits the base-increase calculation and scripts an unsupported readiness to sign."}, {"vs": "opus", "pair": "1", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "tie", "why": "Both accurately use all inputs, propose concrete counteroffers, handle leverage and timing credibly, provide direct language, and advise on concessions."}, {"vs": "opus", "pair": "1", "judge": "deepseek", "order": "opus shown first", "winner": "opus", "why": "A provides direct, usable language for the negotiation and extension request, while B's scripts are less complete and integrated."}, {"vs": "haiku", "pair": "1", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "haiku+skill", "why": "A better preserves a personally chosen acceptance threshold; B prescribes inconsistent thresholds. Both round the 9.375% increase and lack a clear fallback for a refused extension. B provides specific extension dates but implies both processes will resolve by then."}, {"vs": "haiku", "pair": "1", "judge": "gpt", "order": "haiku shown first", "winner": "haiku", "why": "A provides concrete extension dates and more usable scripts. B leaves extension dates blank and delays the counteroffer while awaiting an uncertain outcome. Neither gives the exact 9.375% increase or fully handles extension refusal and interview uncertainty."}, {"vs": "haiku", "pair": "1", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "tie", "why": "Both accurately use all inputs, propose concrete asks, handle leverage and timing credibly, provide usable scripts, and guide evaluation."}, {"vs": "haiku", "pair": "1", "judge": "deepseek", "order": "haiku shown first", "winner": "haiku", "why": "A provides direct, usable scripts and a clearer action sequence. B's scripts are less complete and actionable."}, {"vs": "opus", "pair": "2", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "haiku+skill", "why": "A handles leverage and concessions more credibly. Both miss the exact percentage and interview uncertainty. B assumes process completion, overpromises acceptance, and supplies an unsupported equity discount and annualized valuation."}, {"vs": "opus", "pair": "2", "judge": "gpt", "order": "opus shown first", "winner": "haiku+skill", "why": "B provides safer negotiation language and treats equity more cautiously. A overstates extension prospects, implies unsupported process timing, and scripts premature acceptance commitments. Both omit the exact 9.375% increase and inadequately address time needed after Wednesday\u2019s final interview."}, {"vs": "opus", "pair": "2", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "tie", "why": "Both accurately use all inputs, propose a concrete counteroffer, handle leverage and timing credibly, provide usable language, and advise on concessions."}, {"vs": "opus", "pair": "2", "judge": "deepseek", "order": "opus shown first", "winner": "tie", "why": "Both accurately use inputs, prioritize base and remote day, leverage position credibly, address timing, provide direct language, and guide concessions."}, {"vs": "haiku", "pair": "2", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "haiku+skill", "why": "A provides a stronger usable counteroffer and clearer concession guidance. Both omit the exact 9.375% increase and inadequately address interview uncertainty. B invents an expected decision next week and annualizes speculative equity value."}, {"vs": "haiku", "pair": "2", "judge": "gpt", "order": "haiku shown first", "winner": "haiku+skill", "why": "B better prioritizes base and remote work and explicitly requests both in its script. A dilutes priorities and invents an expected second-process decision. Both omit the exact 9.375% increase and inadequately address uncertainty after Wednesday\u2019s interview."}, {"vs": "haiku", "pair": "2", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "tie", "why": "Both accurately use inputs, prioritize base and remote day, handle leverage and timing well, and provide actionable scripts."}, {"vs": "haiku", "pair": "2", "judge": "deepseek", "order": "haiku shown first", "winner": "tie", "why": "Both accurately use all inputs, prioritize base and remote day, leverage position credibly, address timing, provide usable scripts, and advise on concessions."}, {"vs": "opus", "pair": "3", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "tie", "why": "A handles leverage more credibly; B provides a usable counteroffer script. Both omit the exact $15k/9.375% base increase. A's extension is imprecise; B's next-day comparison plan inadequately addresses uncertainty about receiving another offer."}, {"vs": "opus", "pair": "3", "judge": "gpt", "order": "opus shown first", "winner": "opus", "why": "A provides stronger usable scripts and concession guidance. Both omit the exact base increase and inadequately resolve interview timing. A overstates leverage; B adds arbitrary acceptance conditions and lacks a complete opening script."}, {"vs": "opus", "pair": "3", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "tie", "why": "Both accurately use inputs, propose concrete counteroffers, handle leverage credibly, address timing, provide scripts, and advise on concessions."}, {"vs": "opus", "pair": "3", "judge": "deepseek", "order": "opus shown first", "winner": "opus", "why": "A provides direct, usable scripts; B is overly complex and lacks clear language for the user."}, {"vs": "haiku", "pair": "3", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "haiku+skill", "why": "A better handles the timing conflict, current-job fallback, and written concessions. B supplies stronger scripts but assumes Wednesday yields another offer and treats Friday as soft. Neither states the exact 9.375% base increase."}, {"vs": "haiku", "pair": "3", "judge": "gpt", "order": "haiku shown first", "winner": "haiku", "why": "A better prioritizes base and remote work and supplies usable scripts. Both incompletely report the exact raise and mishandle interview-to-offer timing. B adds unrequested signing conditions and lacks a complete opening script; A suggests an arbitrary acceptance floor."}, {"vs": "haiku", "pair": "3", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "haiku", "why": "B provides a concrete, actionable counteroffer and direct scripts, while A is overly complex and leaves key asks unresolved."}, {"vs": "haiku", "pair": "3", "judge": "deepseek", "order": "haiku shown first", "winner": "haiku", "why": "A provides a concrete counteroffer and direct scripts, while B is overly abstract and fails to prioritize the user's specific asks."}]}, {"id": "neg-3", "family": "negotiation", "prompt": "Prepare me for a negotiation with a supplier. Details in supplier.md.", "input": "supplier.md", "skill": "costa-negotiation-dynamics", "input_text": "I run a coffee roastery (USD 4M revenue). Our main green-coffee importer announced a 22% price increase from next quarter, citing harvest prices.\nFacts: we buy about 60 tonnes/year from them (around 70% of our volume). Futures prices are up around 15% this year. Two other importers have quoted 12-14% above our current price but would need 3 months to qualify their coffees. Our importer's rep told me privately they had a weak sales year.\nI want to keep the relationship; their quality has been consistent for 6 years.\n", "rubric": [{"id": "c1", "criterion": "Uses the supplied facts accurately and compares prices correctly without inventing costs.", "full_marks_when": "States that the supplier's 22% increase exceeds competing quotes of 12\u201314% by 8\u201310 percentage points and the roughly 15% futures increase by 7 percentage points. Recognizes that futures movements alone do not establish the importer's actual costs. Treats 60 tonnes as approximately 70% of annual volume and avoids calculating dollar savings without the current purchase price.", "zero_when": "Misstates material input facts, makes incorrect price comparisons, treats futures prices as proof of unjustified pricing, or invents a purchase price or dollar savings."}, {"id": "c2", "criterion": "Sets a concrete negotiating objective that protects both commercial viability and the valued relationship.", "full_marks_when": "Recommends an opening proposal and a defensible target informed by the 12\u201314% competing quotes, while acknowledging the incumbent's six years of consistent quality. Distinguishes the desired outcome from a provisional acceptance limit that must account for margins, quality, and switching costs.", "zero_when": "Offers no actionable price objective, recommends accepting 22% without negotiation, or makes immediate termination the default despite the user's wish to retain the supplier."}, {"id": "c3", "criterion": "Makes the alternative suppliers usable while addressing the qualification delay and supply dependence.", "full_marks_when": "Explains that the alternative quotes are credible leverage but cannot replace supply immediately because qualification takes three months. Proposes starting qualification now, confirming quote terms and capacity, and negotiating a bridge or phased increase to preserve supply during the transition, especially because the incumbent supplies about 70% of volume.", "zero_when": "Treats the alternatives as immediately interchangeable, advises an immediate switch or unsupported switching threat, or ignores the qualification delay and concentration risk."}, {"id": "c4", "criterion": "Prepares specific questions and evidence to test the proposed increase constructively.", "full_marks_when": "Provides questions about the cost drivers behind 22%, such as coffee origins and grades, purchasing or contract timing, freight, currency, and the duration of the increase. Uses the futures movement and competing quotes as comparison points, and seeks enough information to distinguish justified costs from negotiable pricing without accusing the supplier of dishonesty.", "zero_when": "Simply asserts that 22% is unfair, demands a reduction without investigating the basis, or supplies only generic advice to research the market."}, {"id": "c5", "criterion": "Turns the buyer's commercial value into realistic, conditional bargaining options.", "full_marks_when": "Uses the approximately 60 tonnes of annual business and long relationship to propose concrete exchanges, such as a feasible volume commitment, better forecasts, or affordable payment changes in return for a smaller or staged increase, a price hold, or a review clause. Treats the privately disclosed weak sales year as sensitive context rather than something to expose or weaponize, and avoids unsupported commitments.", "zero_when": "Recommends revealing or threatening to exploit the rep's private disclosure, offering concessions without anything in return, or promising volumes or payment terms the buyer has not established they can support."}, {"id": "c6", "criterion": "Provides preparation the user can take directly into the negotiation.", "full_marks_when": "Includes tailored opening language, a clear proposal, responses to likely supplier objections, and next steps for documenting any agreement. Identifies what the buyer should establish beforehand, especially margin impact, available cash, inventory coverage, and alternative qualification progress, so they can decide whether to accept, counter, or transition supply.", "zero_when": "Delivers only abstract negotiation principles or encouragement, with no usable proposal, meeting language, decision preparation, or follow-through."}], "runs": {"haiku-skill-1": {"cost_usd": 0.01807, "seconds": 106, "output_tokens": 21572, "files": [{"name": "negotiation-brief.md", "text": "# Negotiation brief: 22% price increase from main green-coffee importer\n\nPrepared 2026-10-08. Mode: preparation. Status: conditional (see review note).\nSource: supplier.md.\n\n**Assumptions to confirm:** \"next quarter\" = Q1 2027 (from 1 Jan 2027). \"Futures\" = arabica (ICE \"C\"). 60 t is annual. All figures in USD.\n\n## 1. Assignment and decision\n\n- **Decision needed:** Price per kg and terms for ~60 t/yr bought from the main importer, from Q1 2027.\n- **Parties and authority:** Importer's rep. Ask whether the rep can approve terms or must refer to head office. You decide on your side; no other approvals are known.\n- **Deadline:** Effective date Q1 2027. Notice period and contract status are unknown.\n- **Situation diagnosis:** Looks like a single-issue price dispute, but it is a repeated relationship (6 years) with linkable variables: volume, term, forecast, payment timing. Not a competitive auction. Alternatives give you a real option, but their 3-month lead time sets the timing.\n\n## 2. Key numbers\n\n| Item | Value | Status |\n| --- | --- | --- |\n| Importer increase | +22%, citing harvest prices | Observed (announcement) |\n| Futures move this year | ~+15% | Observed (benchmark; not the importer's cost) |\n| Alternative quotes | +12\u201314% vs current price | Observed; like-for-like unverified |\n| Alternatives vs importer's new price | ~7\u20138% cheaper | Derived: 1.12\u20131.14 \u00f7 1.22 |\n| Volume from importer | ~60 t/yr (~70% of volume), so ~86 t total | Observed |\n| Annual cost per $1/kg of current price | +22%: $13,200; +15%: $9,000; +12%: $7,200 | Derived: 60,000 kg \u00d7 % |\n\n**Core argument:** The market has moved +12\u201315% (futures and both alternative quotes). The importer is at +22%. The gap of roughly 7\u201310 points is what you are negotiating.\n\n**Timing:** If qualification starts this week, the alternatives could be supplying around early January 2027, close to the effective date. The switch threat is credible from Q1 only if you start now. Each week of delay weakens it.\n\n## 3. Relationship state\n\n| Category | Evidence | Immediate effect | Effect on later dealings |\n| --- | --- | --- | --- |\n| Prior agreement | 6 years of consistent quality; no written terms known | Baseline for the relationship | Keep quality consistency as a shared interest |\n| Your unilateral decisions | None yet | \u2014 | Starting alternative qualification is a real, reversible option |\n| Importer's unilateral decision | 22% increase announced | Cost rises from Q1 | Whatever you accept becomes the base for the next increase |\n| External change | Futures +~15%; harvest prices up | Market-wide cost pressure | Prices may fall later; a fixed term locks in today's level |\n| Process and information | Rep privately mentioned a weak sales year | Possible flexibility on volume-linked terms (unverified) | Protect the source; a privately shared remark must not be quoted back |\n\n## 4. Unknowns, in priority order\n\n| # | Priority | Unknown or hypothesis | Question to ask | What the answer changes | Status |\n| --- | --- | --- | --- | --- | --- |\n| 1 | High | Does the 22% apply to all lots and contracts, or only new ones? | \"Which contracts and shipments does the new price apply to, and from what date?\" | Whether there is an existing price to defend | Unknown |\n| 2 | High | What does switching cost? (qualification, samples, cupping time, blend retuning, quality risk) | Internal estimate; sample trials with the alternatives | The limit (section 5) | Unknown |\n| 3 | High | Is the gap between +15% (futures) and +22% explained by differentials, freight, FX, or margin? | \"Can you share the cost build-up behind the 22%?\" | The target and how hard to push | Unknown |\n| 4 | High | Are the alternative quotes like-for-like (origin, grade, lot size, incoterms, ship date, price validity)? | Request written quotes with matching specs | Whether the alternatives are a real option | Unknown |\n| 5 | Medium | Is the importer short of volume? | \"How is your volume planning for next year?\" Do not cite the private remark. | Which volume-linked package they will accept | Hypothesized |\n| 6 | Medium | Is the other ~30% of volume from already-qualified suppliers? | Internal check | If yes, shifting share is faster than 3 months | Unknown |\n| 7 | Medium | Can the importer offer a fixed term, index-linked pricing, or a quarterly review? | \"What pricing structures could you offer for 6 or 12 months?\" | Which package is available | Unknown |\n\n## 5. Positions\n\n- **Limit (unresolved):** Not set. Working formula: accept an importer price only if it is no more than the alternative price plus the one-off switching cost spread over annual spend:\n  `Max increase \u2248 alternative increase (12\u201314%) + switching cost \u00f7 annual spend with importer`\n  Illustration (assumed $5/kg, so ~$300k spend): a $15k one-off switching cost adds ~5 points, putting the limit near 17\u201319%. Replace with your actual cost and switching estimate. Note that the first quarter still has to be bought from the importer during qualification, so compare the periods after the switch.\n- **Target:** Effective increase of ~+10\u201312%, below the alternatives' quotes and near the futures move. Why feasible: the importer keeps the account (a compatible interest) and gains volume certainty. Dependent on unknowns 1 and 3.\n- **Opening:** +15% (matches the futures move, not the 22%), fixed for 12 months, against a 12-month volume commitment of ~60 t with a rolling quarterly forecast, plus agreed payment terms and a cost summary. Defensible because it uses the market's own move.\n- **Concession responses (conditional):**\n  - If they cite cost and show a build-up that supports more: move toward ~13\u201314% (the alternatives' level) only for a 12-month fixed price and the volume commitment. At that point you are buying consistency, not a discount.\n  - If they hold at 22% with no reciprocal terms: accept the increase for Q1 only, with a Q2 reset tied to the futures move and the alternative quotes. Do not sign a volume commitment on those terms.\n  - If the cost build-up shows the increase is mostly fixed differentials or freight: revise the target; do not split the difference.\n- **Likely settlement zone (hypothesis):** ~11\u201314%, if the cost build-up supports it.\n- **Credibility check:**\n  - Do not say you are switching unless qualification has actually started.\n  - Do not present the futures move as the importer's exact cost; it is a benchmark.\n  - Make a 12-month volume commitment only if your demand forecast can honor it. Check the take-or-pay risk.\n  - Do not present alternative quotes as firm unless they are in writing.\n\n## 6. Interests and issue map\n\n| Issue | Your interest | Importer's interest (hypothesis unless noted) | Evidence | Difference or compatible interest | Feasible variable |\n| --- | --- | --- | --- | --- | --- |\n| Price per kg | Lower cost, margin protection | Cost recovery, margin | Observed: 22% announced; cost basis unknown | Difference: their cost basis is unverified | Increase size, as a formula or fixed |\n| Volume | Supply security, ~60 t | Volume certainty, likely valued in a weak year | Hypothesized (private remark) | Difference: they may value volume more than price | 12-month commitment, forecast |\n| Price duration | Price predictability | Flexibility to reprice | Hypothesized | Difference: time horizon | 12-month fixed vs. quarterly review |\n| Payment terms | Cash flow | Cash timing | Unknown | Possible trade if you have room | Payment days, only if you can actually do it |\n| Forecast and planning | Supply planning | Planning visibility | Hypothesized | Compatible: both benefit from planning | Rolling quarterly forecast |\n| Quality and relationship | Consistent quality, 6-year relationship | Repeat account | Observed (6 years) | Compatible: both want to keep the account | Quality tolerance and dispute process |\n\n## 7. Linked packages\n\n**Package A (primary):**\n- You: commit to 12 months at ~60 t with a rolling quarterly forecast; agree payment terms; accept a fixed increase of target size (~+11\u201312%) for 12 months.\n- Importer: holds the increase for 12 months; shares the cost build-up; agrees a re-open trigger if futures move beyond an agreed band.\n- Why you accept: price below the alternatives' quotes; quality and supply continuity.\n- Why they might accept: volume certainty and a repeat account. Hypothesis; test with the question in section 4, row 5.\n- Dependencies: if they refuse the volume commitment, the price does not hold. If they refuse the cost build-up, reduce the target.\n\n**Package B (conditional, if they refuse any commitment):**\n- Accept the +22% for Q1 only, as a transition while qualification completes.\n- From Q2: reset to the lower of the futures move and the alternative quotes, and sign no volume commitment until the reset is agreed in writing.\n- Use only if qualification is actually under way.\n\n## 8. Uncertainty and agreement mechanics\n\n| Condition | Immediate result | Later consequence | Trigger or evidence | Response and owner |\n| --- | --- | --- | --- | --- |\n| Futures fall during 2027 | Fixed price is above market | Lost flexibility | Monthly futures tracking | Re-open trigger in the agreement; you |\n| Futures rise further | Fixed price protects you | Importer may want to re-open | Futures move beyond the agreed band | Pre-agreed re-open mechanism; both |\n| Lot quality falls below standard | Blend consistency at risk | Switch becomes more likely | Cupping on arrival | Quality tolerance and dispute process; you |\n\n**Unresolved mechanics (must be settled before any firm agreement):**\n- Which futures contract and differential basis any index uses.\n- Re-open trigger and band.\n- Whether the volume commitment is take-or-pay.\n- Who at the importer can approve changes.\n- Written confirmation of scope (which contracts, which lots, effective date).\n\n## 9. Process and rehearsal\n\n- **Escalation signals:** \"Take it or leave it\"; threats to stop supply; pressure to decide before the quarter; a refusal to share cost information.\n- **Reset words:** \"We want to keep working with you. A 22% increase is a big step, so we need to understand the cost build-up and whether it applies to all lots before we can plan. Can you walk us through it?\"\n- **Agenda:**\n  1. Confirm scope: which lots, contracts and effective date.\n  2. Cost build-up and the futures reference.\n  3. Your volume and term situation.\n  4. Your price proposal (Package A).\n  5. Written confirmation.\n- **Rehearsal:**\n  - *\"Futures are up, prices have to move.\"* \u2192 \"Agreed, the market is up about 15%. Help us understand the other 7 points: differentials, freight, FX?\"\n  - *\"22% or we can't supply.\"* \u2192 \"We want a fair number and want to keep you. We are looking at other sources as a matter of planning. What would a 12-month commitment at 60 t need to look like for you to hold the increase?\" (Only say \"looking at other sources\" if it is true.)\n  - *Refusal to move.* \u2192 Use Package B. Record non-agreement on Q2 onward.\n  - *New fact (cost sheet shows higher differentials).* \u2192 Revise the target and the limit. Do not simply split the difference.\n- **Largest plausible mistake:** Using the rep's private remark, either by quoting it or by implying you know their sales. This burns the source and the trust. Prevention: ask about volume planning and let the rep raise it.\n- **Second plausible mistake:** Bluffing about switching and then being unable to follow through. Prevention: start qualification now; claim only what is done.\n\n## 10. Next steps (this week)\n\n1. Confirm your landed cost per kg and gross margin. You cannot set the limit without them.\n2. Request written quotes from both alternatives with matching specs, validity dates and incoterms.\n3. Start sample requests and cupping with both alternatives. This is a real, reversible option and the only way the switch threat becomes credible.\n4. Send the importer a written request: effective date, which contracts and lots are affected, the cost build-up, and whether a 12-month fixed price is possible.\n5. Check your 12-month demand forecast to size a volume commitment you can honor.\n\n## 11. Round record\n\nPending. Complete after the negotiation.\n\n## Review note\n\n- **Status:** Conditional. Unresolved gates: limit (switching cost), cost build-up (target), contract scope, like-for-like alternative quotes.\n- **Most damaging plausible error:** Revealing or implying the rep's private remark, or starting a switching threat you cannot execute by January.\n- **Prevention or repair:** Keep the remark as a hypothesis; start qualification now; claim only what is in progress.\n- **Next action:** Complete the five steps in section 10, then revise this brief with the cost, quote and contract answers.\n"}]}, "haiku-skill-2": {"cost_usd": 0.01946, "seconds": 118, "output_tokens": 23295, "files": [{"name": "negotiation-brief.md", "text": "# Negotiation brief: green-coffee importer, 22% price increase\n\nPrepared 2026-10-08. Source: supplier.md. Structure follows the costa-negotiation-dynamics brief template. Dollar figures are illustrative until you replace the placeholder price.\n\n## 1. Assignment and decision\n\n- **Mode:** Preparation.\n- **Decision needed:** Whether to accept the 22% increase, or what price and terms to agree for the next 12 months on about 60 t/yr. The unit of agreement is a 12-month supply arrangement (price mechanism, volume, payment terms), not a single per-kg number.\n- **Parties and authority:** You, for our side. Confirm you can commit to a 12-month volume figure and any change in payment terms. On their side, ask early whether the rep can agree terms or needs approval.\n- **Deadline:** Increase takes effect next quarter, likely 1 Jan 2027 (confirm the effective date). Qualifying an alternative takes about 3 months, so an alternative only exists in time if you start now.\n- **Diagnosis:** A single-issue price dispute inside a six-year relationship with a supplier that provides about 70% of our volume. The framing is price-only, but the package can be widened to volume, term, payment and price mechanism. This is not an auction. Our real leverage is our alternatives (quoted, not yet qualified) and their possible volume needs (a hypothesis).\n\n## 2. What we know\n\n**Observed**\n- Announced increase: 22%, from next quarter, justified by harvest prices.\n- Futures: up about 15% this year. Verify the contract, price date and source before quoting it.\n- Two other importers quoted 12\u201314% above our current price. They need about 3 months to qualify their coffees.\n- We buy about 60 t/yr from this importer, about 70% of our volume (total about 86 t, so about 26 t comes from elsewhere).\n\n**Hypothesized**\n- The 22% is above the cost move they cite, so part of it may be margin recovery.\n- Volume matters more to them this year than usual. This comes from one private remark by the rep and is unverified.\n- The rep may not have authority to set the final price.\n\n**Unknown:** see section 4.\n\n## 3. Relationship state\n\n| Category | Evidence | Immediate effect | Later effect | Confidence |\n| --- | --- | --- | --- | --- |\n| Prior agreement | Six years of consistent quality; current terms unknown | Baseline for the discussion | Quality consistency is our main reason to stay | Known for quality; terms to confirm from our records |\n| Our unilateral decision | None yet | Whatever we do next changes options | Starting alternative qualification now is a no-regret move | Under our control |\n| Their unilateral decision | 22% already announced | They have anchored publicly | Reversing fully costs them face, so offer a route (phasing, a review clause) | Observed |\n| External change | Futures up about 15%; alternatives at 12\u201314% | Gives a market reference for the increase | Sets our realistic alternative cost | Futures to verify |\n| Process and information | Rep shared a private view of their sales | Useful, but must not be exposed | Revealing the source would damage trust with the rep | Hypothesis only |\n\n## 4. Critical unknowns\n\n| Priority | Unknown | Question to ask or evidence to get | What the answer changes |\n| --- | --- | --- | --- |\n| High | Our margin and pass-through: how much green cost can we pass to customers? | Internal: gross margin, green cost share of COGS, price-increase tolerance | Sets the limit |\n| High | Are the alternative quotes comparable? | Same grade, origin profile, Incoterms, delivery timing; are they valid for 12 months? | Sets the real alternative |\n| High | What is in the 22%? | \"Please walk us through the price build-up: benchmark, differential, freight, FX, and what changed.\" | Separates market cost from margin |\n| High | How much green do we hold at the old price? | Internal: stock and forward cover | When the pain starts; how long we can wait |\n| High | Who can approve on their side? | \"Who needs to sign off on a 12-month arrangement?\" | Whether we negotiate with a decision-maker |\n| Medium | Are the other suppliers (30% of volume) also raising prices? | Their current quotes | Strength of the market evidence |\n| Medium | Would they fix price for 6\u201312 months, or accept a benchmark formula? | Ask which structure works for them | Which mechanism to propose |\n| Medium | Can we pay faster, and is that worth something to them? | Internal: cash position | Value of a payment-term trade |\n| Medium | Can alternative samples match our blends? | Cup and roast-test samples now | Whether the alternative is real |\n| Low | Are in-transit contracts affected? | Confirm with them in writing | Exposure timing |\n\n## 5. Positions\n\n- **Limit:** **Unresolved.** You must set it. Reference points: the alternatives at 12\u201314% headline, plus whatever you will pay to avoid switching costs and quality risk (sample roasting, recalibrating blends, customer risk). Rule until the limit is set: any headline above 14% needs a specific return from them (volume, term, payment).\n- **Target:** About 12\u201314% headline on a 12-month commitment, with a benchmark-linked review clause. This matches alternatives we can actually access and is consistent with the +15% futures move.\n- **Opening:** About 10% headline, for a 12-month commitment of about 60 t, quarterly forecasts and, if cash allows, faster payment. Defensible only if we can deliver the commitment and payment terms. If not, open at 12%.\n- **Concession responses:**\n  - If they hold at 22% and say \"take it or leave it\": ask for the build-up and propose a benchmark-linked mechanism. Do not accept.\n  - If they move to about 18%: ask what changed. Offer the commitment in exchange for a number near 15%, with a review clause.\n  - If they reach about 15% (matching the futures move): accept only with the 12-month commitment and review clause in writing.\n  - If they offer a phased increase (for example 15% now, reviewed at 6 months): accept if the review trigger is written down.\n  - Below 12%: accept, provided the commitment is in writing.\n- **Offer revision reason:** Only change an offer for a real reason, such as a verified sample result, a confirmed futures move or a changed volume figure.\n- **Credibility check:**\n  - Do not say we have signed with other importers. Say we have quotes and are starting qualification.\n  - Do not offer a 12-month, 60 t commitment until our demand and cash are confirmed.\n  - Do not quote the futures figure until it is verified.\n  - Do not mention the rep's private remark.\n  - Do not threaten to leave unless we would actually do it.\n\n## 6. Interests and issue map\n\n| Issue | Our interest | Their interest (hypothesis unless noted) | Difference or compatible interest | Feasible variable |\n| --- | --- | --- | --- | --- |\n| Headline price | Pass-through of market cost, margin | Margin recovery (observed: 22% stated) | Difference: they want margin, we want the market move | Price formula: benchmark plus differential |\n| Volume | Supply security | Volume retention in a weak year | Compatible | 12-month commitment of about 60 t, with a band |\n| Price lock or term | Price certainty | Hedge against their own cost risk | Difference in risk: we gain if prices rise, they gain if prices fall | Fixed price for 6 months, or a formula with a collar |\n| Payment timing | Working capital | Cash flow now | Possible difference | Faster payment, if we can afford it |\n| Forecasting and allocation | Priority supply, fewer surprises | Planning | Compatible | Quarterly forecasts, allocation commitment |\n| Quality and spec | Consistency for our blends | Keep a good customer (observed: six years) | Compatible | Unchanged spec, no silent substitution |\n\n## 7. Linked packages\n\n**Package A (lead proposal)**\n- We give: 12-month commitment of about 60 t with a volume band; quarterly forecasts; faster payment if affordable; acceptance of a headline increase in the target range.\n- We get: headline increase in the target range; benchmark review clause (price adjusts if the benchmark moves beyond a set band); price fixed for the first 6 months; allocation priority.\n- Why they might accept: volume security in a weak year, cash timing, and a longer relationship.\n- Dependencies: if they remove the commitment, the price concession goes. If we remove the payment terms, the price moves up.\n- Feasibility: confirm our demand and cash before offering.\n\n**Package B (fallback if they insist on a formula)**\n- Phased increase, for example 15% now, with a review at 6 months against the benchmark. Price falls if the benchmark falls.\n- Cost to us: if the benchmark keeps rising, we pay more than under a fixed price.\n\n**Package C (no-regret, not a deal term)**\n- Qualify one alternative importer for 20\u201330% of volume. This builds a real alternative and reduces dependence. It has costs: minimum lots, testing time, possible quality differences. Only do it if we would actually use it.\n\n**Option D (trade to note)**\n- A fixed-price forward for 6\u201312 months. The importer carries benchmark risk; we carry the risk of falling prices. Consider it if we believe prices will rise further.\n\n## 8. Economics (illustrative)\n\nAssumption: USD 8.00/kg average green price. Replace with the actual figure. Annual volume 60,000 kg, so base spend USD 480,000 (about 12% of revenue).\n\n| Scenario | Headline | Annual cost increase | Share of USD 4M revenue |\n| --- | --- | --- | --- |\n| Their offer | 22% | +USD 105,600 | 2.6% |\n| Benchmark move | 15% | +USD 72,000 | 1.8% |\n| Alternatives, low to high | 12\u201314% | +USD 57,600 to 67,200 | 1.4\u20131.7% |\n| Our opening | 10% | +USD 48,000 | 1.2% |\n\n- Gap between their offer and the alternatives: USD 38,400 to 48,000 per year, about 1.0\u20131.2% of revenue.\n- Each 1 point of increase is about USD 4,800 per year at USD 8/kg. The formula is 0.01 \u00d7 60,000 kg \u00d7 your price.\n- These figures exclude the other 30% of volume.\n\n## 9. Scenarios and agreement architecture\n\n| Condition | Immediate result | Later consequence | Trigger | Response and owner |\n| --- | --- | --- | --- | --- |\n| Futures keep rising (+20% or more) | Their increase looks modest; our leverage falls | A fixed price becomes more valuable to us | Monthly benchmark check | Take a fixed 6-month price if offered; you |\n| Futures fall about 10% | A 12-month fixed price would overpay | Review clause or collar protects us | Benchmark drops past the band | Review clause must be in the contract; you |\n| They cut allocation in their weak year | Shipments late or short | Need alternative supply quickly | Delayed or reduced offers | Allocation clause; accelerate alternative; you |\n| Alternative samples fail cupping | Walkaway disappears | Accept best offer within the limit | Cup scores below our standard | Decide by the set date; you |\n| Alternative samples pass and are competitive | Real leverage | Move 20\u201330% of volume | Cup results and pricing | Proceed with Package C; you |\n\n**Unresolved mechanics** (must be settled before a firm agreement):\n- Benchmark source and contract month\n- Review frequency and collar size\n- Volume band and tolerance\n- Payment terms\n- Treatment of in-transit contracts\n- Who signs on each side\n\n## 10. Process and rehearsal\n\n**Agenda**\n1. Open: thank them for six years of consistent quality; state the shared goal of stable supply at a price we can sustain.\n2. Ask for the build-up: \"Walk us through what changed in the 22%: benchmark, differential, freight, FX.\"\n3. State our market view: the benchmark moved about 15% this year (once verified). We are willing to pay the market move.\n4. Explore their needs: \"What would make a longer commitment valuable to you?\" Do not mention the weak year.\n5. Present Package A, conditionally. Say we will confirm after we receive the written cost basis.\n6. Agree next steps, dates and who approves.\n\n**Reset words (if they say \"take it or leave it\")**\n\"I understand the pressure on your side, and I value the six years we have worked together. Before we decide, help me understand which part of the 22% is harvest cost and which part is the rest of the build-up. If we can agree a structure that tracks the market, I think a 12-month commitment can work for us.\"\n\n**Rehearsal**\n- \"Costs are up 22%, that's just the market.\" Reply: \"The benchmark you cite moved about 15%. Can we see the rest of the build-up?\"\n- \"No flexibility.\" Reply: \"What would make flexibility possible: volume, term or payment?\" Then present Package A.\n- \"We have other buyers.\" Reply: \"We understand, and we would rather find a structure that works for both. What do you need to hold volume?\" Do not escalate.\n- New fact: they offer 18% with a lot change. Compare cup scores and price per quality point. Ask whether the lot change is a substitution. Evaluate against the limit.\n- \"I need to check with my boss.\" Ask for a timeline and agree a decision date.\n\n**Largest plausible mistake:** Treating the alternative quotes as a live walkaway. They need about 3 months of qualification. If we start after the increase, we will have no real alternative and will accept 22%. **Prevention:** start qualification and sample testing this week.\n\n**Other mistakes to avoid**\n- Using the rep's private remark in the room. This exposes the source. Prevention: use questions instead.\n- Committing to 60 t for 12 months before confirming demand and cash. Prevention: confirm first.\n\n## 11. Next steps this week\n\n1. Pull margin, average green cost per kg and pass-through capacity. This sets the limit.\n2. Verify the futures figure: contract, date window and source.\n3. Confirm the effective date, our forward stock and the in-transit position.\n4. Request the written price build-up from the importer.\n5. Request samples from two alternative importers. Cup them against our blends, and get 12-month price validity in writing.\n6. Decide who on our side can approve a 12-month commitment.\n7. Set a decision date for the importer, about 3\u20134 weeks out.\n\n## 12. Round record\n\nPending. Complete after the negotiation.\n\n## 13. Review note\n\n- **Status:** Conditional. Unresolved gates: the limit (needs margin data), economic inputs (actual price per kg), and comparability of the alternative quotes.\n- **Most damaging plausible error:** Starting qualification too late and losing the walkaway.\n- **Prevention or repair:** Start qualification and sample testing this week.\n- **Next action:** Gather margin and cost data, then set the limit before the first meeting.\n"}]}, "haiku-skill-3": {"cost_usd": 0.01608, "seconds": 93, "output_tokens": 18409, "files": [{"name": "negotiation-brief.md", "text": "# Negotiation Brief: Green-Coffee Price Increase\n\n**Mode:** Preparation (no proposal made yet)\n**Status:** Conditional. Limit, cost breakdown and alternative-quote comparability are unresolved (see Section 10).\n\n---\n\n## 1. Assignment and decision\n\n- **Decision:** Accept, reduce or resist the 22% increase from our main importer (about 60 t/yr, roughly 70% of our green volume). Decide whether to start qualifying alternatives.\n- **Unit of agreement:** A 12-month package (price, volume, timing, review and non-price terms), not a single per-kg number.\n- **Parties and authority:** The importer's sales contact, and whoever sets their pricing (ask who that is). Our authority to commit to volume and payment terms is **unresolved**.\n- **Deadline:** The increase takes effect \"next quarter.\" Start date is **unknown**. For planning I assume 1 January 2027; confirm it.\n- **Diagnosis:** A price-and-terms negotiation inside a six-year relationship. It is not an auction. We hold one real lever, volume certainty, and one time-sensitive option, qualifying alternatives, which takes about 3 months.\n\n## 2. Relationship state\n\n| Category | Evidence | Immediate effect | Effect on later dealings |\n| --- | --- | --- | --- |\n| Prior agreement | Six years of consistent quality (our account). No price agreement described. | Quality trust is intact. | **Unknown** whether a written contract fixes price for committed volume. If it does, the 22% may not apply to it. |\n| Our unilateral decision | Two alternative importers have quoted 12\u201314% above current price. | Creates a market reference. | Only a credible option if we start qualification. Otherwise it is a bluff. |\n| Importer's unilateral decision | 22% increase announced, citing harvest prices. | Sets the opening position. | Reversible if they choose. Their volume risk rises if we move. |\n| External change | Futures up about 15% this year. | Shared benchmark for both sides. | Gives a neutral reason to tie price to a published index. |\n| Process / information | Rep privately said they had a weak sales year. | Suggests they value volume certainty (hypothesis). | This was told in confidence. Do not quote it or hint at its source. |\n\n## 3. Facts, hypotheses and unknowns\n\n**Observed (our account):**\n- 22% increase from next quarter, citing harvest prices.\n- About 60 t/yr from this importer, about 70% of volume.\n- Futures up about 15% YTD.\n- Two alternatives quoted 12\u201314% above current price, needing about 3 months to qualify.\n\n**Hypotheses (not facts):**\n- Part of the 22% is margin or other costs, not only origin price.\n- Volume certainty is worth something to the importer this year.\n- The two alternative quotes are comparable in quality and landed terms.\n\n**Unknowns, ranked by consequence:**\n\n| # | Unknown | Question to ask | What the answer changes |\n| --- | --- | --- | --- |\n| 1 | What makes up the 22%? | \"Can you send the cost build-up: origin price, freight, duties, your margin?\" | If mostly origin price, a futures-linked pass-through is defensible. If mostly margin, there is room to negotiate. |\n| 2 | Are the alternative quotes comparable? | Origin, grade, process, Incoterms, validity, minimum volume, payment terms, landed cost. | Whether 12\u201314% is the real market reference. |\n| 3 | Our switching cost | Cupping and trial-roast costs, roast-profile retuning, staff time, quality risk. | Our limit. |\n| 4 | Can we pass the increase on to our customers? | Our price elasticity, margin, competitor prices. | Our limit. Potentially the biggest factor. |\n| 5 | Contract status and effective date | Is any volume already priced? Exact start date? | Timing, and whether the 22% covers committed volume. |\n| 6 | Importer's volume situation | What volume and term would help them plan? (Ask openly. Do not cite the private remark.) | How much the volume lever is worth. |\n| 7 | Shortage allocation | Will they prioritize long-term customers if supply is tight? | Value of non-price terms, and urgency of qualifying alternatives. |\n\n## 4. Numbers (illustrative)\n\nFormula: **annual cost of an increase = 60,000 kg \u00d7 price per kg \u00d7 increase %.** Each percentage point is worth **600 \u00d7 price per kg** per year.\n\nThe price per kg is a placeholder. Replace it with the actual invoice price.\n\n| Price per kg (placeholder) | Cost of 1 point / year | Full 22% / year | Gap between 22% and 14% / year | Gap as % of $4M revenue |\n| --- | --- | --- | --- | --- |\n| $5 | $3,000 | $66,000 | $24,000 | 0.6% |\n| $7 | $4,200 | $92,400 | $33,600 | 0.8% |\n| $9 | $5,400 | $118,800 | $43,200 | 1.1% |\n\n**Read-across:** The price gap is real but small relative to revenue. Supply disruption and quality consistency may matter more than the headline percentage. Confirm the price per kg before relying on these figures.\n\n**Timing:** If the increase starts 1 January 2027 (assumed), starting qualification today finishes around 8 January. That means at least one quarter of exposure to the increase is unavoidable even if we switch.\n\n## 5. Positions\n\n- **Opening: +10% for 12 months.** Defensible because we offer 12-month volume (60 t), a quarterly forecast and firm orders three months ahead, which reduce their risk. It sits below the futures move, so expect pushback. Ask for the cost build-up before presenting it.\n- **Target: +12% for 12 months,** or +12% with a 6-month review tied to a published futures index. This matches the low end of the alternative quotes, which gives it a market anchor.\n- **Limit (candidate: about +14%, whole package, not yet set).** This is the top of the alternative quotes. The true walk-away is: alternative landed price + one-off switching cost + any premium we will pay for quality and the relationship. We cannot finalize it until Unknowns 2, 3 and 4 are answered. Do not disclose it.\n\n**Planned responses:**\n\n| If they... | We respond with... |\n| --- | --- |\n| Show an origin-driven cost build-up | Accept a futures-linked pass-through with a cap at the futures move (about +15%) and a review in 6 months. |\n| Show costs that are mostly margin | Hold at the target. Ask what volume or term would close the gap. |\n| Refuse to share costs | Pause. Say we will decide after receiving the alternative landed quotes. |\n| Ask for take-or-pay or large prepayment | Decline take-or-pay. Offer forecasts and firm 3-month orders. Consider prepayment only if its financing cost is below the increase we would save. |\n| Offer a phased increase | Accept only if the 12-month total cost is within the limit. |\n\n**Offer revision:** If we later change volume or switch suppliers, state the real reason. Do not invent a pretext.\n\n**Credibility check:**\n- Do not describe the alternative quotes as firm unless they are.\n- Do not say we have decided to switch unless qualification has started.\n- Commit only to volume we can actually forecast. Check our demand first.\n- Do not use the rep's private remark in the negotiation.\n\n## 6. Linked package and recommendation\n\n**Package:** We commit to 12 months at about 60 t, with a quarterly forecast, firm orders 3 months ahead and payment terms [to be set by us]. In return, they hold the increase at +12% for 12 months, with a 6-month review tied to a named futures index. Non-price asks: early notice of future price changes, and allocation priority in a shortage.\n\n**Why we might accept:** Landed cost stays under the switching alternative. We keep the quality we know. We avoid qualifying a new supplier.\n\n**Why they might accept:** Volume certainty in a weak year (hypothesis). Lower risk of losing 70% of our volume to a competitor (inference; we have not seen evidence of this).\n\n**Feasibility:** Volume is within our buying history. Payment terms depend on our cash position (unresolved).\n\n**Fallback:** If they refuse 12 months, offer 6 months at +12% with a review. If that still exceeds the limit, begin switching.\n\n**Recommendation:**\n1. Request the cost build-up and the alternatives' landed quotes in writing this week.\n2. Start qualifying one or both alternatives now, whatever happens in the negotiation. It is cheap insurance, makes the option credible, and reduces our 70% dependency.\n3. Propose the package after receiving the cost build-up.\n4. If the whole package is above the limit, decline it and switch from the start of the next quarter. Record non-agreement.\n\n## 7. Uncertainty and agreement architecture\n\n| Condition | Immediate result | Later consequence | Response |\n| --- | --- | --- | --- |\n| Futures keep rising (scenario: +25%) | The importer may raise further. | A fixed 12-month price becomes more valuable to us. | Lock the price early at the cap. |\n| Futures fall back to flat | The 22% looks inflated. | Weakens their justification. | Request a review and a reduced pass-through. |\n| Qualification takes longer than 3 months, or samples disappoint | No switch option this quarter. | We stay at the importer's price. | Start earlier. Keep negotiating with the review clause. |\n| Importer cuts allocation in a shortage | Supply risk. | Switching becomes urgent, even at a higher price. | Accelerate qualification. |\n| Alternative quotes expire | The reference weakens. | Less leverage. | Ask for firm validity dates now. |\n\n**Unresolved mechanics:**\n- Which futures contract and index (likely arabica) and how the basis or differential is handled.\n- Review date and who can authorize a change on each side.\n- What happens if the importer fails to deliver committed volume.\n- Whether the 22% applies to any already-committed volume.\n\n## 8. Process and rehearsal\n\n- **Escalation signals:** \"Take it or leave it.\" \"Other roasters are paying this.\" A hint that allocation will be reduced.\n- **Reset words:** \"I understand harvest costs have moved, and I want to keep working with you. Can you walk me through which parts of the 22% are origin cost and which are other costs? That is what I need to decide.\"\n- **Agenda:**\n  1. Confirm the effective date and contract status.\n  2. Review the cost build-up.\n  3. Ask about their volume planning needs.\n  4. Present the package.\n  5. Agree the next step and a date.\n- **Rehearsal:**\n  - *\"Futures are up 15%, we're at 22%.\"* Reply: \"Agreed, that gap is what I need explained.\"\n  - *\"We can't move on the 22%.\"* Reply: \"Then I need to review options. What minimum volume and term would let you move?\"\n  - *New cost data justifying 18%.* Re-run the limit and package. Check whether the remaining gap is margin.\n- **Largest plausible mistake:** Bluffing a switch we have not started, or revealing the alternatives early so the threat looks hollow. **Prevention:** start qualification now and present the quotes as market reference, not as a deadline.\n- **Second mistake:** Accepting 22% to protect the relationship without a number. **Prevention:** write the limit down before the meeting.\n\n## 9. Round record\n\n*Pending. Complete after the negotiation.*\n\n- Agreement or non-agreement:\n- Unilateral decisions:\n- Open commitments (owner, action, deadline):\n- Process observation:\n- New information and changed hypotheses:\n- Relationship-state update:\n- Next-round adjustment:\n- Review status:\n\n## 10. Review status\n\n- **Status:** Conditional. Diagnosis, relationship state, interests, credibility, linked exchange and process gates pass. Preparation positions and critical information remain open.\n- **Open gates:**\n  - Limit depends on our pass-through ability and switching cost (Unknowns 3 and 4).\n  - Cost breakdown and quote comparability are unverified (Unknowns 1 and 2).\n  - Effective date is assumed (Unknown 5).\n- **Most damaging plausible error:** Setting the limit from the 12\u201314% quotes without a landed-cost comparison, then either overpaying or switching to a supplier whose total cost exceeds the increase.\n- **Next action:** Send the written cost-breakdown request. Get the alternatives' landed quotes and validity dates. Fix the limit. Start qualification.\n\n---\n\n*Framework adapted from Lu\u00eds Almeida Costa's public negotiation method. This is a practical adaptation, not an official course template. Not affiliated with or endorsed by Lu\u00eds Almeida Costa.*\n"}]}, "opus-1": {"cost_usd": 0.1617, "seconds": 60, "output_tokens": 5084, "files": [{"name": "negotiation-prep.md", "text": "# Negotiation Prep: Response to the Importer's 22% Price Increase\n\n**Goal:** Keep the 6-year relationship and its consistent quality, and bring the increase down to what the market actually supports, which is roughly 10\u201314%. Do it by trading things the importer values: volume certainty, commitment and payment terms.\n\n---\n\n## 1. The situation in one paragraph\n\nThey've asked for +22%. The market data (futures +15%) and competitor quotes (+12\u201314%) both point lower. Your short-term leverage is limited, though. You depend on them for 70% of your volume, and the alternatives need 3 months to qualify. Their weak sales year means **they need your volume more than usual.** The best deal secures that volume for them in exchange for a price close to the market.\n\n---\n\n## 2. What the numbers say\n\n### Why 22% is hard to justify\n- **Futures are up ~15%, but futures are only part of the price you pay.** Your price per kg = futures (\"C\" price) + the origin/quality differential + importer costs (freight, warehousing, financing, margin). If the futures component is about 60\u201375% of your landed price, a 15% futures rise works out to roughly **9\u201311%** on your total price. That holds only if differentials and other costs stayed flat. *Check this before the meeting:* ask for, or estimate, the split for your main coffees.\n- **Competitors quoted +12\u201314%** for comparable coffee. That is the market's own view of the right pass-through.\n- **Importers often hedge or buy forward.** Part of what they deliver next quarter may have been bought before prices peaked.\n- The ~7-point gap between 22% and the futures move looks like **margin recovery after a weak year**, not cost.\n\n### What it costs you (fill in your real price)\nSpend \u2248 60,000 kg \u00d7 current price/kg. Example at **$8.00/kg landed \u2192 ~$480k/yr**:\n\n| Increase | Extra cost/yr | As % of $4M revenue |\n|---|---|---|\n| 22% (their ask) | ~$105,600 | 2.6% |\n| 15% | ~$72,000 | 1.8% |\n| 12% | ~$57,600 | 1.4% |\n| 9% | ~$43,200 | 1.1% |\n\n**Each percentage point is worth about $4,800/yr** at $8/kg. Rescale with your actual price: points \u00d7 600 \u00d7 price/kg.\n\nGetting from 22% down to 12\u201313% saves about **$45\u201350k/yr**. That likely matters more to your bottom line than anything else you'll negotiate this year.\n\n---\n\n## 3. Your position (set these before the meeting)\n\n| | Level | Rationale |\n|---|---|---|\n| **Opening ask** | **+9%** | Cost-based pass-through of the futures move. Ambitious but defensible. |\n| **Target** | **+12%** | Matches the low end of competitor quotes. They keep the volume plus the extras below. |\n| **Walk-away for this quarter** | **~+15\u201316%** | Above this, start shifting volume to the alternatives once they're qualified. Accept up to this only for a single quarter while you qualify them. |\n\n### Your BATNA (best alternative if no deal)\n- **Today it's weak.** You can't switch for 3 months, and 70% concentration with one supplier is a risk in itself.\n- **Strengthen it now.** Start qualifying at least one alternative importer **this week**, whatever happens in the negotiation. Request samples and cup them. Then:\n  - your walk-away becomes credible (you can honestly say \"we're qualifying alternatives\"), and\n  - you reduce the concentration risk you'd want to fix anyway. Moving 10\u201320% of volume over time is healthy, not hostile.\n\n### Their BATNA\n- Losing a 60 t/yr customer of 6 years after a weak sales year is costly. Replacing that volume takes time and sales effort.\n- They may also need cash flow and predictable forward sales.\n\n---\n\n## 4. What you can trade (cheap for you, valuable to them)\n\n| Lever | Value to them | Cost to you |\n|---|---|---|\n| **Volume commitment** (e.g., 12-month contract for 55\u201360 t) | High: they can plan and hedge, and it fixes a weak year | Low: you buy this anyway |\n| **Longer term** (12\u201318 months) | Revenue visibility | Low, as long as there's a price mechanism |\n| **Faster payment** (e.g., net 15 instead of net 30/60) or partial prepayment | Cash flow, possibly their biggest pain point | Some working capital. Check what you can afford. |\n| **Forward booking/fixing** of specific lots | Lets them hedge cleanly | Low to medium |\n| **Reference, case study or co-marketing** (\"6-year partner\") | Helps their sales pipeline | Near zero |\n| **Larger, less frequent shipments** | Lower handling and logistics costs | Storage space |\n\n**What you ask for in return:** a lower increase, plus one or more of:\n- **A phased increase**, e.g., +8% now and +4% in 6 months, conditional on the market.\n- **A price indexed to the C market:** an agreed formula (futures + fixed differential). It's transparent, and **you get the decrease if prices fall.** If they insist on high prices \"because of harvest prices,\" this is the logical consequence.\n- **A price cap** for the contract period.\n- **A review clause:** renegotiate if futures move \u00b110%.\n- Priority allocation of their best lots, or samples of new arrivals first.\n\n---\n\n## 5. How to handle the private information\n\nThe rep told you **privately** that they had a weak sales year. **Do not cite it.** It would burn the rep, who is a valuable relationship, and it would put the importer on the defensive.\n\n**Use it silently instead.** It tells you that volume commitment and payment terms are your strongest currencies. Lead with them. The pitch \"we want to commit volume to you for the next year\" lands best with a seller who is short on sales, and you never need to say why.\n\n---\n\n## 6. Meeting plan\n\n**Who:** Try to meet with someone who has pricing authority, not only the rep. The rep may be on your side internally. Help them build the case to their manager.\n\n**1. Open with the relationship (2 min)**\n> \"Six years, consistent quality. You're our most important supplier and we'd like to keep it that way, ideally to grow together. So I want to find something that works for both of us.\"\n\n**2. Ask questions before stating a number. Let them explain the 22%.**\n- \"Can you walk me through how you got to 22%? How much is futures, how much is differentials, how much is other costs?\"\n- \"How much of next quarter's coffee was already bought or hedged at earlier prices?\"\n- \"Is this a fixed new price or tied to where the market goes?\"\n\nListen. Every part of the answer that isn't \"futures\" is negotiable.\n\n**3. Present your view of the facts**\n> \"Futures are up about 15%, and futures are only part of our price, so on cost alone I calculate closer to 9\u201310%. We've also been getting market quotes, and they're coming in at 12\u201314% above our current price. At 22% the gap is too large for us to absorb or pass on to our customers.\"\n\n(Mention that you're getting quotes as plain fact, not a threat. Don't name the importers or give exact figures unless you have to.)\n\n**4. Make a package offer, not just a counter-number**\n> \"Here's what I can offer: a 12-month commitment for around 60 tonnes, [faster payment terms], and we book the lots forward so you can hedge. In return I need the increase at 9%, with a mechanism that tracks the market both ways.\"\n\n**5. Concede slowly and only in exchange**\n- Move in shrinking steps: 9 \u2192 11 \u2192 12 \u2192 (13). Never give two points at once.\n- Every concession must come with something in return: \"I can go to 11% if we add the review clause / you hold that price for 12 months.\"\n- If stuck around 13\u201315%, propose a **phased or conditional structure** rather than going higher.\n\n**6. Close and confirm in writing**\nSummarise every element (price, term, volume, payment, review clause, lots) and send an email the same day.\n\n---\n\n## 7. Likely pushback and your answers\n\n| They say | You answer |\n|---|---|\n| \"Harvest prices are up, everyone is raising prices.\" | \"Agreed, and we accept an increase. We just need it to match the cost change. Futures +15% on part of the price isn't +22%.\" |\n| \"Other importers' coffees aren't the same quality.\" | \"That's exactly why we want to stay with you. But the gap has to be reasonable. 22% versus 12\u201314% is a big premium.\" |\n| \"22% is company policy / I can't change it.\" | \"I understand. Can we bring in whoever can? I'm offering a volume commitment that I think they'll want to hear about.\" |\n| \"Take it or leave it.\" | Don't react. \"Let me think about it. In the meantime I'll need to move forward with qualifying other coffees, which I'd rather not.\" Then actually do it. |\n| \"We can do 20%.\" | Don't split the difference. \"That's movement, thanks. We're still far apart. Let's look at the structure: what if we commit X tonnes and Y payment terms?\" |\n| \"Prices might rise even more.\" | \"Then let's index it, and we both benefit from transparency in either direction.\" |\n\n---\n\n## 8. Mistakes to avoid\n- **Accepting on the spot.** Always take time (\"let me run the numbers\").\n- **Revealing your walk-away** or that you can't switch for 3 months.\n- **Mentioning the rep's private comment.**\n- **Splitting the difference** between 22% and your number. That anchors the result at around 16\u201317%.\n- **Bluffing about leaving** before you've started qualifying alternatives.\n- **Committing 100% of your volume** in a long contract without a price review clause.\n\n---\n\n## 9. Checklist before the meeting\n- [ ] Current price per kg for each main coffee, and the total annual spend\n- [ ] Rough split of futures vs. differential in your price (ask your importer or estimate)\n- [ ] Written competitor quotes (+12\u201314%) with dates, coffees and terms\n- [ ] Sample requests sent to at least one alternative importer (start qualifying now)\n- [ ] How much working capital you can offer for faster or prepaid payment\n- [ ] How much of the increase you can pass on to your own customers (wholesale/retail)\n- [ ] Decide your opening (+9%), target (+12%) and walk-away (~+15\u201316%), and write them down\n- [ ] Draft the package offer (volume + term + payment + mechanism)\n\n---\n\n## 10. One-line summary\n**Acknowledge the cost increase, contest the extra margin, and buy a market-level price with the thing they need most right now: committed volume.** At the same time, start qualifying an alternative supplier so you're never this dependent again.\n"}]}, "opus-2": {"cost_usd": 0.18625, "seconds": 74, "output_tokens": 6091, "files": [{"name": "negotiation-prep.md", "text": "# Negotiation Prep: Green-Coffee Importer Price Increase\n\n**Situation:** Main importer announced +22% from next quarter (starts Jan 1, about 12 weeks away), citing harvest prices.\n**Goal:** Keep the relationship and the quality, and cut the increase to something you can defend from market data, ideally with terms that protect you next time.\n\n---\n\n## 1. The one-page version\n\n| | |\n|---|---|\n| **Their ask** | +22% |\n| **Market evidence** | Futures up ~15% this year. Two competitors quote +12\u201314%. |\n| **Your opening** | ~+9\u201310%, based on how much of your landed price actually moves with futures (see \u00a73) |\n| **Your target** | +12\u201313% (in line with the competitor quotes) |\n| **Your walk-away** | Above ~+15% with no extra terms, move part of your volume to the alternatives |\n| **Your best leverage** | 60 t/yr is a lot of volume to a supplier with a weak sales year. You have priced alternatives. |\n| **Your weakness** | 70% concentration, and the alternatives need ~3 months to qualify |\n| **Key move** | **Start qualifying the alternatives this week**, whatever happens in the negotiation. That makes your BATNA real by January. |\n\n---\n\n## 2. Where the power sits\n\n**Your strengths**\n- **Volume.** 60 t/yr is probably a meaningful account for them, and a supplier coming off a weak year needs to keep it.\n- **Six years of payment history and loyalty.** You're a low-risk, low-cost-to-serve customer, and that's worth money to them.\n- **Market data.** +22% is above the futures move (+15%), and the futures move only affects part of your price (\u00a73).\n- **Two live quotes at +12\u201314%.** That's real evidence of the market price, not a bluff.\n\n**Your weaknesses**\n- **Concentration.** 70% of your volume is with them, so switching overnight isn't possible.\n- **Qualification lag.** The alternatives need ~3 months and the increase lands in ~12 weeks. The timing is tight but workable if you start now.\n- **Quality risk.** Their consistency affects your blends and your customers. A cheaper coffee that tastes different has a cost too.\n\n**The rep's private comment about a weak sales year**\n- Use it **only to set your own confidence and expectations**. It tells you they have more room than the letter suggests.\n- **Don't cite it, hint at it, or repeat it.** It would burn the rep, who is your best inside contact, and it would make the conversation adversarial. You don't need it anyway: the public data makes the same case.\n\n---\n\n## 3. The numbers\n\n### Why 22% is hard to justify\nYour price per kg is not all coffee futures. It's roughly:\n\n> **Landed price = C-market (futures) component + origin differential/premium + importer costs & margin (freight, warehousing, finance, margin)**\n\nA 15% futures rise only raises the **futures part** by 15%. Example: if futures make up about 60% of your price, a full pass-through is **15% \u00d7 60% \u2248 9%** on your total price. Even if futures were 100% of your price, the increase would be 15%, not 22%.\n\n> \u279c **Before the meeting:** use your invoices or contracts to estimate the futures share of your price (or ask them for it; see \u00a75). Replace the 60% with your real figure. This becomes your opening number.\n\nDifferentials may also be rising at origin this harvest, which is a legitimate part of their case. Ask for evidence instead of assuming it either way.\n\n### What it's worth (fill in your price)\nLet **P** = your current average price per kg. Annual volume = 60,000 kg.\n\n- Current annual spend from this importer = **60,000 \u00d7 P**\n- **Each 1 percentage point = 600 \u00d7 P per year**\n\n| Increase | Extra cost/yr (formula) | Example at P = $8.00/kg* |\n|---|---|---|\n| 22% (their ask) | 13,200 \u00d7 P | $105,600 |\n| 15% (futures) | 9,000 \u00d7 P | $72,000 |\n| 13% (target) | 7,800 \u00d7 P | $62,400 |\n| 10% | 6,000 \u00d7 P | $48,000 |\n\n\\*$8/kg is **illustrative only**. Use your actual price.\n\nAt the example price, getting from 22% down to 13% saves about **$43k/year**, roughly **1% of revenue**, and probably a large share of your annual profit. That's worth a well-prepared meeting.\n\n### Switching costs to include in your walk-away\nMoving volume isn't free. Before the meeting, estimate:\n- Qualification time (cupping, sample roasts, profile development)\n- Blend re-engineering and the risk of customers noticing a change\n- Possibly smaller volumes, so worse terms with a new importer at first\n- Management time\n\nThese costs are why your walk-away sits a little **above** the competitor quotes (~15%), not exactly at them.\n\n---\n\n## 4. Your BATNA, and how to make it real\n\n**BATNA (best alternative to a negotiated agreement):** move 20\u201340% of total volume to the two alternative importers at +12\u201314%, and keep the rest with the current importer.\n\n**Do this week, regardless:**\n1. Ask both alternative importers for samples of comparable coffees (match your core blend components).\n2. Start cupping and test roasts now, so qualification is done by about January.\n3. Ask them to hold their quotes for 90 days, or to give you a forward price.\n\nThis isn't about leaving. It's about **diversifying 70% concentration, which is risky even if this negotiation goes well.** A healthy target could be 50\u201360% with the main importer. You can say this openly; it's normal supply-chain practice.\n\n---\n\n## 5. Meeting plan\n\n**Who:** You plus ideally one other person (head roaster or ops), so one person talks and one takes notes. Ask that their decision-maker attends. The rep may not have authority to move the price.\n\n**Format:** In person or video, not email. Ask for the meeting now. Don't negotiate in writing first.\n\n### Phase 1: Open on the relationship (2\u20133 min)\n> \"We've worked together for six years and your quality has been consistent. That matters to us and we want to keep working together. That's why I wanted to talk directly about the increase instead of just reacting to it.\"\n\n### Phase 2: Understand their number, and ask before you state (10 min)\nAsk, then listen. Don't fill silences.\n- \"Can you walk me through how you got to 22%? How much is futures, how much is differentials, and how much is your own costs?\"\n- \"Which origins and coffees in our mix are driving it most?\"\n- \"Is this the same for all your customers, or does it depend on volume and commitment?\"\n- \"What would help you most next year: volume certainty, longer contracts, payment timing?\"\n\nThe last question matters most. Their weak year means **certainty is probably worth more to them than a few points of price.**\n\n### Phase 3: Present your view (5 min)\n> \"Here's what we're seeing. Futures are up about 15% this year, and futures are only part of our landed price, so a full pass-through is closer to [X]%. We've also had other importers quote 12\u201314% for comparable coffees. So 22% is a long way above anything we can justify.\"\n\nStay factual and calm. **Don't name the competitors.** Don't threaten. Saying you have quotes is enough.\n\n### Phase 4: Propose a package, not just a number\nOpening: **~9\u201310% plus something you can offer them** (see the trades below). Packages make it easier for both sides to move without losing face.\n\n### Phase 5: Trade concessions\n**Never give something without getting something back.** Use \"if\u2026 then\u2026\":\n> \"If we could commit to [X] tonnes for 12 months, could you get to [Y]%?\"\n\n| You can offer (cheap for you, valuable to them) | You ask for in return |\n|---|---|\n| 12-month volume commitment (e.g. 50\u201355 t, **not** all 60, to keep room for diversification) | Lower %, ideally 11\u201313% |\n| Faster payment (e.g. net 15 instead of net 30), **if your cash flow allows** | Price reduction of about the financing value |\n| Forward contracts / fixed price on part of your volume | Price protection if futures fall |\n| Testimonial, case study, or referrals | Priority on top lots |\n| Earlier purchase forecasts | Phased increase (e.g. half in Q1, half in Q3) |\n| | Price **indexed to futures** going forward (cost-plus formula), so increases follow the market |\n| | **Downward adjustment clause** if futures fall X% |\n\n### Concession plan (set this in advance; don't improvise)\n| Step | Your position | Condition |\n|---|---|---|\n| Open | ~9\u201310% | \u2014 |\n| Move 1 | 11% | In return for a volume commitment |\n| Move 2 | 12\u201313% | In return for commitment + payment terms, **plus** a futures-indexed or downward-adjustment clause |\n| Limit | ~14\u201315% | Only with strong terms (phasing, price protection, indexing). Otherwise move toward the BATNA. |\n\nMake the steps smaller as you go (that signals you're near your limit). Take a break before any move above 13%.\n\n### Phase 6: Close\n- Summarize what was agreed out loud, including price, volume, terms, start date, and review clause.\n- Follow up in writing within 24 hours.\n- If they can't decide in the room: \"Can we agree to talk again by [date]? I need to finalize our Q1 plans.\"\n\n---\n\n## 6. Likely pushback and how to answer\n\n| They say | You say |\n|---|---|\n| \"Everyone is getting 22%.\" | \"I understand. Our situation is that we're seeing 12\u201314% elsewhere, and futures are up 15% on only part of our price. Help me understand what's different about your costs.\" |\n| \"Those other importers don't have our quality.\" | \"That's exactly why I want to stay with you. But the gap has to be reasonable. Quality is worth something to us, not 8\u201310 points.\" |\n| \"The price is set by head office.\" | \"I understand. Who can we bring in on this? Or what flexibility do you have on terms, like phasing, indexing, or volume pricing?\" |\n| \"Harvest prices / differentials are up much more than futures.\" | \"That may be true. Can you show me the differentials on our main coffees? If it's documented, I'll take it into account.\" (Then actually do.) |\n| \"Take it or leave it.\" | \"I'd rather not leave it. We're not there yet. Let's look at which terms could make this work for both of us.\" Then pause. Ultimatums often soften. |\n| \"If you diversify, we can't give you the best price.\" | \"Diversifying is standard risk management for us, not a vote against you. You'd still be our main supplier by far.\" |\n\n---\n\n## 7. Things to avoid\n\n- **Don't mention the rep's comment** about their sales year, directly or indirectly.\n- **Don't bluff about leaving entirely.** It isn't credible at 70% and they know it. Talk about partial reallocation instead.\n- **Don't accept or reject on the spot.** \"Let me think about it overnight\" is always allowed.\n- **Don't negotiate only on price.** Terms (indexing, phasing, adjustment clauses) protect you next year too.\n- **Don't commit to all 60 t for a long period** in exchange for a small discount. It locks in your dependence.\n- **Don't split the difference by reflex** (e.g. 22% and 10% \u2192 16%). Anchor your number to your data.\n\n---\n\n## 8. Checklist before the meeting\n\n- [ ] Pull your actual price per kg (P) and fill in the table in \u00a73\n- [ ] Estimate the futures share of your landed price, and set your exact opening %\n- [ ] Order samples from both alternative importers, and start cupping\n- [ ] Get the alternative quotes in writing, with how long they're valid\n- [ ] Check your cash flow: can you offer faster payment? How many tonnes can you safely commit to?\n- [ ] Model the hit to margin at 10%, 13%, 15%, and 22%, and how much you could pass on to customers via your own prices\n- [ ] Decide your walk-away **in writing** before the meeting\n- [ ] Ask that their decision-maker attends\n- [ ] Bring a printed one-pager: futures chart + your figures (no competitor names)\n\n## 9. After the meeting\n\n- Send a written summary within 24 hours.\n- Keep qualifying the alternatives even if the deal is good. Target 50\u201360% concentration in the next 12 months.\n- Set up a quarterly price review based on futures, so next time isn't a surprise.\n- Thank the rep privately. Keeping that relationship is worth a lot.\n"}]}, "opus-3": {"cost_usd": 0.19106, "seconds": 74, "output_tokens": 6293, "files": [{"name": "negotiation-prep.md", "text": "# Negotiation Prep: Green-Coffee Importer Price Increase\n\n**Situation:** Your main importer announced +22% from next quarter (starts Jan 1, 2027, about 12 weeks away), citing harvest prices.\n**Goal:** Keep the relationship and its 6-year quality record, and land the increase in the low teens or below, with terms that protect you from the next price swing.\n\n---\n\n## 1. One-page summary (read this before the call)\n\n| | |\n|---|---|\n| **Their ask** | +22% |\n| **Market evidence** | Futures up ~15%. Since futures are only part of the landed price, a pure pass-through should come out *below* 15% (see \u00a73) |\n| **Your alternative** | Two importers at +12\u201314%, but they need ~3 months to qualify |\n| **Opening position** | ~+10%, backed by the cost math |\n| **Target** | +12\u201313%, or a market-indexed price |\n| **Walk-away line** | Above ~+15% with no offsetting terms: start shifting 20\u201330% of volume to an alternative |\n| **Your best trades** | 12-month volume commitment, faster payment, forward contracting, indexed pricing |\n| **Don't** | Reveal what the rep told you. Threaten to leave everything. Accept or reject on the first call |\n\n**Do this week, before you negotiate:** ask both alternative importers for samples and start qualifying them now. That turns your alternative from a 3-month hypothetical into a real option by the time the increase takes effect. It's the biggest source of leverage you have, and it costs very little.\n\n---\n\n## 2. What each side has\n\n### Your position\n- **Strengths:** 60 t/yr is a significant account. You've been a steady 6-year customer. You have credible quotes 8\u201310 points lower. Their stated reason (harvest prices) can be checked against public futures data.\n- **Weaknesses:** 70% of your volume sits with them, so switching fully would be slow and risky for your blends and consistency. Qualifying the alternatives takes 3 months. You value the relationship, and they probably know that.\n- **Your real alternative (BATNA):** You wouldn't actually switch everything. The realistic move is to keep them as primary supplier and **move part of your volume** (e.g. 15\u201320 t) to a cheaper importer once qualified. That's credible, and they'll feel it.\n\n### Their position (inferred)\n- **Pressure on them:** A weak sales year means losing even part of a 60 t account hurts. They need volume and dependable customers more than usual right now.\n- **What they likely want:** Volume certainty, fast payment (higher coffee prices mean higher financing costs for them), and margin recovery after a bad year.\n- **Read on the 22%:** It's probably anchored high, partly to rebuild margin. It's also possible their own costs rose more than futures did (differentials, freight, financing). Ask before you assume.\n\n### On the rep's private comment\nTreat it as background only. **Never cite it, and never hint that you know it.** Doing so would burn the rep, who may be useful to you for years, and would expose that you're working from inside information. You can use the same leverage legitimately by stressing what *you* bring: volume, reliability, commitment. Those are worth more to a supplier in a weak year, and you don't need to say why.\n\n---\n\n## 3. The numbers\n\n### Why \"futures +15%\" doesn't justify +15%, let alone +22%\nGreen coffee is priced as **futures (\"C\" price) + differential** (origin and quality premium), plus logistics. If only the futures part moved, the total price rises by *less* than 15%.\n\nIllustrative (plug in your real numbers):\n\n| Component | Before | After (futures +15%) |\n|---|---|---|\n| C price | $2.50/lb | $2.875/lb |\n| Differential + logistics | $0.80/lb | $0.80/lb |\n| **Total** | **$3.30/lb** | **$3.675/lb \u2192 +11.4%** |\n\nSo a pure pass-through of the futures move is roughly **+10\u201312%**, depending on your coffees' differential share. Higher-premium specialty lots move even less in percentage terms. That's the evidence for your opening number.\n\n**What could legitimately push it higher:** differentials rising for specific origins, higher freight, or higher financing costs because each container now ties up more cash. Ask them to show which of these apply.\n\n> \u26a0\ufe0f Check whether \"futures up 15% this year\" lines up with the period your current price was set. If your current contract was fixed when futures were already partly up, the gap is smaller. If it was fixed earlier, the gap may be larger. Pull the C price on the date your current pricing was agreed.\n\n### What the gap costs you\nLet **S** = your current annual spend with them (60 t \u00d7 current price).\n\n| Scenario | Extra annual cost | If S = $400k |\n|---|---|---|\n| +22% (their ask) | 0.22 \u00d7 S | $88,000 |\n| +15% (futures) | 0.15 \u00d7 S | $60,000 |\n| +12% (target) | 0.12 \u00d7 S | $48,000 |\n| +10% (opening) | 0.10 \u00d7 S | $40,000 |\n\n**The gap between 22% and 12% is worth about 10% of S per year (~$40k on a $400k spend, or about 1% of your $4M revenue).** That's worth a well-prepared negotiation, but not worth jeopardizing quality on your core blends.\n\n### True cost of switching (count this honestly)\nQualifying samples, roast-profile development, re-dialing blends, customer-noticeable flavor changes, staff time, and the risk of an inconsistent first shipment. If switching 20 t costs, say, $10\u201315k all-in for the first year, then an alternative at +13% really costs about +16\u201317% in year one. **That's why your walk-away is around 15%, not 13%.** It also means a partial switch makes more sense than a full one.\n\n---\n\n## 4. Your positions\n\n| | Number | Rationale |\n|---|---|---|\n| **Opening** | +10% | Grounded in the futures pass-through math. Credible, not insulting |\n| **Target** | +12\u201313% | Matches the alternative quotes, and you keep the relationship |\n| **Acceptable with good terms** | +14\u201315% | Only with a real benefit attached: indexed pricing, phasing, fixed price for 12 months, better payment or credit terms |\n| **Walk-away** | >15% flat with no concessions | Tell them you'll move part of your volume (not all). Follow through |\n\n---\n\n## 5. Things to trade (cheap for you, valuable to them)\n\nOffer these **only in exchange** for price movement, one at a time, and get something back each time (\"If I do X, can you do Y?\").\n\n1. **12-month volume commitment.** Commit to e.g. 55\u201360 t in writing. This is the most valuable item in a weak sales year.\n2. **Forward contracting.** Fix price on part of the next 6\u201312 months now. They get a sale on the books, you get price certainty.\n3. **Faster payment.** E.g. 15 days instead of 30, or a deposit. This cuts their financing cost, which rises with coffee prices. Only offer it if your cash flow allows.\n4. **Indexed pricing (strongly recommended).** Price = C price + fixed agreed differential. This removes the \"harvest prices\" argument in both directions: you pay more if the market rises and less if it falls. It's fair, it's transparent, and it protects you if the market cools in 2027.\n5. **Phasing.** E.g. +8% now and +4% in 6 months, or reviewed against the index at that point.\n6. **Non-price value.** Priority on the best lots, consigned warehousing, longer credit terms, free sample shipments, co-marketing (origin stories, naming them as a partner).\n\n**What to ask for if price won't move:** a 12-month price lock, a cap on future increases, or a market-reversal clause (\"if futures fall X%, our price is reviewed downward\").\n\n---\n\n## 6. Running the conversation\n\n### Before the call\n- [ ] Request samples from both alternative importers **this week**\n- [ ] Pull the C-price history: today's level vs. the date your current pricing was set\n- [ ] Calculate your actual annual spend S and the differential share for your main coffees\n- [ ] Decide which trades you can afford (cash flow for faster payment? can you commit to 12 months?)\n- [ ] Set up a meeting or call. Don't negotiate over email, and if you can, talk to someone with pricing authority, not just the rep\n\n### Opening (acknowledge the relationship, then question the number)\n> \"We've worked together for six years and your quality is a big reason our coffee is what it is. I want to keep building on that. Higher harvest prices are real and I expect to share some of that. But 22% is well above what I'm seeing in the market. Futures are up about 15%, and since that's only part of our landed price, a straight pass-through comes out closer to 10\u201312%. Help me understand what's behind the rest.\"\n\nThen **stop talking** and let them explain.\n\n### Questions to ask\n- \"What's driving the gap between the futures move and 22%? Differentials, freight, financing?\"\n- \"Is this a flat increase across all coffees, or does it vary by origin?\"\n- \"Is this the same increase all your customers are seeing?\"\n- \"What would it take to get closer to the market move? What matters most to you right now: volume, timing, payment?\"\n  (That last question gives them room to say they need volume. Let them be the one to say it.)\n\n### Mentioning the alternative (calmly, not as a threat)\n> \"I'll be open with you: I've had quotes in the 12\u201314% range, and I've started qualifying samples, partly just because 70% with one supplier is a concentration I should manage. I'd much rather keep the bulk of our volume with you. But at 22% I'd have to move a meaningful part of it.\"\n\nThis is true, specific, and not an ultimatum. It also gives you a lasting reason to diversify a bit regardless of the outcome.\n\n### Handling likely responses\n\n| They say | You respond |\n|---|---|\n| \"Everyone's raising prices, it's the market.\" | \"Agreed, and I'll pay a market increase. Let's tie it to the market directly: C price plus a fixed differential.\" |\n| \"Other importers' quality won't match ours.\" | \"That may be true, which is why I'd rather stay. But I need the number to be defensible, and I'm qualifying samples to find out.\" |\n| \"22% is already our best price.\" | \"What could you do with a 12-month volume commitment and faster payment?\" |\n| \"We can do 18%.\" | Don't split the difference. \"That's moving in the right direction. Here's what I'd need to get to 12\u201313%: \u2026\" and make a trade |\n| \"We need an answer now.\" | \"I want to get this right for both of us. I'll come back by [date].\" Never close on the first call |\n| \"Our differentials went up a lot.\" | \"That's fair. Can you show me which origins? Let's set the differential per coffee and index the rest.\" |\n\n### Closing\n- Summarize the agreement out loud and **confirm it in writing the same day**: price or formula, volume, duration, payment terms, review or reversal clause.\n- End warmly. You'll be working with these people for years.\n\n---\n\n## 7. Risks and what to watch\n- **Overplaying the alternatives:** they know switching is hard for you. Keep it to *partial* volume and keep it believable.\n- **Quality slippage after a hard negotiation:** watch the next few shipments' cupping scores. Locking in specific lots or grades in writing helps.\n- **Concentration risk:** 70% with one supplier is a business risk regardless of this negotiation. Qualifying a second importer for 15\u201325% of volume is worth doing even if you get a great deal. Doing it also keeps every future negotiation honest.\n- **The market can fall:** that's the main reason indexed pricing or a reversal clause is worth more than a few points of fixed discount.\n\n---\n\n## 8. Fill in before the meeting\n\n| Item | Value |\n|---|---|\n| Current price ($/kg or $/lb), main coffees | |\n| Annual spend S | |\n| C price when current price was set | |\n| C price today | |\n| Differential share of your price (%) | |\n| Alternative quote A (% / $) + sample ETA | |\n| Alternative quote B (% / $) + sample ETA | |\n| Max you can commit (t, months) | |\n| Payment terms you can offer | |\n| Decision-maker at the importer | |\n| Your deadline to respond | |\n"}]}, "haiku-1": {"cost_usd": 0.00988, "seconds": 65, "output_tokens": 12351, "files": [{"name": "negotiation-prep.md", "text": "# Negotiation prep: green-coffee importer price increase\n\n## Situation\n\n- Main importer: about 60 t/yr, roughly 70% of our volume (so total buying is about 86 t/yr).\n- Importer announced a **+22%** increase from next quarter, citing harvest prices.\n- Futures are up about **15%** this year.\n- Two other importers quoted **+12 to 14%** over our current price, but need about **3 months** to qualify their coffees.\n- The importer's rep privately said they had a weak sales year. **Do not cite this or hint at where it came from.**\n- We want to keep the relationship. Quality has been consistent for 6 years.\n\n## The numbers\n\nLet P = our current price per kg. Annual volume with the incumbent is 60,000 kg, so the extra cost of an increase of x% is **x% \u00d7 P \u00d7 60,000**.\n\n| Outcome | Increase | Extra cost per year | Saving vs. accepting 22% |\n|---|---|---|---|\n| Hold current price | 0% | 0 | 13,200 \u00d7 P |\n| Opening ask | 5% | 3,000 \u00d7 P | 10,200 \u00d7 P |\n| Target | 10\u201312% | 6,000\u20137,200 \u00d7 P | 6,000\u20137,200 \u00d7 P |\n| Walk-away | 14% | 8,400 \u00d7 P | 4,800 \u00d7 P |\n| Accept their number | 22% | 13,200 \u00d7 P | \u2014 |\n\nIllustrative only, at P = $8/kg (replace with our actual price):\n- 22% costs about **$106k/yr** (about 2.6% of $4M revenue).\n- 15% (futures-matched) costs about $72k/yr.\n- 14% (walk-away) costs about $67k/yr.\n\nThe gap between 22% and futures (7 points) is the part we need them to justify.\n\n## Our leverage\n\n- **Credible alternatives at +12\u201314%.** They are 8 to 10 points cheaper than the incumbent's new price. This is our anchor.\n- **Their volume need.** We are a large, six-year customer. In a weak sales year, keeping 60 t committed is worth something to them.\n- **An objective benchmark.** Futures up 15% is neutral and easy to defend.\n- **Terms we can offer.** A 12-month volume commitment, index-linked pricing, faster payment, or consolidating our remaining volume with them.\n\n## Their leverage\n\n- **Our switching cost and quality risk.** Sample roasting, cupping, roast-profile retuning, and blend consistency for our customers.\n- **Time.** Alternatives take about 3 months, so the increase lands before they are live unless we start now.\n- **Our dependence.** They supply 70% of our volume. We need to look like we can actually move.\n- **Partly real cost pressure.** Harvest prices may really be up. We should acknowledge this and then test how much of it is real.\n\n## Strategy\n\n1. **Start qualifying an alternative this week**, whether or not we end up using it. If \"next quarter\" means 1 January 2027, three months from now is early January, roughly when the increase takes effect. A real alternative is worth more than any argument.\n2. **Get written quotes** from both alternatives: price, validity date, grade and lot spec, capacity for 60 t, Incoterms, payment terms, and a sample shipment.\n3. **Trade, don't concede.** Offer one non-price lever at a time, each in exchange for movement on price:\n   - 12-month volume commitment (60 t, or 60 t \u00b110%) in exchange for a capped increase.\n   - **Index-linked pricing with a collar**: fixed differential plus futures pass-through, capped, and reversing symmetrically if futures fall. This is the most valuable structural ask. It turns \"15% vs 22%\" into a formula.\n   - **Phased increase**: part now, the rest at month 6, adjusted to futures.\n   - **Faster payment terms** (for example 14 days instead of 45). They may value cash in a weak year. Price the benefit against our cost of capital first.\n   - **Lot or grade options**: a different lot, grade, or origin with a similar cup profile at lower cost.\n   - **Share of wallet**: if we buy the other 30% from anyone, offer it to them at a stable price.\n4. **Concede in shrinking steps**, each paired with a request (for example 5% \u2192 8% \u2192 10% \u2192 12% \u2192 13%). Do not split the difference early.\n\n## Concession plan\n\n- **Open: +5%**, tied to a 12-month commitment and index-linked pricing.\n- **Target: +10 to 12%.**\n- **Walk-away: +14%** (top of the alternative range), and only with the commitment and pricing mechanism in place. Raise this only if the switching-cost estimate is high.\n- **If they hold at 22%:** do not sign a multi-quarter deal at that rate. Accept one quarter at most while we qualify alternatives, then move 20 to 30% of volume to an alternative (dual-sourcing). Keep the incumbent on the rest, and reopen at renewal.\n\n## Talking points (draft)\n\n**Open: relationship first**\n\"We value the last six years and want to keep working together. A 22% increase is large, and I need to understand it before I can take it back to my side. Can you walk me through the build-up?\"\n\n**Anchor on the market**\n\"Futures are up about 15% this year. That's a fair benchmark for the market part, and I'm open to moving with it. The other seven points need a justification.\"\n\n**Offer a trade**\n\"If we can agree a number that works for a 12-month term, I can commit 60 tonnes and give you forward visibility. That's worth something to you.\"\n\n**Show the alternative, without naming the price**\n\"I'm not in a position to accept 22% without looking at alternatives. Two other importers have quoted us, and their prices are meaningfully below your new number. I'd rather stay, but I need a number that makes sense.\"\n\nOnly cite quotes we have in writing. Do not overstate them.\n\n**Close or set a deadline**\n\"Could we agree on 12% from next quarter, reviewed at six months against futures, with a collar? If not, let's agree what each of us needs to decide by [date].\"\n\n## Questions to ask them\n\n1. What is in the 22%? Split it into futures-linked, differentials, freight, FX, and other costs.\n2. Does the increase apply to all lots or only some?\n3. Will you commit to a 12-month price with index-linked adjustment?\n4. What volume and price can you commit to for next year?\n5. Would faster payment terms change your number?\n6. Is there a lot or grade with a similar profile at a lower cost?\n7. Can you hold the current price until [date] while we discuss?\n\n## Before the meeting\n\n- Our current price per kg, and whether the 22% applies to FOB or landed price.\n- The breakdown of the increase (futures vs. differentials vs. freight and FX).\n- Our gross margin, and how much of the increase we can pass on to our customers. If we can pass on half, the real pressure is lower.\n- Switching costs: sample roasting, cupping, roast-profile retuning, blend changes, staff time, customer testing, new minimum orders, and payment terms.\n- Alternative quotes: validity dates, grade and lot spec, capacity for 60 t, payment terms.\n- The exact start date of \"next quarter.\"\n- Pricing trend for the other ~30% of our volume.\n\n## Cautions\n\n- **The rep's private comment about a weak sales year.** Do not cite it or hint at the source. Use it only to shape what we offer (volume and terms), and ask open questions such as \"What would help you hit volume this year?\"\n- **Do not overstate alternatives.** Only mention quotes we actually hold, and do not say qualification is done when it isn't.\n- **Do not reveal our walk-away.** Show the range of alternatives, not our internal ceiling.\n- **Test alternative samples** before they go into customer product.\n- **Check credit** before offering any prepayment.\n"}]}, "haiku-2": {"cost_usd": 0.00938, "seconds": 62, "output_tokens": 11535, "files": [{"name": "negotiation-prep.md", "text": "# Negotiation prep: green-coffee importer price increase\n\n## Situation\n\n- Main importer announced **+22%** from next quarter, citing harvest prices.\n- We buy **~60 t/yr** from them (~70% of our volume).\n- Futures are **up ~15%** this year.\n- Two other importers quoted **+12\u201314%** over our current price, but need **~3 months** to qualify their coffees.\n- The importer's rep privately said they had a **weak sales year**. Do not cite this directly (see Pitfalls).\n- Goal: keep the relationship. Quality has been consistent for 6 years.\n\n## Numbers\n\nAssumption: landed price of **$8/kg** (replace with actual). Current annual spend with importer: 60,000 kg \u00d7 $8 = **$480,000**.\n\n| Scenario | Increase | Added annual cost |\n|---|---|---|\n| Importer ask | 22% | $105,600 |\n| Futures move passed through in full | 15% | $72,000 |\n| Alternative quotes | 12\u201314% | $57,600\u201367,200 |\n| Our target | 10\u201312% | $48,000\u201357,600 |\n\n- Importer's ask is **7 pts above the futures move**: ~$33,600/yr.\n- Importer's ask is **8\u201310 pts above the alternatives**: ~$38,400\u201348,000/yr.\n- $105,600 is **2.6% of our $4M revenue**.\n\n**Key check:** futures are only one part of landed cost. If futures are ~half the cost, a 15% futures move passes through to roughly **7\u20138%** on the total price. A 22% increase then requires ~14 pts from differentials, freight, or importer margin. Ask for the breakdown. The rep may argue farm-gate prices rose more than futures, so get the actual index data.\n\n## Position\n\n- **Open: +6\u20138%.** Justify as the futures pass-through (~half of 15%), with nothing for margin or differential increases.\n- **Target: +10\u201312%.** Covers the commodity move plus a modest differential change, and beats the alternatives.\n- **Walk-away: above +14%.** That is the higher alternative quote. Above it, we move volume to the alternatives once qualified.\n- Between 12% and 14%, stay only if terms beat the alternatives (price formula, phasing, term).\n\n## Leverage\n\n**Ours**\n- Volume and commitment: 60 t/yr is a meaningful account. Offer a 12-month commitment, quarterly forecasts, and reliable payment.\n- Real alternatives: two quotes at +12\u201314%. Cite them only if we are prepared to act on them.\n- Six years of consistent quality and a working relationship.\n- Their weak year: they likely value volume certainty. Use it through questions, not by quoting the rep.\n\n**Theirs**\n- We depend on them for ~70% of volume, and switching takes ~3 months plus sample roasting.\n- Our blend profiles are built around their coffee. Switching carries quality risk.\n- We have not yet sampled alternatives, so our walk-away is not yet credible.\n\n## Timing\n\n\"Next quarter\" is probably Q1 2027. Confirm the effective date. Requesting samples from the two alternatives **this week** makes them roast-ready around the same time the increase lands. Without that, the walk-away is an empty threat.\n\n## Concessions\n\n**Give (cheap for them, valuable in a weak year)**\n- 12-month term at a fixed 60 t volume.\n- Quarterly volume forecasts.\n- Faster payment, or a prepayment discount.\n- Consistent lot sizes and shipment schedule.\n\n**Get**\n- A lower percentage increase.\n- A price formula with a two-way adjustment (prices fall when futures fall).\n- Phasing (e.g., half now, half in 6 months).\n- A price review clause at 6 months.\n- Open-book cost breakdown.\n\n## Opening script\n\n> \"We want to stay with you, and six years of consistent quality is the reason. A 22% increase is well above the futures move, and we have quotes at 12\u201314% from others. Before we discuss numbers, can you walk us through the cost build-up: farm price, differential, freight, duties, and margin? If the increase is futures-driven, we can agree a formula. If it is differentials or margin, we need to look at those. In return we can commit to 12 months at 60 t, with quarterly forecasts and [payment terms].\"\n\nAsk instead of asserting: \"What volume are you aiming for this year, and what would help you hit it?\"\n\n## Likely objections\n\n| They say | Response |\n|---|---|\n| \"Harvest prices are up more than futures.\" | \"Show us the index data. Futures are up 15%. Anything more needs a documented change in differential or freight.\" |\n| \"The whole market is moving.\" | \"The two alternatives are 12\u201314%, not 22%.\" |\n| \"It's 22% or we can't guarantee supply.\" | \"Then we need that in writing for 60 t over 12 months. We'll plan around it.\" |\n| \"Other importers can't match our quality.\" | \"Then let's cup the samples side by side. Our blend profile is what matters, so we'll test it.\" |\n| \"We can only do 22%.\" | \"Then we'll split volume and qualify the alternatives now. Can you phase the increase or apply it to some lots only?\" |\n\n## Questions to send in writing before the meeting\n\n1. Itemised breakdown of the price: farm/index, differential, freight, insurance, duties, importer margin. Which changed, and by how much?\n2. Which futures contract and date is the reference?\n3. Does the 22% apply to all lots or only some?\n4. Will the price reprice down if futures fall?\n5. Can they guarantee 60 t at the new price for 12 months?\n6. What payment terms do they offer for a volume commitment?\n\n## Pitfalls\n\n- **Don't bluff.** Only cite alternatives we can actually buy from. Start sampling now so the threat is real.\n- **Don't cite the rep's private remark.** It burns the source. Use the question in the script instead.\n- **Don't concede price before seeing the cost breakdown.**\n- **Don't lock a 12-month term at a fixed price** without a two-way adjustment clause.\n- **Check their financial health.** A weak sales year can mean cash strain, late shipments, or allocation shortfalls. Qualification of alternatives is also a hedge against that.\n\n## Next steps (this week)\n\n1. Confirm the effective date, actual landed price per kg, and total annual spend.\n2. Send the written questions above.\n3. Request sample lots from both alternatives and set up side-by-side cupping and roasting.\n4. Decide how much of the increase we can pass on to customers. This sets our real ceiling.\n5. Agree the walk-away number with whoever signs off on the deal.\n\n## Missing information\n\n- Current price per kg and actual annual spend (the table uses $8/kg as a placeholder).\n- Who supplies the other ~30% of volume.\n- Gross margin and customer price sensitivity.\n- Importer's share of their own sales, and our payment history with them.\n- Current contract terms, if any.\n"}]}, "haiku-3": {"cost_usd": 0.01213, "seconds": 84, "output_tokens": 15949, "files": [{"name": "negotiation-prep.md", "text": "# Negotiation prep: green-coffee importer price increase\n\nPrepared 2026-10-08. Source: supplier.md.\n\n## 1. Situation\n\n- Importer raises price 22% from next quarter (likely 1 Jan 2027, about 12 weeks away), citing harvest prices.\n- We buy about 60 t/yr from them, roughly 70% of our green volume (about 86 t total).\n- Futures are up about 15% this year.\n- Two other importers quoted 12-14% above our current price. Each needs about 3 months to qualify.\n- The importer's rep privately said they had a weak sales year.\n- Goal: keep the relationship and most of the volume, and bring the increase down.\n\n**Timing:** Qualifying an alternative starting this week finishes about when the increase starts. There is no slack. Start qualification now, whatever happens in the negotiation.\n\n## 2. The numbers\n\nI don't have our current price per kg, so the figures use a range. Replace P0 with the real number.\n\n- Annual cost of the 22% increase = 60,000 kg x P0 x 0.22 = **13,200 x P0**\n- Each 1 point of price = 60,000 x P0 x 1% = **600 x P0 per year**\n\n| Current price (P0) | Annual spend at current price | Cost of +22% | Cost of +13% (alt. midpoint) | Saving from switching (22% vs 13%) |\n|---|---|---|---|---|\n| $6/kg | $360,000 | $79,200 | $46,800 | $32,400 |\n| $8/kg | $480,000 | $105,600 | $62,400 | $43,200 |\n| $10/kg | $600,000 | $132,000 | $78,000 | $54,000 |\n\nAt $8/kg:\n- The increase is about 2.6% of revenue ($105.6k of $4M).\n- Each 1 point of price is about $4,800 a year.\n- The price gap versus the futures move (22% vs 15%) is 7 points, about $33,600 a year. This is the core argument, with the caveat that futures are only one part of the price.\n- Alternatives at +12-14% over our current price are about 7-8% cheaper than the importer's new price (1.12/1.22 = 0.918; 1.14/1.22 = 0.934).\n\n**Check before using the alternative quotes:** Confirm they are quoted against the pre-increase price. If they are quoted against a new price, they are not cheaper. Also confirm they are comparable: origin, crop year, grade, cupping score, FOB versus landed, currency, lot size, payment terms, and how long the quote is valid.\n\n## 3. Leverage\n\n**Ours:**\n- 70% of our green volume is significant for them, especially in a weak sales year.\n- Six years of consistent quality and, presumably, reliable payment (confirm).\n- Real alternatives exist, and the futures data supports part of their increase. Our argument should be about the excess, not the whole increase.\n\n**Theirs:**\n- Quality consistency and our switching cost (roast profiles, customer-facing consistency).\n- Our 3-month lead time means we cannot switch before the increase starts.\n- Switching saves us real money but a modest share of revenue (about 1% at $8/kg). Switching carries its own costs and risks.\n\n**The weak sales year:**\n- They probably value volume certainty and cash flow more than a headline price. Use this to shape the offer.\n- Do not quote the rep's private comment back. It would expose the rep and damage trust.\n- A weak-sales importer may also be financially stressed. Ask about supply continuity, and do not prepay.\n\n## 4. Positions\n\n- **Opening:** +10%. Justify it with the futures move (15%) and a 12-month volume commitment.\n- **Target:** +12%. Roughly matches the alternatives and keeps the volume.\n- **Ceiling:** +15%. Above this, start moving volume to the alternatives, beginning with about 30% (about 18 t).\n\nWhy +15%: at $8/kg, switching saves about $4,800 a year per point above the 13% alternative price. Above roughly 15%, the saving starts to cover the one-off cost of switching. Revisit the ceiling once switching costs are estimated, and move it up if their cost breakdown shows real increases in differential or freight.\n\n## 5. Package to offer\n\nTrade the things they value (volume certainty, forecast visibility, payment timing) for price.\n\n1. **Term:** 12-month commitment at about 60 t (or 65 t) with a stated minimum.\n2. **Price:** +10-12% fixed for 6 months, then reset to futures plus a fixed differential. Include a cap. This also lets our price fall when futures fall, which is fair and easy to justify.\n3. **Payment:** Move from net 30 to net 15, or pay on shipment. Avoid prepayment given their weak year and counterparty risk. Confirm our cash position allows this.\n4. **Forecast:** Quarterly volume forecast, and first look at new lots.\n5. **Lot flexibility:** Let them choose lots for part of the volume, if quality stays within our spec.\n6. **Transparency:** Ask for an open-book breakdown of the 22%.\n\n**Fallbacks, in order:**\n- A (best): 12 months, +12%, index-linked after 6 months, net 15.\n- B: 6 months at +14%, then review.\n- C: Split volume (for example 80/20) and begin shifting to the alternatives.\n\n## 6. Opening script\n\n> \"We've valued this partnership for six years, and the quality has been consistent. That's why we want to find a way to stay. A 22% increase is hard for us to absorb. Futures are up about 15%, so I'd like to understand the rest of the increase. Can you walk me through the cost build: differential, freight, currency, and any change in your margin?\"\n\nThen:\n\n> \"We're willing to commit to 12 months of volume. What price can you offer in return?\"\n\nIf you raise the alternatives, state only what is true and what you have actually done:\n\n> \"We have quotes from two other importers at about 12-14% above our current price. We'd rather stay with you, but the numbers need to work. We've also started qualifying them, so we're not in a position where we have no options.\"\n\nOnly say the last sentence if qualification has started.\n\n## 7. Objections and responses\n\n| They say | You say |\n|---|---|\n| \"Harvest prices are up across the market.\" | \"Futures are up about 15%. Show me how you get to 22%. Other importers are quoting 12-14%.\" |\n| \"This is the new price. We can't go lower.\" | \"Then the alternatives are the market price, and we'll have to qualify them. We'd rather find a structure that works for both of us. What can you do on term or payment?\" |\n| \"Volume discounts are only for bigger accounts.\" | \"We're 70% of our green volume with you. A 12-month commitment is worth something to you in a weak year.\" |\n| \"We need cash flow certainty.\" | \"We can move to net 15 or pay on shipment. We can't prepay, but we can commit to volume and forecasts.\" |\n| \"Quality is why we charge more.\" | \"We agree, which is why we want to stay. Let's cup our current lots against the alternative samples side by side.\" |\n| \"Take it or we'll allocate elsewhere.\" | \"I understand. If allocation is tight, we'd like priority in return for a 12-month commitment. What can you offer?\" |\n| \"We'll have to think about it.\" | \"That's fine. When can we agree on a decision date, before the new quarter starts?\" |\n\n## 8. Do not\n\n- Bluff about alternatives you have not qualified.\n- Reveal your deadline or that you need coffee before the new quarter.\n- Accept a verbal deal. Get price, volume, term, payment, and a decision date in writing.\n- Quote the rep's private remark.\n- Make the first move a split-the-difference offer.\n- Concede on price without getting something in return.\n- Prepay.\n\n## 9. Before the meeting\n\n- **Contract:** Price-adjustment clause, notice period, minimum volume, exclusivity, and whether shipments already agreed are at the current price.\n- **Scope:** Does the 22% apply to all coffees or only some lots?\n- **Current price:** Our $/kg, so the table above can be replaced with real numbers.\n- **Margin:** Gross margin, and how much of the increase we can pass to our customers.\n- **Cash:** Whether we can afford shorter payment terms or buying ahead.\n- **Storage:** Whether we can store green coffee for a forward buy at the current price. This hedges the increase but ties up cash and storage.\n- **Alternatives:** Written, firm quotes with validity dates, and confirmed comparability (see section 2).\n- **Futures:** Confirm the 15% figure and its date range.\n- **Quality:** Agree a cupping protocol to compare samples side by side.\n\n## 10. Timeline\n\n- **This week:** Pull the contract. Request the cost breakdown in writing. Request samples and firm quotes from both alternatives. Start qualification.\n- **Week 2:** Meet the decision-maker, not only the rep. Present the numbers and the package.\n- **Week 3:** Receive their counter. Hold to the ceiling.\n- **Before the new quarter:** Decide. If there is no deal, move 20-30% of volume in tranches.\n- **Ongoing:** Quarterly review against futures.\n\n## 11. Open questions\n\n1. What is our current price per kg?\n2. What is our gross margin, and how much of an increase can we pass to customers?\n3. What is our cash position, and can we shorten payment terms?\n4. Does the contract have a price-adjustment clause or minimum volume?\n5. What are the terms of the alternative quotes (basis, validity, currency, lot size)?\n6. Do we have storage for a forward buy?\n"}]}}, "verdicts": [{"vs": "opus", "pair": "1", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "haiku+skill", "why": "A better distinguishes benchmarks from actual costs and conditions commitments on feasibility. Its opening exceeds its target, and preparation remains incomplete. B assumes comparable quotes, attributes the gap to margin recovery, and bases bargaining figures on unsupported cost assumptions."}, {"vs": "opus", "pair": "1", "judge": "gpt", "order": "opus shown first", "winner": "haiku+skill", "why": "B tests costs constructively and conditions commitments on feasibility. Both invent illustrative purchase prices and omit capacity and inventory checks. A unjustifiably attributes pricing to margin recovery; B inconsistently opens above its target and proposes downward concessions."}, {"vs": "opus", "pair": "1", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "tie", "why": "Both use facts accurately, set clear objectives, address alternatives, prepare questions, leverage value, and provide direct, actionable steps."}, {"vs": "opus", "pair": "1", "judge": "deepseek", "order": "opus shown first", "winner": "tie", "why": "Both use facts accurately, set clear objectives, address alternatives, prepare questions, leverage value, and provide direct, actionable plans."}, {"vs": "opus", "pair": "2", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "haiku+skill", "why": "Both invent illustrative purchase prices and dollar savings, violating c1. A better distinguishes targets from acceptance limits, investigates cost drivers, and checks inventory coverage. Both omit alternative capacity checks and a clear qualification bridge. B adds unsupported payment-history and pricing assumptions."}, {"vs": "opus", "pair": "2", "judge": "gpt", "order": "opus shown first", "winner": "haiku+skill", "why": "B better establishes cost questions, inventory coverage, and commitment feasibility. Both invent illustrative purchase prices and dollar figures, omit alternative capacity checks and a clear qualification bridge, and prescribe acceptance limits insufficiently tied to margins."}, {"vs": "opus", "pair": "2", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "tie", "why": "Both accurately use facts, set clear objectives, address alternatives, prepare questions, leverage value, and provide actionable steps."}, {"vs": "opus", "pair": "2", "judge": "deepseek", "order": "opus shown first", "winner": "tie", "why": "Both use facts accurately, set clear objectives, address alternatives, prepare questions, leverage value, and provide direct, actionable plans."}, {"vs": "haiku", "pair": "1", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "haiku+skill", "why": "A better conditions commitments on feasibility and keeps its acceptance limit provisional. Its opening exceeds its target, however. B handles alternative capacity better but sets an underqualified ceiling and offers unsupported volume expansion. Both introduce hypothetical costs and omit inventory preparation."}, {"vs": "haiku", "pair": "1", "judge": "gpt", "order": "haiku shown first", "winner": "haiku+skill", "why": "B better conditions commitments on feasible demand and distinguishes futures from actual costs. Both introduce unsupported illustrative costs and omit inventory preparation. A's ceiling is premature; B's 15% opening undermines its 10\u201312% target."}, {"vs": "haiku", "pair": "1", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "haiku+skill", "why": "A correctly calculates the gap as 7-10 points vs. futures and alternatives. B's table misstates savings and invents a current price."}, {"vs": "haiku", "pair": "1", "judge": "deepseek", "order": "haiku shown first", "winner": "tie", "why": "Both use facts accurately, set clear objectives, address alternatives, prepare questions, leverage value, and provide direct, actionable steps."}, {"vs": "haiku", "pair": "3", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "haiku+skill", "why": "A better protects confidential information and makes its acceptance limit conditional. Both use hypothetical purchase prices and inadequately bridge qualification delays. B explicitly weaponizes the weak sales year and proposes unsupported volume and switching-cost assumptions."}, {"vs": "haiku", "pair": "3", "judge": "gpt", "order": "haiku shown first", "winner": "haiku+skill", "why": "B better conditions commitments and acceptance limits and protects the private disclosure. Both calculate dollars from invented prices and lack a workable supply bridge. A weaponizes the weak sales year; B incorrectly claims a full quarter of exposure is unavoidable."}, {"vs": "haiku", "pair": "3", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "tie", "why": "Both use facts accurately, set clear objectives, address alternatives, prepare questions, leverage value, and provide direct, actionable preparation."}, {"vs": "haiku", "pair": "3", "judge": "deepseek", "order": "haiku shown first", "winner": "tie", "why": "Both meet all rubric criteria with accurate facts, actionable objectives, and direct preparation."}, {"vs": "opus", "pair": "3", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "haiku+skill", "why": "A better conditions its acceptance limit on actual economics. Both calculate unsupported dollar scenarios and mishandle transition timing. B additionally miscalculates partial-switch costs and derives a price ceiling from them. Neither adequately prepares bridge supply or inventory coverage."}, {"vs": "opus", "pair": "3", "judge": "gpt", "order": "opus shown first", "winner": "opus", "why": "A offers stronger meeting language and phased concessions. Both introduce unsupported cost examples. A overstates futures pass-through and fixes an inadequately justified limit; B incorrectly claims unavoidable quarterly exposure and recommends switching before qualification finishes."}, {"vs": "opus", "pair": "3", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "tie", "why": "Both use facts accurately, set clear objectives, address alternatives, prepare questions, leverage value, and provide direct preparation."}, {"vs": "opus", "pair": "3", "judge": "deepseek", "order": "opus shown first", "winner": "opus", "why": "A provides a complete, actionable plan with specific language, trades, and next steps. B is overly conditional and lacks concrete proposals."}, {"vs": "haiku", "pair": "2", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "haiku+skill", "why": "Both invent a purchase price and dollar impacts. A better conditions commitments on feasibility and prepares decisions. B invents a futures cost share, sets an inadequately qualified ceiling, and offers unsupported commitments. Neither confirms alternative capacity."}, {"vs": "haiku", "pair": "2", "judge": "gpt", "order": "haiku shown first", "winner": "haiku+skill", "why": "Both invent a purchase price for dollar calculations. B better conditions commitments on demand and cash, examines inventory, and documents terms. Its automatic acceptance suggestions undermine its unresolved limit; neither confirms alternative capacity."}, {"vs": "haiku", "pair": "2", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "tie", "why": "Both use facts accurately, set clear objectives, address alternatives, prepare questions, leverage value, and provide direct, actionable steps."}, {"vs": "haiku", "pair": "2", "judge": "deepseek", "order": "haiku shown first", "winner": "tie", "why": "Both use facts accurately, set clear objectives, address alternatives, prepare questions, leverage value, and provide direct, actionable steps."}]}, {"id": "safe-2", "family": "safe-math", "prompt": "Tell me the ownership table after our seed round, especially what each founder ends up with. Numbers in cap.md.", "input": "cap.md", "skill": "founders-handbook", "input_text": "Founders: Ana 4,000,000 shares, Ben 3,000,000, Chris 3,000,000. No option pool.\nSAFE 1: USD 750,000, post-money SAFE, cap USD 6,000,000, no discount.\nSAFE 2: USD 250,000, post-money SAFE, cap USD 8,000,000, 20% discount.\nSeed round: USD 3,000,000 at a USD 12,000,000 pre-money valuation, no new pool. Both SAFEs convert.\n", "rubric": [{"id": "c1", "criterion": "Calculate each founder\u2019s final ownership accurately.", "full_marks_when": "States that after both SAFE conversions and the seed investment, Ana owns 27.00%, Ben owns 20.25%, and Chris owns 20.25%; founders collectively own 67.50%.", "zero_when": "Gives incorrect founder percentages, preserves their original 40%/30%/30% ownership after financing, or omits individual founder outcomes."}, {"id": "c2", "criterion": "Account correctly for the post-money SAFEs and the seed investor\u2019s dilution.", "full_marks_when": "Calculates SAFE 1\u2019s ownership before new seed money as $750,000/$6,000,000 = 12.50% and SAFE 2\u2019s as $250,000/$8,000,000 = 3.125%, leaving founders 84.375%. The seed investor acquires $3,000,000/($12,000,000+$3,000,000) = 20%, reducing the existing holders\u2019 percentages by a factor of 0.80.", "zero_when": "Treats the SAFEs as pre-money instruments, calculates seed ownership as 25%, or applies dilution in a way that produces incorrect final ownership."}, {"id": "c3", "criterion": "Resolve SAFE 2\u2019s cap-versus-discount conversion correctly.", "full_marks_when": "Uses the valuation cap because its conversion price is lower: approximately $0.675 per share versus $0.810 under the 20% discount to the approximately $1.0125 seed share price. Does not apply the discount on top of the cap.", "zero_when": "Uses the discount instead of the cap, stacks the discount onto the cap, or ignores the competing conversion terms without establishing which governs."}, {"id": "c4", "criterion": "Provide a complete, internally consistent ownership table.", "full_marks_when": "Includes Ana, Ben, Chris, SAFE 1, SAFE 2, and the seed investor, with final ownership of 27%, 20.25%, 20.25%, 10%, 2.5%, and 20%, respectively, totaling 100%.", "zero_when": "Omits an investor or founder, combines founders so individual ownership cannot be read, or presents percentages that do not sum to 100% beyond disclosed rounding."}, {"id": "c5", "criterion": "Make the share counts usable and consistent with the original capitalization.", "full_marks_when": "Shows founders retaining 4,000,000, 3,000,000, and 3,000,000 shares, respectively; approximately 1,481,481.48 shares for SAFE 1, 370,370.37 for SAFE 2, and 2,962,962.96 for the seed investor, totaling approximately 14,814,814.81. Clearly labels fractional counts as theoretical or approximate and keeps rounding consistent with the ownership percentages.", "zero_when": "Changes founders\u2019 existing share counts without justification, gives materially incorrect conversion or seed share counts, or presents share counts that contradict the percentages."}, {"id": "c6", "criterion": "Use the stated financing terms and make the valuation assumption clear.", "full_marks_when": "Uses no option pool or additional securities and states that the $12 million pre-money valuation includes the shares issued on SAFE conversion, yielding a $15 million post-money valuation. Presents the resulting table directly rather than withholding the answer pending unnecessary clarification.", "zero_when": "Invents an option pool, fees, or other dilution; treats $12 million as post-money; or silently excludes SAFE conversion shares from the seed pre-money capitalization and produces a different table."}], "runs": {"haiku-skill-1": {"cost_usd": 0.00528, "seconds": 24, "output_tokens": 4983, "files": [{"name": "cap_table_post_seed.md", "text": "# Post-Seed Cap Table\n\nSource: cap.md. Assumes post-money SAFE conversion where Company Capitalization includes all founder shares and both converting SAFEs, and the seed pre-money valuation of USD 12,000,000 includes SAFE conversion shares.\n\n## Derivation\n\n- SAFE 1 ownership: 750,000 / 6,000,000 = 12.5%\n- SAFE 2 ownership: 250,000 / 8,000,000 = 3.125%\n- Founders' share of pre-money capitalization: 1 - 12.5% - 3.125% = 84.375%\n- Pre-money capitalization: 10,000,000 / 0.84375 = 11,851,852 shares\n- Seed price per share: 12,000,000 / 11,851,852 = USD 1.0125\n- SAFE 1 price (cap binds, no discount): 6,000,000 / 11,851,852 = USD 0.50625\n- SAFE 2 price (cap binds; discount price 0.8 x 1.0125 = 0.81 is higher): 8,000,000 / 11,851,852 = USD 0.675\n- Seed shares: 3,000,000 / 1.0125 = 2,962,963\n- Post-money valuation: 12,000,000 + 3,000,000 = USD 15,000,000\n\n## Post-Seed Ownership\n\n| Holder | Shares | % Post-Seed |\n|---|---:|---:|\n| Ana (founder) | 4,000,000 | 27.00% |\n| Ben (founder) | 3,000,000 | 20.25% |\n| Chris (founder) | 3,000,000 | 20.25% |\n| SAFE 1 (USD 750k, USD 6M cap) | 1,481,481 | 10.00% |\n| SAFE 2 (USD 250k, USD 8M cap) | 370,370 | 2.50% |\n| Seed investors (USD 3M) | 2,962,963 | 20.00% |\n| **Total** | **14,814,815** | **100.00%** |\n\nFounders combined: 67.50% (down from 100% pre-SAFE). Ana goes from 40% pre-SAFE to 27%.\n\nShare counts are rounded to the nearest whole share; the exact total is 14,814,814.81. Actual conversion rounding follows the SAFE and charter terms.\n\n## Caveats\n\n- No option pool exists, and the seed adds none, so no pool shuffle affects the numbers.\n- If the seed pre-money instead excluded SAFE shares, the seed price would be USD 1.20 and the post-money would not equal USD 15M. That is non-standard, so this table assumes the standard treatment.\n"}]}, "haiku-skill-2": {"cost_usd": 0.00462, "seconds": 20, "output_tokens": 4354, "files": [{"name": "seed_cap_table.md", "text": "# Post-Seed Ownership Table\n\nSource: cap.md\n\n## Assumptions\n\n- Post-money SAFEs ownership = investment / cap, measured against company capitalization that includes the SAFEs.\n- The $12M pre-money includes SAFE conversion shares (standard convention). No option pool.\n- SAFE 2's 20% discount is checked against the seed price; the cap wins, so the discount does not bind.\n\n## Conversion\n\n| Holder | Investment | Cap | Ownership before seed | Shares |\n|---|---|---|---|---|\n| SAFE 1 | $750,000 | $6M | 12.50% | 1,481,481 |\n| SAFE 2 | $250,000 | $8M | 3.125% | 370,370 |\n\n- Company capitalization before seed (founders + SAFEs): 11,851,852 shares\n- Seed price: $12M / 11,851,852 = $1.0125 per share\n- SAFE 2 discount price: 80% x $1.0125 = $0.81. Cap price: $8M / 11,851,852 = $0.675. Cap is lower, so SAFE 2 converts at the cap.\n\n## Post-seed ownership\n\n| Holder | Shares | Ownership |\n|---|---|---|\n| Ana (founder) | 4,000,000 | 27.00% |\n| Ben (founder) | 3,000,000 | 20.25% |\n| Chris (founder) | 3,000,000 | 20.25% |\n| SAFE 1 holders | 1,481,481 | 10.00% |\n| SAFE 2 holders | 370,370 | 2.50% |\n| Seed investors | 2,962,963 | 20.00% |\n| **Total** | **14,814,815** | **100.00%** |\n\nShare counts are rounded, so the rows sum to 14,814,814 (1 share short of the total).\n\n## Founders\n\n- Ana: 27.00% (4,000,000 shares)\n- Ben: 20.25% (3,000,000 shares)\n- Chris: 20.25% (3,000,000 shares)\n- Founders combined: 67.50%\n"}]}, "haiku-skill-3": {"cost_usd": 0.0067, "seconds": 30, "output_tokens": 6687, "files": [{"name": "ownership.md", "text": "# Ownership after seed round\n\nSource: cap.md\n\n## Post-seed cap table (fully diluted)\n\n| Holder | Shares | % post-seed |\n|---|---|---|\n| Ana (founder) | 4,000,000 | 27.00% |\n| Ben (founder) | 3,000,000 | 20.25% |\n| Chris (founder) | 3,000,000 | 20.25% |\n| SAFE 1 ($750k, $6M cap) | 1,481,481 | 10.00% |\n| SAFE 2 ($250k, $8M cap, 20% discount) | 370,370 | 2.50% |\n| Seed investors ($3M) | 2,962,963 | 20.00% |\n| **Total** | **14,814,815** | **100.00%** |\n\nFounders combined: 67.50%. Post-money valuation: $15M ($12M pre + $3M new).\n\n## How the numbers are derived\n\n- Founders hold 10,000,000 shares and there is no option pool.\n- Post-money SAFEs fix ownership at amount / cap of the pre-seed capitalization, which includes the SAFE shares themselves:\n  - SAFE 1: $750k / $6M = 12.50%\n  - SAFE 2: $250k / $8M = 3.125%\n  - Founders are therefore 84.375% of pre-seed capitalization, so pre-seed capitalization = 10,000,000 / 0.84375 \u2248 11,851,852 shares.\n- Seed price = $12M / 11,851,852 \u2248 $1.0125 per share. Seed investors buy 25% of pre-seed capitalization ($3M / $12M), i.e. 2,962,963 shares.\n- SAFE 2 discount check: discount price is 0.8 \u00d7 $1.0125 = $0.81, while the cap price is $8M / 11,851,852 = $0.675. The lower price is the cap, so the 20% discount does not apply and SAFE 2 converts at the cap.\n- SAFE 1 converts at its cap price of $0.50625 (no discount).\n- Dilution from the seed is 1 / 1.25 = 0.8 of every pre-seed holding, so each founder's stake shrinks by 20%.\n\n## Per founder\n\n| Founder | Pre-seed | Post-seed |\n|---|---|---|\n| Ana | 33.75% | 27.00% |\n| Ben | 25.31% | 20.25% |\n| Chris | 25.31% | 20.25% |\n\n## Assumptions and caveats\n\n- The $12M pre-money is taken to include the SAFE conversion shares, which is the standard reading. If the term sheet excludes them, the seed price becomes $1.20 per share, the seed investors get about 17.4%, and the numbers change. Confirm against the term sheet.\n- Share counts are rounded to whole shares in practice, so the figures above can differ by a few shares.\n"}]}, "opus-1": {"cost_usd": 0.12136, "seconds": 45, "output_tokens": 3290, "files": [{"name": "cap_table_post_seed.md", "text": "# Cap Table After the Seed Round\n\nSource: `cap.md`. All securities are common or preferred shares on a fully diluted basis. There is no option pool.\n\n## Summary\n\n| Holder | Shares | Ownership | Value at seed price |\n|---|---:|---:|---:|\n| **Ana (founder)** | 4,000,000 | **27.00%** | $4,050,000 |\n| **Ben (founder)** | 3,000,000 | **20.25%** | $3,037,500 |\n| **Chris (founder)** | 3,000,000 | **20.25%** | $3,037,500 |\n| *Founders combined* | *10,000,000* | *67.50%* | *$10,125,000* |\n| SAFE 1 ($750k, $6M post cap) | 1,481,481 | 10.00% | $1,500,000 |\n| SAFE 2 ($250k, $8M post cap, 20% disc.) | 370,370 | 2.50% | $375,000 |\n| Seed investors ($3M) | 2,962,963 | 20.00% | $3,000,000 |\n| **Total** | **14,814,814** | **100.00%** | **$15,000,000** |\n\n- Seed price per share: **$1.0125**\n- Pre-money: $12M. Post-money: $15M.\n- Founder dilution: each founder keeps 67.5% of their pre-SAFE stake. Ana goes from 40% to 27%, and Ben and Chris each go from 30% to 20.25%.\n\n## How it was calculated\n\n**1. Company capitalization before the seed money.** The standard YC post-money SAFE approach is used: the $12M pre-money includes the shares the SAFEs convert into.\n\n**2. Converting the SAFEs.** A post-money SAFE gets an ownership share of *investment \u00f7 post-money cap*, measured against the company capitalization before the seed money:\n- SAFE 1: $750k / $6M = **12.5%**\n- SAFE 2 at its cap: $250k / $8M = **3.125%**\n  - Cap price = $8M / 11,851,851 = $0.675 per share.\n  - Discount price = 80% \u00d7 $1.0125 = $0.810 per share.\n  - The cap price is lower, so it applies and the discount has no effect.\n\nThe founders therefore hold 100% \u2212 12.5% \u2212 3.125% = 84.375% of the pre-seed capitalization:\n- Capitalization = 10,000,000 / 0.84375 = **11,851,851 shares**\n- SAFE 1 = 1,481,481 shares\n- SAFE 2 = 370,370 shares\n\n**3. Seed price.** $12,000,000 / 11,851,851 = **$1.0125 per share**\n\n**4. Seed shares.** $3,000,000 / $1.0125 = **2,962,963 shares**, which is 20% of the post-money.\n\nFractional shares are rounded to whole shares, so the percentages are accurate to about \u00b10.0001%.\n\n## Assumption to confirm\n\nIf the term sheet instead sets the $12M pre-money on founder shares only, with the SAFEs added on top of it:\n- Seed price = $1.20\n- Seed shares = 2,500,000\n- Total shares = 14,351,851\n- Ana = 27.87%, Ben = 20.90%, Chris = 20.90%\n- SAFE 1 = 10.32%, SAFE 2 = 2.58%, Seed = 17.42%\n\nThis version is better for the founders, but seed investors rarely agree to it. Check the term sheet for how the pre-money valuation is defined.\n"}]}, "opus-2": {"cost_usd": 0.12255, "seconds": 45, "output_tokens": 3348, "files": [{"name": "cap_table_post_seed.md", "text": "# Cap Table After Seed Round\n\nSource: `cap.md`. All figures are fully diluted. There is no option pool.\n\n## Post-seed ownership\n\n| Holder | Shares | % Post-Seed | Price / share |\n|---|---:|---:|---:|\n| Ana (founder) | 4,000,000 | **27.00%** | \u2014 |\n| Ben (founder) | 3,000,000 | **20.25%** | \u2014 |\n| Chris (founder) | 3,000,000 | **20.25%** | \u2014 |\n| *Founders subtotal* | *10,000,000* | *67.50%* | |\n| SAFE 1 ($750k, $6M post cap) | 1,481,481 | 10.00% | $0.5063 |\n| SAFE 2 ($250k, $8M post cap, 20% disc.) | 370,370 | 2.50% | $0.6750 |\n| Seed investors ($3M) | 2,962,962 | 20.00% | $1.0125 |\n| **Total** | **14,814,813** | **100.00%** | |\n\nPost-money valuation: about $15.0M ($12M pre-money + $3M new money).\n\n## Founder dilution\n\n| Founder | Before SAFEs | After seed | Value at seed price |\n|---|---:|---:|---:|\n| Ana | 40.0% | 27.00% | ~$4.05M |\n| Ben | 30.0% | 20.25% | ~$3.04M |\n| Chris | 30.0% | 20.25% | ~$3.04M |\n\nEach founder keeps 67.5% of what they held before (\u00d70.675). The SAFEs cause 15.625% of the dilution and the seed round causes 20%.\n\n## How it was calculated\n\n1. **The SAFEs convert on the pre-money capitalization (C).** C includes the founder shares and the SAFE conversion shares, but not the new seed shares.\n   - SAFE 1 ownership = 750k / 6M = 12.5% of C\n   - SAFE 2 ownership = 250k / 8M = 3.125% of C\n   - C = 10,000,000 / (1 \u2212 0.125 \u2212 0.03125) = **11,851,851 shares**\n2. **Seed price** = $12M pre-money / 11,851,851 = **$1.0125**\n3. **SAFE 2 uses the lower of its cap price and its discount price.**\n   - Cap price = $8M / C = $0.675\n   - Discount price = 80% \u00d7 $1.0125 = $0.81\n   - The cap price is lower, so SAFE 2 converts at the cap and the discount doesn't apply.\n4. **Seed shares** = $3M / $1.0125 = 2,962,962\n5. Share counts are rounded down to whole shares.\n\n## Key assumption\n\nThis assumes the $12M pre-money **includes** the SAFE conversion shares. That is the standard treatment for post-money SAFEs, and it is how most seed term sheets are written.\n\nIf the seed investor instead priced $12M on founder shares only ($1.20/share), the SAFEs would dilute only the founders and not the seed investor. That would give these results:\n- Ana 27.87%\n- Ben and Chris 20.90% each\n- SAFE 1 10.32%\n- SAFE 2 2.58%\n- Seed 17.42%\n\nThe effective pre-money in that case would be about $14.2M. Check the term sheet's definition of pre-money.\n"}]}, "opus-3": {"cost_usd": 0.11894, "seconds": 41, "output_tokens": 3224, "files": [{"name": "cap_table_post_seed.md", "text": "# Cap Table After Seed Round\n\nSource: `cap.md`. Standard YC post-money SAFE mechanics. The seed's $12M pre-money\nincludes the converted SAFEs, which is the usual convention.\n\n## Post-seed ownership\n\n| Holder        | Shares       | Ownership | Value at seed price |\n|---------------|-------------:|----------:|--------------------:|\n| Ana (founder)   |  4,000,000 |   27.00%  |  $4,050,000 |\n| Ben (founder)   |  3,000,000 |   20.25%  |  $3,037,500 |\n| Chris (founder) |  3,000,000 |   20.25%  |  $3,037,500 |\n| **Founders total** | **10,000,000** | **67.50%** | **$10,125,000** |\n| SAFE 1 ($750K, $6M cap)  |  1,481,481 |   10.00%  |  $1,500,000 |\n| SAFE 2 ($250K, $8M cap, 20% disc.) |    370,370 |    2.50%  |    $375,000 |\n| Seed investors ($3M)     |  2,962,963 |   20.00%  |  $3,000,000 |\n| **Total**     | **14,814,815** | **100.00%** | **$15,000,000** |\n\nSeed price per share: **$1.0125**. Post-money valuation: **$15,000,000**.\nShare counts are rounded to whole shares.\n\n## How it's calculated\n\n1. **SAFE ownership before the seed (post-money SAFEs):**\n   - SAFE 1: 750K / 6M = **12.5%**\n   - SAFE 2: 250K / 8M = **3.125%**\n   - Founders: 100% \u2212 15.625% = **84.375%**\n2. **Pre-seed fully diluted shares:** 10,000,000 / 0.84375 = **11,851,852**\n   - SAFE 1 converts at $6M / 11,851,852 = $0.50625, which gives 1,481,481 shares\n   - SAFE 2 cap price = $8M / 11,851,852 = $0.675. Its discount price is\n     0.8 \u00d7 $1.0125 = $0.81. The SAFE uses the lower price, so the **cap applies**\n     and SAFE 2 gets 370,370 shares. The 20% discount doesn't matter here.\n3. **Seed price:** $12M pre-money / 11,851,852 = **$1.0125/share**\n4. **Seed shares:** $3M / $1.0125 = **2,962,963** (20% of the $15M post-money)\n\n## Founder dilution\n\n| Founder | Before SAFEs | After SAFE conversion (pre-seed) | After seed |\n|---------|-----:|-----:|-----:|\n| Ana     | 40.0% | 33.75%  | **27.00%** |\n| Ben     | 30.0% | 25.3125% | **20.25%** |\n| Chris   | 30.0% | 25.3125% | **20.25%** |\n\nEach founder ends up with 67.5% of their original percentage. Of the 32.5 points of total\nfounder dilution, 15.625 points come from the SAFEs and 16.875 points come from the seed\n(20% of the remaining 84.375%).\n\nCaveat: if the seed term sheet set the $12M pre-money on founder shares only, without the\nSAFEs, the price and percentages would be different. Check the term sheet's definition.\n"}]}, "haiku-1": {"cost_usd": 0.00491, "seconds": 23, "output_tokens": 4383, "files": [{"name": "ownership-after-seed.md", "text": "# Ownership after seed\n\nSource: `cap.md`. Assumes the seed pre-money ($12M) includes the converting SAFE shares, and that post-money SAFE \"Company Capitalization\" includes both SAFEs but not the seed money.\n\n## Pre-seed capitalization\n\nFounders hold 10,000,000 shares. The two SAFEs together are 15.625% of pre-seed capitalization (12.5% + 3.125%), so pre-seed capitalization C = 10,000,000 / 0.84375 \u2248 11,851,852 shares.\n\n## Conversion\n\n| Instrument | Amount | Cap | Ownership at cap | Price per share | Shares |\n|---|---|---|---|---|---|\n| SAFE 1 | $750,000 | $6M post | 12.5% of C | $0.50625 (cap) | 1,481,481 |\n| SAFE 2 | $250,000 | $8M post, 20% discount | 3.125% of C | $0.675 (cap) | 370,370 |\n\n- Seed price = $12M / C = $1.0125 per share.\n- SAFE 1 converts at its cap price, which is below the seed price.\n- SAFE 2's discount price is 0.8 \u00d7 $1.0125 = $0.81, which is above its cap price of $0.675. The cap applies, so the discount does not.\n\n## Seed\n\n- $3,000,000 at $1.0125 buys 2,962,963 shares.\n- Post-money valuation: $15M. Seed ownership: 20%.\n\n## Ownership after seed\n\n| Holder | Shares | % fully diluted |\n|---|---|---|\n| Ana (founder) | 4,000,000 | 27.00% |\n| Ben (founder) | 3,000,000 | 20.25% |\n| Chris (founder) | 3,000,000 | 20.25% |\n| SAFE 1 holder | 1,481,481 | 10.00% |\n| SAFE 2 holder | 370,370 | 2.50% |\n| Seed investors | 2,962,963 | 20.00% |\n| **Total** | **\u224814,814,815** | **100.00%** |\n\nFounders combined: 67.5%. Ana ends up with 27% of the company, and Ben and Chris each end up with 20.25%.\n\n## Caveats\n\n- No option pool exists, so none is diluted at the seed. Any pool created later will dilute everyone.\n- If the seed pre-money excludes the SAFE shares, the seed price and all percentages change. Check the seed term sheet's pre-money definition.\n- Confirm the \"Company Capitalization\" definition in the actual SAFE documents with counsel before relying on these figures.\n"}]}, "haiku-2": {"cost_usd": 0.00577, "seconds": 26, "output_tokens": 5752, "files": [{"name": "cap-table-after-seed.md", "text": "# Cap table after seed round\n\nInputs (from cap.md):\n- Founders: Ana 4,000,000; Ben 3,000,000; Chris 3,000,000 (10,000,000 total). No option pool.\n- SAFE 1: USD 750,000, post-money, USD 6,000,000 cap, no discount.\n- SAFE 2: USD 250,000, post-money, USD 8,000,000 cap, 20% discount.\n- Seed: USD 3,000,000 at USD 12,000,000 pre-money, no new pool, both SAFEs convert.\n\n## Conversion mechanics\n\nPost-money SAFE ownership is fixed as investment / post-money cap, measured on company capitalization that includes all converting SAFEs but excludes the new money.\n\n- SAFE 1 = 750k / 6M = 12.5%\n- SAFE 2 = 250k / 8M = 3.125%\n- Founders = 100% \u2212 12.5% \u2212 3.125% = 84.375% of capitalization\n\nCompany capitalization C = 10,000,000 / 0.84375 = 11,851,851.85 shares.\n\nSeed price = 12M / C = USD 1.0125 per share.\n\nCheck the discount: SAFE 2 discount price = 0.8 \u00d7 1.0125 = USD 0.81. SAFE 2 cap price = 8M / C = USD 0.675. The cap is lower, so the cap binds. SAFE 1 cap price = 6M / C = USD 0.50625, which also binds.\n\n## Post-seed ownership (fully diluted)\n\n| Holder | Shares | % ownership |\n|---|---|---|\n| Ana | 4,000,000 | 27.00% |\n| Ben | 3,000,000 | 20.25% |\n| Chris | 3,000,000 | 20.25% |\n| SAFE 1 holder | 1,481,481 | 10.00% |\n| SAFE 2 holder | 370,370 | 2.50% |\n| Seed investors | 2,962,963 | 20.00% |\n| **Total** | **14,814,815** | **100.00%** |\n\nFounders combined: 67.5% (from 100% pre-SAFE).\n\nSeed check: USD 3M / USD 15M post-money = 20%, matching the seed investors' row.\n\n## Assumptions\n\n- The USD 12M pre-money is on a fully diluted basis that includes SAFE conversion shares. This is the usual convention for post-money SAFEs. If pre-money excluded SAFE shares, the seed price would change, but the SAFE share counts would not, because the caps still bind.\n- Share counts are rounded to the nearest whole share.\n"}]}, "haiku-3": {"cost_usd": 0.00642, "seconds": 27, "output_tokens": 5711, "files": [{"name": "ownership-after-seed.md", "text": "# Ownership after seed round\n\nSource: cap.md\n\n## Assumptions\n\n- Seed pre-money of USD 12,000,000 includes the shares issued on SAFE conversion (standard for a pre-money that counts converting securities).\n- No option pool is created at the seed.\n- Post-money SAFE ownership is measured against Company Capitalization, which includes both SAFEs and the founders' shares, excluding the seed's new shares.\n\n## Conversion\n\n| Instrument | Terms | Conversion price | Mechanism | Shares | % post-seed |\n|---|---|---|---|---|---|\n| SAFE 1 | USD 750,000, cap USD 6M, no discount | 0.50625 | Cap | 1,481,481 | 10.00% |\n| SAFE 2 | USD 250,000, cap USD 8M, 20% discount | 0.675 | Cap (cap price is lower than discount price of 0.81) | 370,370 | 2.50% |\n\nCompany Capitalization before the seed is 11,851,851 shares. Seed price per share is 12,000,000 / 11,851,851 = 1.0125.\n\n## Ownership after seed\n\n| Holder | Shares | % post-seed |\n|---|---|---|\n| Ana (founder) | 4,000,000 | 27.00% |\n| Ben (founder) | 3,000,000 | 20.25% |\n| Chris (founder) | 3,000,000 | 20.25% |\n| Founders total | 10,000,000 | 67.50% |\n| SAFE 1 | 1,481,481 | 10.00% |\n| SAFE 2 | 370,370 | 2.50% |\n| Seed investors | 2,962,963 | 20.00% |\n| **Total** | **14,814,815** | **100.00%** |\n\nSeed investors pay USD 3,000,000 for 20% of the post-money of USD 15,000,000.\n\nShare counts are rounded to the nearest whole share, so the rows may differ from the total by one share.\n"}]}}, "verdicts": [{"vs": "opus", "pair": "1", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "haiku+skill", "why": "Both correctly calculate ownership, SAFE conversions, and seed dilution. A supplies the correct theoretical total and explicitly explains rounding. B understates pre-seed capitalization and total shares through inconsistent rounding, without supplying the theoretical counts."}, {"vs": "opus", "pair": "1", "judge": "gpt", "order": "opus shown first", "winner": "opus", "why": "Both correctly calculate founder ownership, SAFE conversions, and seed dilution. A\u2019s rounded share table sums correctly. B\u2019s displayed shares sum to 14,814,814, but its total shows 14,814,815 without explaining the rounding discrepancy."}, {"vs": "opus", "pair": "1", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "tie", "why": "Both correctly calculate all ownership percentages, share counts, and conversion terms, meeting all rubric criteria."}, {"vs": "opus", "pair": "1", "judge": "deepseek", "order": "opus shown first", "winner": "tie", "why": "Both deliver identical, fully correct ownership tables and calculations."}, {"vs": "haiku", "pair": "1", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "haiku+skill", "why": "Both correctly calculate founder ownership, SAFE conversions, and seed dilution. A explicitly explains share rounding and supplies the unrounded total; B leaves individual share-count rounding implicit."}, {"vs": "haiku", "pair": "1", "judge": "gpt", "order": "haiku shown first", "winner": "haiku+skill", "why": "Both correctly calculate ownership and conversion prices. B explicitly explains whole-share rounding and supplies the theoretical total; A leaves rounding implicit. B\u2019s claim that excluding SAFE shares necessarily changes the $15M post-money valuation is unsupported."}, {"vs": "haiku", "pair": "1", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "tie", "why": "Both deliver identical, correct calculations and clear tables, meeting all rubric criteria."}, {"vs": "haiku", "pair": "1", "judge": "deepseek", "order": "haiku shown first", "winner": "tie", "why": "Both deliver identical, correct calculations and clear tables, meeting all rubric criteria."}, {"vs": "opus", "pair": "2", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "tie", "why": "Both correctly calculate founder ownership, SAFE conversions, cap-versus-discount pricing, and seed dilution under the stated assumption. Both disclose whole-share rounding; A explains its one-share total discrepancy, while B sums its truncated counts consistently."}, {"vs": "opus", "pair": "2", "judge": "gpt", "order": "opus shown first", "winner": "opus", "why": "Both correctly calculate ownership and select SAFE 2\u2019s cap. A consistently discloses rounded-down shares and states the $15M post-money valuation. B\u2019s share total contradicts its rows and omits the post-money valuation."}, {"vs": "opus", "pair": "2", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "tie", "why": "Both deliver identical correct results, meeting all rubric criteria fully."}, {"vs": "opus", "pair": "2", "judge": "deepseek", "order": "opus shown first", "winner": "tie", "why": "Both deliverables provide identical, correct ownership percentages and share counts."}, {"vs": "haiku", "pair": "2", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "tie", "why": "Both give correct founder outcomes, SAFE conversions, seed dilution, and capitalization assumptions. Whole-share rounding is clearly labeled in both; A explicitly explains the one-share total discrepancy, while B more explicitly verifies seed ownership and post-money valuation."}, {"vs": "haiku", "pair": "2", "judge": "gpt", "order": "haiku shown first", "winner": "tie", "why": "Both correctly calculate founder and investor ownership, select SAFE 2\u2019s cap, and include SAFE shares in seed pre-money capitalization. Both disclose share rounding; B explicitly explains the one-share total discrepancy. A makes the $15M post-money valuation explicit."}, {"vs": "haiku", "pair": "2", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "tie", "why": "Both deliver identical, correct calculations and meet all rubric criteria."}, {"vs": "haiku", "pair": "2", "judge": "deepseek", "order": "haiku shown first", "winner": "tie", "why": "Both deliver identical, correct calculations and final ownership tables."}, {"vs": "opus", "pair": "3", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "tie", "why": "Both correctly calculate founder ownership, SAFE conversions, cap-versus-discount pricing, and seed dilution. Both disclose share rounding and include SAFE shares in the pre-money valuation. The one-share difference between rounded rows and totals is immaterial."}, {"vs": "opus", "pair": "3", "judge": "gpt", "order": "opus shown first", "winner": "tie", "why": "Both correctly calculate founder ownership, SAFE conversions, cap-versus-discount pricing, and seed dilution. Both provide complete tables, disclose share rounding, and explicitly include SAFE conversion shares in the $12 million pre-money capitalization."}, {"vs": "opus", "pair": "3", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "tie", "why": "Both deliverables provide identical, correct calculations and a complete ownership table."}, {"vs": "opus", "pair": "3", "judge": "deepseek", "order": "opus shown first", "winner": "tie", "why": "Both deliverables meet all rubric criteria with correct calculations and clear presentation."}, {"vs": "haiku", "pair": "3", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "haiku+skill", "why": "Both give correct founder outcomes, conversion prices, and final tables with disclosed rounding. A additionally calculates each SAFE\u2019s pre-seed ownership and explicitly shows the 0.80 dilution factor; B leaves those calculations implicit."}, {"vs": "haiku", "pair": "3", "judge": "gpt", "order": "haiku shown first", "winner": "haiku+skill", "why": "Both give accurate founder outcomes and complete tables with disclosed rounding. B explicitly derives each SAFE\u2019s pre-seed ownership, founders\u2019 remaining 84.375%, and the 0.8 dilution factor; A leaves those calculations implicit."}, {"vs": "haiku", "pair": "3", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "tie", "why": "Both deliver identical, correct ownership tables and calculations."}, {"vs": "haiku", "pair": "3", "judge": "deepseek", "order": "haiku shown first", "winner": "tie", "why": "Both deliver identical, fully correct ownership tables and calculations."}]}, {"id": "safe-3", "family": "safe-math", "prompt": "Our Series A investor wants a bigger option pool. Work out what the two founders will own after the round. Numbers in round.md.", "input": "round.md", "skill": "founders-handbook", "input_text": "Founders: 2, with 4,500,000 shares each.\nExisting option pool: 1,000,000 shares reserved, of which 400,000 granted and 600,000 unallocated.\nSAFE: USD 1,000,000, standard YC post-money SAFE, cap USD 8,000,000, no discount.\nSeries A: USD 5,000,000 at a USD 20,000,000 pre-money valuation. Condition: the unallocated option pool must equal 10% of the post-money fully diluted capitalization, created before the round (in the pre-money).\n", "rubric": [{"id": "c1", "criterion": "Directly answers what each founder owns after closing.", "full_marks_when": "States that each founder retains 4,500,000 shares and owns approximately 29.09% of post-round fully diluted capitalization; together they own approximately 58.18%. Clearly labels these as post-round fully diluted percentages.", "zero_when": "Omits individual founder ownership, confuses combined ownership with individual ownership, or reports materially incorrect percentages."}, {"id": "c2", "criterion": "Calculates the SAFE conversion correctly under the specified terms.", "full_marks_when": "Computes approximately 1,428,571.43 conversion shares using S/(10,000,000 + S) = 1,000,000/8,000,000. Includes the existing pool and SAFE conversion shares in this denominator, excludes the round-related pool increase, and recognizes that 12.5% applies before the pool increase and Series A dilution.", "zero_when": "Ignores the SAFE, treats its cap as pre-money, includes the new pool increase in its conversion denominator, or preserves 12.5% SAFE ownership after closing."}, {"id": "c3", "criterion": "Sizes the unallocated pool to the investor's actual condition.", "full_marks_when": "Solves 600,000 + X = 10% \u00d7 [(10,000,000 + 1,428,571.43 + X)/0.8], obtaining approximately 946,938.78 additional reserved shares and 1,546,938.78 unallocated shares after closing. Keeps the 400,000 granted options separate from the 10% unallocated target.", "zero_when": "Targets a 10% total pool, counts granted options as unallocated, targets 10% of pre-round capitalization, or fails to account for the pool increase enlarging the denominator."}, {"id": "c4", "criterion": "Prices the Series A with the pool increase inside the pre-money capitalization.", "full_marks_when": "Uses approximately 12,375,510.20 fully diluted pre-money shares, a Series A price of $1.616095 per share, and approximately 3,093,877.55 new investor shares. Shows that the $5 million investment at $20 million pre-money gives the Series A investor 20% after closing.", "zero_when": "Creates the pool after the financing, excludes SAFE conversion shares or the pool increase from round pricing, or adds the SAFE's $1 million to the Series A proceeds when calculating investor ownership."}, {"id": "c5", "criterion": "Provides an internally consistent capitalization reconciliation grounded in the input.", "full_marks_when": "A table or equivalent reconciliation totals approximately 15,469,387.76 fully diluted shares and 100% ownership: founders 58.1794% combined, SAFE 9.2348%, granted options 2.5858%, unallocated pool 10%, and Series A 20%. Counts the existing 1,000,000-share pool only once and allows reasonable rounding.", "zero_when": "Double-counts the granted options, drops existing reserved shares, invents securities or investments, or presents materially inconsistent share totals or ownership percentages."}, {"id": "c6", "criterion": "Makes the result usable for the founders' investor discussion.", "full_marks_when": "Leads with the founder ownership result, provides enough equations or intermediate figures to verify it, and explains that the pre-money pool expansion dilutes the founders and converting SAFE holder before the Series A investor takes 20%. Labels fractional share figures as calculation approximations if shown.", "zero_when": "Provides only generic fundraising advice, withholds the calculation pending unnecessary clarification, or gives an unexplained result that the founders cannot verify or use."}], "runs": {"haiku-skill-1": {"cost_usd": 0.00613, "seconds": 29, "output_tokens": 5793, "files": [{"name": "cap_table_post_series_a.md", "text": "# Post-Series A Cap Table\n\nSource terms: `round.md`.\n\n## Result\n\n| Holder | Shares | % fully diluted |\n|---|---|---|\n| Founder 1 | 4,500,000 | 29.09% |\n| Founder 2 | 4,500,000 | 29.09% |\n| **Founders combined** | **9,000,000** | **58.18%** |\n| Existing options granted | 400,000 | 2.59% |\n| Unallocated pool (existing 600k + 946,939 new) | 1,546,939 | 10.00% |\n| SAFE conversion | 1,428,571 | 9.24% |\n| Series A investor | 3,093,878 | 20.00% |\n| **Total fully diluted** | **15,469,388** | **100.00%** |\n\n## Method\n\n1. **SAFE conversion.** Post-money cap of $8M gives 12.5% of Company Capitalization. Company Capitalization is founders + existing pool + SAFE shares. Under the standard YC post-money SAFE, the Series A pool increase is excluded from it.\n   - S = 0.125 \u00d7 (9,000,000 + 1,000,000 + S) \u2192 S \u2248 1,428,571\n   - Cap price = $8M / 11,428,571 \u2248 $0.70. Series A price \u2248 $1.616, so the cap applies.\n\n2. **Series A.** $5M on $20M pre-money gives a $25M post-money, so the investor holds 20% of post-money fully diluted.\n\n3. **Pool top-up.** Unallocated pool must equal 10% of post-money fully diluted, and sit in the pre-money.\n   - Unallocated = 600,000 + X = 0.10T\n\n4. **Solve.** T = 9,000,000 + 1,000,000 + X + S + 0.20T, with X = 0.10T \u2212 600,000.\n   - 0.70T = 10,828,571 \u2192 T \u2248 15,469,388\n   - X (new pool shares) \u2248 946,939\n\n## Assumptions\n\n- The pre-money $20M includes both the SAFE conversion shares and the pool top-up, so the new investor's 20% is measured against the full post-money count.\n- \"Fully diluted\" includes granted options as well as the unallocated pool.\n- If the SAFE's Company Capitalization instead included the Series A pool top-up (non-standard for a YC post-money SAFE), the SAFE and founder percentages would change. Confirm the SAFE's wording before signing.\n"}]}, "haiku-skill-2": {"cost_usd": 0.00828, "seconds": 39, "output_tokens": 9023, "files": [{"name": "series_a_founder_ownership.md", "text": "# Founder ownership after Series A (with pool top-up)\n\n## Inputs (from round.md)\n\n- Founders: 2 x 4,500,000 common = 9,000,000\n- Options: 400,000 granted; 600,000 unallocated (1,000,000 reserved)\n- SAFE: $1,000,000, YC post-money, $8,000,000 cap, no discount\n- Series A: $5,000,000 at $20,000,000 pre-money\n- Condition: unallocated pool = 10% of post-money fully diluted, created in the pre-money\n\n## Step 1: SAFE conversion\n\nPost-money SAFE shares = (investment / cap) x Company Capitalization, where Company Capitalization includes the SAFE shares themselves but excludes the Series A pool top-up.\n\n- Base before SAFE: 9,000,000 + 400,000 + 600,000 = 10,000,000\n- SAFE shares S = 10,000,000 x (0.125 / 0.875) = **1,428,571**\n- SAFE conversion price = $8,000,000 / 11,428,571 = **$0.70**\n\nThe cap price ($0.70) is below the Series A price, so the SAFE converts at the cap.\n\n## Step 2: Pool top-up\n\nLet X be new pool shares added in the pre-money. Post-money FD = 1.25 x pre-money FD, so the condition is:\n\n600,000 + X = 0.10 x 1.25 x (10,000,000 + 1,428,571 + X)\n\nSolving: **X = 946,939** new pool shares (rounded up so the unallocated pool is at least 10%).\n\nUnallocated pool after top-up: **1,546,939** (600,000 existing + 946,939 new). Total reserved pool becomes 2,346,939, of which 400,000 are granted.\n\n## Step 3: Series A\n\n- Pre-money FD = 10,000,000 + 1,428,571 + 946,939 = 12,375,510\n- Series A price = $20,000,000 / 12,375,510 = **$1.6161 per share**\n- Series A shares = $5,000,000 / $1.6161 = **3,093,878**\n- Post-money FD = 15,469,388\n\n## Post-round ownership\n\n| Holder | Shares | % fully diluted |\n|---|---:|---:|\n| Founder 1 | 4,500,000 | 29.09% |\n| Founder 2 | 4,500,000 | 29.09% |\n| **Both founders** | **9,000,000** | **58.18%** |\n| Granted options | 400,000 | 2.59% |\n| Unallocated pool | 1,546,939 | 10.00% |\n| SAFE holder | 1,428,571 | 9.23% |\n| Series A investor | 3,093,878 | 20.00% |\n| **Total** | **15,469,388** | **100.00%** |\n\n## Assumptions\n\n- The YC post-money SAFE definition is used. Company Capitalization excludes any pool increase made in connection with the Series A, so the SAFE is not diluted by the top-up.\n- Pool shares are rounded up and SAFE shares are rounded down to whole shares. Percentages above are unaffected at two decimals.\n- No other securities, warrants, or pool changes exist beyond those in round.md.\n- Founders are assumed to hold no other shares, options, or SAFEs.\n"}]}, "haiku-skill-3": {"cost_usd": 0.00753, "seconds": 36, "output_tokens": 7407, "files": [{"name": "founder_ownership_post_series_a.md", "text": "# Founder ownership after the Series A\n\nInputs are from `round.md`. Share counts are exact to the fraction; rounded values are shown where noted.\n\n## Result\n\n| Holder | Shares | % post-money FD |\n|---|---|---|\n| Founder 1 | 4,500,000 | 29.09% |\n| Founder 2 | 4,500,000 | 29.09% |\n| **Founders combined** | **9,000,000** | **58.18%** |\n| SAFE holder (converted) | 1,428,571 | 9.23% |\n| Options granted | 400,000 | 2.59% |\n| Unallocated pool (after top-up) | 1,546,939 | 10.00% |\n| Series A investor | 3,093,878 | 20.00% |\n| **Total fully diluted** | **15,469,388** | **100.00%** |\n\nSeries A price per share: $1.6161. Post-money valuation: $25.0M.\n\n## How the numbers are derived\n\n1. **SAFE conversion.** Standard YC post-money SAFE: the SAFE holds $1M / $8M = 12.5% of Company Capitalization. Company Capitalization before the round is 9,000,000 founders + 1,000,000 pool (400k granted, 600k unallocated) + SAFE shares. Solving S = 0.125 \u00d7 (10,000,000 + S) gives S = 1,428,571 shares. Conversion price = $8M / 11,428,571 = $0.70.\n2. **Pool top-up.** The unallocated pool must be 10% of post-money FD. Existing unallocated is 600,000, so the top-up N satisfies 600,000 + N = 0.10 \u00d7 T. Top-up N = 946,939 shares, which sits in the pre-money.\n3. **Series A.** $5M of a $25M post-money is 20% of T, so Series A shares = 0.20 \u00d7 T.\n4. **Solve.** T = 9,000,000 + 1,000,000 + N + S + 0.20 \u00d7 T with N = 0.10 \u00d7 T \u2212 600,000, giving 0.70 \u00d7 T = 10,828,571 and T = 15,469,388.\n\n## Notes\n\n- **Pool shuffle cost.** The 946,939-share top-up is borne entirely by existing holders (founders and SAFE), not the new investor. The pre-money of $20M is effectively about $18.47M for the existing holders once the top-up is counted as pre-money dilution.\n- **Assumption: YC post-money SAFE mechanics.** The SAFE's Company Capitalization excludes the Series A pool increase, so the SAFE stays at 12.5% of pre-round capitalization. If the SAFE were instead computed on a capitalization that includes the top-up, the SAFE would receive more shares and the founders would be diluted further. Confirm the SAFE form with counsel.\n- **Assumption: cap applies.** The $8M cap price ($0.70) is below the Series A price ($1.616), so the cap governs conversion and the lack of a discount does not matter.\n- **Fractional shares.** Rounded to whole shares in the table; the actual cap table will need rounding rules agreed with counsel.\n"}]}, "opus-1": {"cost_usd": 0.15668, "seconds": 64, "output_tokens": 4701, "files": [{"name": "cap_table.csv", "text": "holder,shares,pct_post_money_fd\nFounder 1,4500000,29.0897%\nFounder 2,4500000,29.0897%\nOptions granted,400000,2.5858%\nUnallocated pool (existing),600000,3.8786%\nUnallocated pool (top-up),946939,6.1214%\nSAFE holder,1428571,9.2348%\nSeries A investor,3093877,20.0000%\nTotal,15469387,100.0000%\n"}, {"name": "series_a_dilution.md", "text": "# Series A: founder ownership after the round\n\nInputs come from `round.md`. You can reproduce the numbers with `python3 cap_table.py`, which also writes `cap_table.csv`.\n\n## Result\n\n**Each founder ends with 4,500,000 shares, which is 29.09% fully diluted. Together the founders own 58.18%.**\n\n| Holder | Shares | % post-money FD |\n|---|---:|---:|\n| Founder 1 | 4,500,000 | 29.09% |\n| Founder 2 | 4,500,000 | 29.09% |\n| Options granted | 400,000 | 2.59% |\n| Unallocated pool (existing) | 600,000 | 3.88% |\n| Unallocated pool (top-up) | 946,939 | 6.12% |\n| SAFE holder | 1,428,571 | 9.23% |\n| Series A investor | 3,093,877 | 20.00% |\n| **Total** | **15,469,387** | **100.00%** |\n\n## How it's calculated\n\n1. **SAFE conversion.** The SAFE is a YC post-money SAFE with an $8M cap, so it converts into $1M / $8M = 12.5% of the Company Capitalization. Under the standard YC definition, that figure counts the existing unissued pool. It leaves out any pool increase made for the Series A. So S = 12.5% \u00d7 (10,000,000 + S), which gives 1,428,571 shares at $0.70.\n2. **Pool top-up (pre-money).** The unallocated pool has to equal 10% of post-money FD. The investor takes $5M / $25M = 20%. The founders, granted options and SAFE add up to 10,828,571 shares, and those make up the remaining 70%. That puts post-money FD at about 15.47M shares. The unallocated pool comes to 1,546,939 shares, so the top-up is 946,939 new shares.\n3. **Series A price.** $20M \u00f7 12,375,510 pre-money FD shares = $1.6161 per share. $5M buys 3,093,877 shares.\n\n## What the bigger pool costs the founders\n\n| Scenario | Each founder | Both founders | A price/share |\n|---|---:|---:|---:|\n| No pool top-up (600k unallocated stays) | 31.50% | 63.00% | $1.7500 |\n| **10% unallocated pool in pre-money (investor's term)** | **29.09%** | **58.18%** | **$1.6161** |\n\nEach founder loses about 2.4 points (4.8 points for the two together). The founders pay for the whole top-up. The Series A investor keeps exactly 20%, and the standard post-money SAFE protects the SAFE holder from the pool increase. The effective pre-money is about $18.5M rather than $20M: 11,428,571 existing FD shares \u00d7 $1.6161.\n\n## Caveats\n- If the SAFE were *also* diluted by the top-up (that is non-standard, so check the signed SAFE), each founder would own about 28.72% (57.45% together).\n- The figures are on a fully diluted basis and include the whole pool. Share counts are rounded to whole shares, so percentages can be off by \u00b10.01%.\n"}]}, "opus-2": {"cost_usd": 0.19217, "seconds": 66, "output_tokens": 5708, "files": [{"name": "founder_ownership.md", "text": "# Series A pro-forma: founder ownership after the round\n\nInputs are from `round.md`. You can re-run `python3 cap_table.py` to regenerate the tables.\n\n## Answer\n\n| | Shares | % fully diluted post-money |\n|---|---:|---:|\n| Founder 1 | 4,500,000 | **29.09%** |\n| Founder 2 | 4,500,000 | **29.09%** |\n| Founders combined | 9,000,000 | **58.18%** |\n\nIf the pool were not topped up, each founder would hold 31.50%. **The bigger pool costs each founder about 2.4 points.**\n\n## How it was worked out\n\n1. **SAFE conversion (YC post-money SAFE, $8M cap, no discount).** The SAFE gets $1M / $8M = 12.5% of the \"Company Capitalization\". Under the YC definition, that capitalization includes the existing unissued pool (600k). It **excludes** any pool increase made for the Equity Financing, except to the extent Promised Options exceed the existing pool, which is not the case here.\n   - Base = 9,000,000 founders + 400,000 granted + 600,000 unallocated = 10,000,000\n   - SAFE shares S = 12.5% \u00d7 (10,000,000 + S) \u2192 **S = 1,428,571**\n2. **Pool top-up and Series A, solved together.** Let T be the post-money fully diluted shares.\n   - Series A = $5M / $25M post = 20% of T\n   - Unallocated pool = 10% of T (created in the pre-money)\n   - Founders + granted + SAFE = 9,000,000 + 400,000 + 1,428,571 = 10,828,571, which is the remaining 70% of T\n   - **T = 15,469,387**, pool = 1,546,939 (a top-up of **946,939** new options), Series A shares = 3,093,877\n   - Series A price = $20M / 12,375,510 pre-money FD shares = **$1.6161/share** ($1.75 without the top-up)\n3. Founders: 4,500,000 / 15,469,387 = **29.09% each**.\n\nThe founders and the SAFE holder carry the whole top-up. The Series A investor still gets a clean 20%, so their effective pre-money value is lower than $20M.\n\n## Full cap tables\n\n### Base case: YC post-money SAFE terms, pool topped up pre-money\n\nSeries A price/share: $1.6161\n\n| Holder | Shares | % FD post-money |\n|---|---:|---:|\n| Founder 1 | 4,500,000 | 29.09% |\n| Founder 2 | 4,500,000 | 29.09% |\n| Granted options | 400,000 | 2.59% |\n| Unallocated pool | 1,546,939 | 10.00% |\n| SAFE (converted) | 1,428,571 | 9.23% |\n| Series A investor | 3,093,877 | 20.00% |\n| **Total** | **15,469,387** | **100.00%** |\n\n### Reference: no pool top-up (600k pool kept)\n\nSeries A price/share: $1.7500\n\n| Holder | Shares | % FD post-money |\n|---|---:|---:|\n| Founder 1 | 4,500,000 | 31.50% |\n| Founder 2 | 4,500,000 | 31.50% |\n| Granted options | 400,000 | 2.80% |\n| Unallocated pool | 600,000 | 4.20% |\n| SAFE (converted) | 1,428,571 | 10.00% |\n| Series A investor | 2,857,142 | 20.00% |\n| **Total** | **14,285,713** | **100.00%** |\n\n### Reference: if the top-up were counted in the SAFE's capitalization (non-standard)\n\nSeries A price/share: $1.5957\n\n| Holder | Shares | % FD post-money |\n|---|---:|---:|\n| Founder 1 | 4,500,000 | 28.72% |\n| Founder 2 | 4,500,000 | 28.72% |\n| Granted options | 400,000 | 2.55% |\n| Unallocated pool | 1,566,667 | 10.00% |\n| SAFE (converted) | 1,566,666 | 10.00% |\n| Series A investor | 3,133,333 | 20.00% |\n| **Total** | **15,666,666** | **100.00%** |\n\n## Notes and assumptions\n- \"Fully diluted\" counts all common shares, granted options and the unallocated pool. The pool rule says \"unallocated must equal 10%\", so the 400k granted options are counted in addition to the 10%.\n- Fractional shares are rounded down for the SAFE and Series A, so the percentages may differ by about 1 share.\n- In the third table the SAFE holder is protected from the top-up. That gives them more shares and leaves the founders worse off (28.72%). It is shown only to make the point that the standard YC wording helps the founders here.\n- The size of the top-up is open to negotiation. Founders often push for a pool sized to a 12\u201318 month hiring plan rather than a flat 10%. Every 1% of extra post-money pool costs each founder roughly 0.4 points.\n- This is an illustrative model, not legal or tax advice. Check it against the actual SAFE and term sheet wording.\n"}]}, "opus-3": {"cost_usd": 0.17052, "seconds": 59, "output_tokens": 5065, "files": [{"name": "cap_table.md", "text": "# Series A pro-forma cap table\n\nInputs are from `round.md`. Generated by `cap_table.py`.\n\n## Answer\n\nAfter the round, **each founder owns 29.09%** (4,500,000 of 15,469,387 fully diluted shares), **58.18% combined**.\n\nWithout the pool top-up they would each own 31.50% (63.00% combined). The larger pool costs each founder about 2.41 percentage points.\n\n## Before the round (fully diluted)\n\n| Holder | Shares | % |\n|---|---:|---:|\n| Founder 1 | 4,500,000 | 45.00% |\n| Founder 2 | 4,500,000 | 45.00% |\n| Granted options | 400,000 | 4.00% |\n| Unallocated pool | 600,000 | 6.00% |\n| **Total** | **10,000,000** | **100.00%** |\n\n## After the Series A (fully diluted)\n\n| Holder | Shares | % |\n|---|---:|---:|\n| Founder 1 | 4,500,000 | 29.09% |\n| Founder 2 | 4,500,000 | 29.09% |\n| Granted options | 400,000 | 2.59% |\n| Unallocated pool (600,000 existing + 946,939 new) | 1,546,939 | 10.00% |\n| SAFE investor | 1,428,571 | 9.23% |\n| Series A investor | 3,093,877 | 20.00% |\n| **Total** | **15,469,387** | **100.00%** |\n\n## Workings\n\n1. **SAFE conversion.** A post-money SAFE buys $1.0M / $8.0M = 12.5% of the Company Capitalization. Under the YC post-money SAFE, that capitalization includes the existing pool and the SAFE's own shares but **excludes the new pool shares created for the Series A**. So 12.5% of (10,000,000 + S) = S, giving S = 1,428,571 shares at $0.7000/share.\n2. **Series A.** $5M on $20M pre is 20% of post. The unallocated pool must be 10% of post, so founders, granted options and SAFE shares (10,828,571) make up the remaining 70%. Post-money fully diluted = 15,469,387 shares.\n3. **Pool top-up.** Unallocated pool needed = 1,546,939 (10.00% of post), so 946,939 new options are added in the pre-money. Because they sit in the pre-money, the Series A investor still gets a full 20%. The founders, existing optionholders and the SAFE holder share that dilution. The SAFE gets no extra shares to make up for it.\n4. **Series A price** = $20,000,000 / 12,375,510 pre-money shares = $1.6161/share, for 3,093,877 shares (20.00%). The SAFE converts at the cap price ($0.7000) because it is below the Series A price.\n\n## Assumptions\n\n- \"Fully diluted\" includes granted options and the whole unallocated pool.\n- The SAFE is the standard YC post-money SAFE (pool increase excluded from its Company Capitalization); no other SAFEs or convertibles exist.\n- Fractional shares are rounded down for investors and up for the pool. Actual numbers will follow the closing documents.\n- Percentages are economic ownership, not voting control.\n"}]}, "haiku-1": {"cost_usd": 0.00748, "seconds": 38, "output_tokens": 7924, "files": [{"name": "cap_table.md", "text": "# Post-Series A cap table\n\nSource terms: round.md\n\n## Result\n\n| Holder | Shares | % post-money (fully diluted) |\n|---|---|---|\n| Founder 1 | 4,500,000 | 29.09% |\n| Founder 2 | 4,500,000 | 29.09% |\n| **Founders combined** | **9,000,000** | **58.18%** |\n| Existing options (granted) | 400,000 | 2.59% |\n| Unallocated pool (existing 600,000 + top-up 946,939) | 1,546,939 | 10.00% |\n| SAFE holder (converted) | 1,428,571 | 9.23% |\n| Series A investor | 3,093,878 | 20.00% |\n| **Total** | **15,469,388** | **100.00%** |\n\nSeries A price per share: $1.6161. SAFE conversion price: $0.70.\n\n## Workings\n\n1. **SAFE conversion.** Post-money SAFE, cap $8M, so the SAFE owns $1M / $8M = 12.5% of Company Capitalization (founders + existing pool + SAFE shares). Solving S = 0.125 \u00d7 (10,000,000 + S) gives S = 1,428,571 shares.\n2. **Pool top-up.** Let x be the new unallocated shares. Pre-money fully diluted = 10,000,000 + 1,428,571 + x. Series A takes 20% of post, so pre-money is 80% of post. Requiring unallocated pool = 10% of post gives 600,000 + x = 0.125 \u00d7 (11,428,571 + x), so x = 946,939.\n3. **Series A.** Pre-money FD = 12,375,510. Post-money FD = 12,375,510 / 0.8 = 15,469,388. Investor shares = 20% \u00d7 15,469,388 = 3,093,878.\n\n## Assumptions\n\n- The $20M pre-money includes SAFE conversion shares and the pool top-up (standard).\n- The post-money SAFE's Company Capitalization excludes the Series A pool increase (YC standard). The top-up therefore dilutes the SAFE holder too, which is why the SAFE ends at 9.23% rather than 12.5%.\n- Share counts are rounded to whole shares in the table; the percentages are computed on unrounded values.\n\n## Cost of the top-up to founders\n\nWithout the top-up (unallocated pool stays at 600,000), the founders would hold 63.00% combined (31.50% each). The required top-up takes about 4.8 points, bringing them to 58.18% combined.\n"}]}, "haiku-2": {"cost_usd": 0.00758, "seconds": 36, "output_tokens": 8085, "files": [{"name": "cap_table_series_a.md", "text": "# Post-Series A cap table (founder ownership)\n\n## Result\n\n| Holder | Shares | % post-money FD |\n|---|---|---|\n| Founder 1 | 4,500,000 | 29.09% |\n| Founder 2 | 4,500,000 | 29.09% |\n| **Both founders** | **9,000,000** | **58.18%** |\n| Granted options | 400,000 | 2.59% |\n| Unallocated pool (existing 600,000 + top-up 946,939) | 1,546,939 | 10.00% |\n| SAFE holder (converted at $8M cap) | 1,428,571 | 9.24% |\n| Series A investor ($5M) | 3,093,878 | 20.00% |\n| **Total post-money FD** | **15,469,388** | **100.00%** |\n\n## Key figures\n\n- SAFE conversion price: $8,000,000 / 11,428,571 = $0.70 per share. The Series A price (about $1.616) is higher, so the cap applies.\n- Pool top-up needed: 946,939 new shares, created in the pre-money.\n- Pre-money fully diluted shares: 12,375,510.\n- Series A price: $20,000,000 / 12,375,510 = $1.6161 per share.\n\n## Method\n\n1. SAFE: under the YC post-money definition, the SAFE owns $1M / $8M = 12.5% of company capitalization before the round. Company capitalization excludes the Series A pool top-up. So S = 0.125 \u00d7 (10,000,000 + S), giving S = 1,428,571.\n2. Pool: the unallocated pool must be 10% of post-money FD. Post-money FD = 1.25 \u00d7 pre-money FD. So 600,000 + P = 0.125 \u00d7 (11,428,571 + P), giving P = 946,939.\n3. Series A: $5M / $20M pre = 25% of pre-money FD, giving 3,093,878 shares. Post-money FD = 15,469,388.\n\n## Assumptions\n\n- Standard YC post-money SAFE, with the Series A pool increase excluded from company capitalization for conversion.\n- No other convertible securities, warrants, or promised options.\n- The founders' 9,000,000 shares are all the common stock outstanding before the round.\n- The Series A pre-money of $20M includes the SAFE conversion shares and the top-up pool (the option pool shuffle).\n"}]}, "haiku-3": {"cost_usd": 0.00622, "seconds": 27, "output_tokens": 5927, "files": [{"name": "cap_table.md", "text": "# Post-Series A cap table\n\nFounders after the round: **9,000,000 shares, 58.18% combined** (4,500,000 each, 29.09% each).\n\n## Post-money fully diluted\n\n| Holder | Shares | % |\n|---|---:|---:|\n| Founder 1 | 4,500,000 | 29.09% |\n| Founder 2 | 4,500,000 | 29.09% |\n| Granted options | 400,000 | 2.59% |\n| Unallocated pool (600,000 existing + 946,939 top-up) | 1,546,939 | 10.00% |\n| SAFE conversion | 1,428,571 | 9.24% |\n| Series A | 3,093,878 | 20.00% |\n| **Total** | **15,469,388** | **100.00%** |\n\n## Derivation\n\n- **SAFE:** $1M / $8M cap = 12.5% of Company Capitalization. Under the YC post-money SAFE definition, Company Capitalization includes founders, existing options and the existing unallocated pool, plus the SAFE itself, but excludes the Series A pool top-up. So CC = 10,000,000 + S, and S = 0.125 \u00d7 CC gives S = 10,000,000 / 7 \u2248 1,428,571 shares at $0.70/share.\n- **Series A:** $5M / $25M post-money = 20% of post-money FD, so A = 0.20 T.\n- **Pool:** unallocated shares must equal 10% of T, so U = 0.10 T. The top-up is U \u2212 600,000 = 946,939 shares, which sits in the pre-money.\n- **Solve:** T = 9,000,000 + 400,000 + S + 0.10T + 0.20T, so 0.70T = 9,400,000 + 1,428,571, giving T \u2248 15,469,388.\n- **Check:** the cap binds. Series A price is $20M / 12,375,510 pre-money shares \u2248 $1.616, well above the SAFE's $0.70.\n\n## Assumptions\n\n- The 10% target is total unallocated pool (existing 600,000 included), not a new 10% block.\n- Standard YC post-money SAFE conversion mechanics, with the pool top-up excluded from Company Capitalization.\n- Fully diluted includes granted options. Share counts are rounded to whole shares.\n"}]}}, "verdicts": [{"vs": "opus", "pair": "1", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "haiku+skill", "why": "Both calculate founder ownership correctly. B contradicts its own calculation by claiming the SAFE is protected from pool expansion; the founders and SAFE holder both bear that dilution. B\u2019s alternative scenario is therefore misleading. A correctly describes the SAFE treatment."}, {"vs": "opus", "pair": "1", "judge": "gpt", "order": "opus shown first", "winner": "haiku+skill", "why": "Both calculate founder ownership correctly. A falsely claims the SAFE is protected from pool dilution and supplies a misleading alternative. B is consistent, though it omits explicit Series A pricing arithmetic and a clear explanation of who bears pool dilution."}, {"vs": "opus", "pair": "1", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "tie", "why": "Both correctly calculate and present the required founder ownership and full cap table."}, {"vs": "opus", "pair": "1", "judge": "deepseek", "order": "opus shown first", "winner": "tie", "why": "Both correctly calculate all required figures and present them clearly for the founders."}, {"vs": "haiku", "pair": "1", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "haiku", "why": "Both reach correct founder ownership. B explicitly reconciles pre-money capitalization and round pricing, explains dilution from the pool expansion, and clarifies rounding. A leaves pricing less verifiable and omits the explicit dilution explanation."}, {"vs": "haiku", "pair": "1", "judge": "gpt", "order": "haiku shown first", "winner": "haiku", "why": "Both correctly calculate founder ownership, SAFE conversion, pool expansion, and financing. A more clearly explains how the pre-money top-up dilutes founders and the SAFE holder, quantifies its founder impact, and correctly rounds SAFE ownership."}, {"vs": "haiku", "pair": "1", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "tie", "why": "Both deliver identical, correct results and clear methodologies, fully meeting all rubric criteria."}, {"vs": "haiku", "pair": "1", "judge": "deepseek", "order": "haiku shown first", "winner": "tie", "why": "Both deliver identical correct results and clear calculations."}, {"vs": "opus", "pair": "2", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "opus", "why": "Both calculate founder ownership correctly. A incorrectly says the SAFE avoids top-up dilution and overstates total reserved options by 400,000 shares. B leads with the answer and correctly explains the dilution mechanics."}, {"vs": "opus", "pair": "2", "judge": "gpt", "order": "opus shown first", "winner": "opus", "why": "A correctly leads with founder ownership and reconciles the capitalization. B calculates ownership correctly but falsely says the SAFE avoids top-up dilution and overstates total reserved options: 1,946,939, not 2,346,939."}, {"vs": "opus", "pair": "2", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "tie", "why": "Both deliver the same correct final ownership percentages and calculations, meeting all rubric criteria."}, {"vs": "opus", "pair": "2", "judge": "deepseek", "order": "opus shown first", "winner": "tie", "why": "Both correctly calculate and present founder ownership and all intermediate steps."}, {"vs": "haiku", "pair": "2", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "haiku", "why": "B leads with correct founder ownership and reconciles capitalization. A incorrectly says the top-up does not dilute the SAFE and overstates total reserved options by 400,000 shares. B\u2019s SAFE percentage rounding error is minor."}, {"vs": "haiku", "pair": "2", "judge": "gpt", "order": "haiku shown first", "winner": "haiku", "why": "Both calculate founder ownership correctly. A leads with the result. B overstates total reserved options by 400,000 and incorrectly says the SAFE avoids top-up dilution. A\u2019s SAFE percentage has a minor rounding error; neither clearly explains dilution."}, {"vs": "haiku", "pair": "2", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "tie", "why": "Both deliver identical correct calculations and clear, usable results."}, {"vs": "haiku", "pair": "2", "judge": "deepseek", "order": "haiku shown first", "winner": "tie", "why": "Both deliverables correctly calculate all required figures and present the same final founder ownership percentages."}, {"vs": "opus", "pair": "3", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "opus", "why": "Both calculate founder ownership correctly. B explicitly reconciles pre-money pricing and correctly explains that existing optionholders also bear pool dilution. A incorrectly attributes that dilution entirely to founders and the SAFE holder."}, {"vs": "opus", "pair": "3", "judge": "gpt", "order": "opus shown first", "winner": "tie", "why": "Both correctly calculate each founder\u2019s 29.09% post-round fully diluted ownership, SAFE conversion, unallocated pool expansion, and Series A pricing. Both provide verifiable, consistent reconciliations; whole-share rounding differences are immaterial."}, {"vs": "opus", "pair": "3", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "tie", "why": "Both deliver the correct ownership percentages, calculations, and a usable reconciliation."}, {"vs": "opus", "pair": "3", "judge": "deepseek", "order": "opus shown first", "winner": "tie", "why": "Both deliverables correctly compute all required figures and meet all rubric criteria."}, {"vs": "haiku", "pair": "3", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "haiku+skill", "why": "Both correctly calculate founder ownership, SAFE conversion, pool expansion, and financing. A additionally explains who bears the pre-money pool dilution. B omits that explanation; its SAFE percentage rounding discrepancy is immaterial."}, {"vs": "haiku", "pair": "3", "judge": "gpt", "order": "haiku shown first", "winner": "haiku+skill", "why": "Both calculate founder ownership correctly. B better explains who bears the pre-money pool dilution and correctly rounds SAFE ownership to 9.23%. Its dilution explanation should also include existing granted-option holders."}, {"vs": "haiku", "pair": "3", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "tie", "why": "Both deliver the same correct final numbers and calculations, meeting all rubric criteria."}, {"vs": "haiku", "pair": "3", "judge": "deepseek", "order": "haiku shown first", "winner": "tie", "why": "Both deliver identical correct calculations and clear presentation."}]}, {"id": "msg-2", "family": "message-testing", "prompt": "Help us figure out which value proposition to use on our new site. Details in brief.md.", "input": "brief.md", "skill": "laja-wynter-message-testing", "input_text": "Product: Clinicflow, patient intake software for private physiotherapy clinics (online forms, insurance checks, automatic reminders).\nCandidate messages:\n1) \"Cut no-shows by 40%.\"\n2) \"Your front desk, without the phone tag.\"\n3) \"Intake that's ready before the patient walks in.\"\nWe have: 80 customers we can survey, an email list of 4,000 clinic owners, and EUR 2,000.\nOur agency wants to run Google Ads A/B tests on the three headlines for two weeks.\n", "rubric": [{"id": "c1", "criterion": "Give a clear, usable recommendation for the new site's value proposition.", "full_marks_when": "Recommends one of the three messages or a concrete revision, explains its fit for private physiotherapy clinic owners, and distinguishes a provisional choice from a validated winner.", "zero_when": "Only offers generic positioning advice or lists pros and cons without recommending a message or a concrete way to choose."}, {"id": "c2", "criterion": "Assess the messages against what Clinicflow actually does and what each headline promises.", "full_marks_when": "Connects the messages to online forms, insurance checks, and automatic reminders; identifies that the phone-tag headline could imply broader front-desk or telephone capabilities; and assesses whether the intake-readiness headline communicates a relevant, credible benefit.", "zero_when": "Treats the headlines as interchangeable copy styles or recommends a promise that depends on capabilities absent from the brief."}, {"id": "c3", "criterion": "Maintain factual and numerical accuracy, especially around the no-show claim.", "full_marks_when": "Accurately uses the three candidate messages, 80 customers, 4,000 clinic owners, EUR 2,000, and the proposed two-week test; treats the 40% reduction as an unverified claim requiring supporting evidence; and makes no unsupported quantitative predictions or arithmetic errors.", "zero_when": "Presents the 40% reduction as established, invents results or product facts, materially misstates the available resources, or makes arithmetic errors that undermine the recommendation."}, {"id": "c4", "criterion": "Judge whether the agency's proposed advertising test can answer the positioning question.", "full_marks_when": "Explains that ad clicks alone do not establish the best site value proposition, that two weeks and EUR 2,000 do not guarantee enough relevant conversions, and that testing three headlines requires a three-variant comparison or an explicit staged design. Advises whether to proceed, revise, or defer the proposal with reasons specific to this brief.", "zero_when": "Endorses the proposed test without examining traffic relevance, conversion volume, or what its results would actually establish."}, {"id": "c5", "criterion": "Provide a feasible research plan using the audiences and budget already available.", "full_marks_when": "Gives actionable steps for learning from the 80 customers about experienced benefits and from the 4,000 clinic owners about prospect relevance, while recognizing that customers and prospects provide different evidence. Specifies what to ask or show, how to recruit or distribute, and how the work fits within EUR 2,000.", "zero_when": "Offers only vague instructions to survey or test, ignores the available audiences, assumes everyone will respond, or proposes work that exceeds the budget without justification."}, {"id": "c6", "criterion": "Define how evidence will translate into a final headline decision.", "full_marks_when": "Sets practical decision rules using comprehension, relevance, credibility, and a meaningful prospect action such as a qualified demo request. Keeps comparisons fair, explains how to handle mixed or sparse results, and avoids declaring a winner solely from headline popularity or click-through rate.", "zero_when": "Provides no decision rule, confuses preference or clicks with buying intent, or treats a small or inconclusive difference as proof of a winning proposition."}], "runs": {"haiku-skill-1": {"cost_usd": 0.01289, "seconds": 74, "output_tokens": 14741, "files": [{"name": "value-prop-diagnosis.md", "text": "# Clinicflow value proposition: diagnosis and test plan\n\n**Status: provisional expert review and proposed retest.** No buyer responses exist yet. Nothing below is buyer-validated, and no respondent comments or scores are included.\n\n## 1. Decision and evidence status\n\n- **Decision:** Which of three headlines leads the new Clinicflow site, and whether the EUR 2,000 Google Ads test should run as currently planned.\n- **Research question:** Which headline lets prospective private physiotherapy clinic owners correctly say what Clinicflow does, and see it as relevant to a problem they have?\n- **Desired next action:** Assumed to be a demo request. The brief does not name one; confirm before the test.\n- **Message hypothesis:** Headline 1 is the most specific and tied to a costly problem (missed appointments), but its number (\"40%\") has no source in the brief. Headlines 2 and 3 describe outcomes, but none of the three says what the product is, so the subheading must name the category for any of them to work.\n- **Success condition (set before any data arrives):** A headline is preferred only if:\n  - (a) a majority of prospect paraphrases name the category (patient intake software for physio clinics) and the main outcome;\n  - (b) it ranks highest on relevance among prospects, not just among existing customers;\n  - (c) its claim is verified, or rewritten to a verified version.\n- **Kill condition:** If no headline meets (a) among prospects, fix the subheading and retest before choosing a headline.\n- **What would change this recommendation:**\n  - Source data for the 40% figure would strengthen Headline 1.\n  - Customer data showing front-desk time or intake paperwork is the main pain would shift weight toward Headlines 2 and 3.\n  - Agency conversion data showing very low expected volume would rule out the ad test.\n\n## 2. Buyer context and gaps\n\n| Field | Current evidence | Gap or assumption |\n| --- | --- | --- |\n| Role, seniority, size | Target is \"clinic owners\" (email list). Customers: 80, roles unknown. | Confirm whether owner, practice manager, or front desk is the real buyer and user. Clinic size unknown. |\n| Budget owner | Not in brief | Assumed to be the owner. |\n| Priority / pain | Not in brief. The three headlines imply three different priorities: missed appointments, front-desk phone load, pre-visit paperwork. | Unknown which one drives purchase. |\n| Trigger and success measure | Not in brief | Unknown. |\n| Current approach and alternatives | Not in brief. Likely: paper forms, manual reminder calls or texts, phone booking, existing practice software. | Unverified. |\n\n**Cheapest buyer context:** The 80 customers can answer what problem made them buy and how they handled it before. They are knowledgeable, so their results must be reported separately from prospect results.\n\n## 3. Test record (planned, not completed)\n\n- **Study type:** Two parts. (1) Monadic comprehension, then comparative preference, sent to prospects from the 4,000-contact list. (2) The same comparative question with the 80 customers, plus a question about the problem they bought for.\n- **Stimulus:** A mock site hero for each arm. The only difference between arms is the headline. Subheading held constant for all three, as a draft:\n  > \"Patient intake software for private physiotherapy clinics: online forms, insurance checks and automatic reminders.\"\n\n  The \"insurance checks\" wording is a claim from the brief and needs verification before use.\n- **Prospect audience:** Clinic owners on the list who are not customers. Randomly split into three arms.\n- **Customer audience:** The 80 existing customers. Report separately.\n- **Sample:** Plan for at least 15 interpretable prospect responses per headline, the skill's qualitative default. This is planning guidance, not a significance threshold. Response rate from a cold list is unknown; if fewer than 15 per arm arrive, extend the send or treat results as directional only.\n- **Questions, in order:**\n  1. *(Prospects, one headline each, unaided)* \"In your own words, what does this company do and who is it for?\"\n  2. *(Prospects)* \"How relevant is this to a problem you have at your clinic?\" 1 = not a problem for us; 5 = a top problem we would act on. Then: \"Why?\"\n  3. *(Prospects, all three shown)* \"Which would you most want to learn more about? What does it tell you that the others don't?\"\n  4. *(Customers)* \"What problem were you trying to solve when you bought? How did you handle it before?\" Then the same comparative question as 3.\n- **Behavioral measure:** Demo-request click-through from the email is a separate measure of action. Do not report it as comprehension evidence, and do not treat comprehension results as proof it will convert.\n- **Compliance:** Confirm the email list was collected under a consent basis that covers research or marketing mail (EU). Do not ask respondents for patient data.\n- **Limitations:** Mock hero is not the full page. Customers are not prospects. Clinic owners on the list may already know the company, which biases results.\n\n## 4. Findings: ordered diagnosis (expert hypothesis, not evidence)\n\n| Layer | Assessment | Hypothesis |\n| --- | --- | --- |\n| Clarity | **Unknown, likely fail for H2 and H3.** H1 states an outcome but not a category. H2 (\"phone tag\") does not say what the product is. H3 (\"intake\") may be jargon to some owners. | All three need the subheading to name the category. Test with a constant subheading so the headline effect is isolated. |\n| Relevance | **Unknown.** H1 names the most concrete pain. H2 is relevant if front-desk load is a pain. H3 is indirect for owners unless they care about pre-visit paperwork. | Depends on which pain the 80 customers actually bought for. |\n| Value | H1 promises a measurable result that is unverified. H2 and H3 benefits are qualitative and easy to agree with, but also easy to ignore. | H1 is only as strong as its source. |\n| Differentiation | **Not assessed.** None of the three names an alternative. Against paper forms or manual reminders the difference is plausible but unproven. H2 and H3 resemble generic software claims. | Name the alternative buyers actually use, then substantiate. |\n\n- **Earliest failed layer (hypothesis):** Clarity for H2 and H3. For H1 the main risk is the proof gap, not clarity.\n- **Friction:** Doubt about the 40% figure. Unclear which payers \"insurance checks\" covers.\n\n## 5. Claim and sameness audit\n\n| Claim | Buyer question | Real alternative | Proof in brief | Decision |\n| --- | --- | --- | --- | --- |\n| H1: \"Cut no-shows by 40%\" | Will this work for my clinic, and by how much? | Manual reminder calls or texts from staff | None | **Verify or remove.** Do not publish the number without documented source data (e.g., before-and-after no-show rates from customers). |\n| H2: \"Without the phone tag\" | Does this cut calls to and from patients? | Phone booking and call-backs | None | **Verify.** Needs a measured reduction in calls or a customer statement with a source. |\n| H3: \"Ready before the patient walks in\" | Will patients finish forms before arrival? | Paper forms on arrival | None | **Verify.** Needs a measured pre-visit completion rate. |\n| Subheading: \"insurance checks\", \"automatic reminders\" | Which insurers? What happens automatically? | Manual checks and reminders | None in brief | **Verify** the exact capability before use. |\n\n## 6. Proposed revision (provisional)\n\n- **Hero headline:** Headline 1 is the lead candidate only if its figure is verified. If not, use a version without a number, for example \"Fewer no-shows, with reminders that run themselves.\" Provisional wording, not tested.\n- **Subheading:** Category plus concrete capability, as in section 3. Verify each capability named.\n- **Difference:** Not yet defensible. It needs a named alternative (manual reminders, paper forms) and a measured comparison.\n- **Proof placement:** Customer result or measured figure directly beneath the claim it supports, with its source and date.\n- **CTA:** \"Book a demo\" (assumed).\n- **Page outline:**\n  1. Hero: headline, subheading, CTA.\n  2. Problem: missed appointments, phone calls, paper forms, each in the clinic's own words.\n  3. How it works: online forms, insurance checks, reminders.\n  4. Proof: customer result with source.\n  5. CTA repeated.\n- **Claim status:** H1 number: unverified. H2 and H3 outcomes: unverified. Insurance and reminder capabilities: unverified. No claim is validated.\n\n## 7. Prioritized changes\n\n1. **Check the 40% source.** Ask the company where it came from. If it is not documented, remove the number before any ad or page uses it. Owner: unassigned.\n2. **Add the category to the subheading for every candidate.** Owner: unassigned.\n3. **Pause the three-way ad test as designed** and run the prospect email test first (section 8).\n4. **Collect verified proof** for each outcome claim, starting with the 80 customers.\n5. **Ask customers which problem they bought for**, in the same survey as section 3.\n\n## 8. Recommendation and retest\n\n- **Recommendation: revise, and gather buyer context before choosing.** Do not ship any headline as validated yet.\n- **Why not the EUR 2,000 three-way ad test as planned:**\n  - Ads measure clicks, not comprehension. A vague headline can win clicks and attract people who misunderstand the product.\n  - Three arms split the budget thinly. Illustrative assumptions (not from the brief): at a EUR 3 cost per click and 3% of clicks becoming demo requests, each arm gets about EUR 667 of spend, about 220 clicks, and about 7 demo requests. That is too few to separate three headlines reliably. Ask the agency for its actual cost-per-click and conversion estimates before spending.\n- **Proposed sequence:**\n  1. **Survey the 80 customers and run the prospect email test** (section 3). Cost is mainly staff time and a survey tool.\n  2. **If one headline clearly meets the success condition**, use it as the lead and retest only the page, not the headline.\n  3. **If two finalists remain tied**, run a short ad test on those two only, measured on demo requests, not clicks. Hold the EUR 2,000 until then.\n  4. **Verify all claims** before any ad or page goes live.\n- **Remaining uncertainty:** This plan cannot show that a headline drives sign-ups, only how prospects interpret and value it. The customer sample is not representative of prospects.\n- **Maintenance trigger:** Review in three months, or when pricing, features, or the target segment changes. Do not rerun the test for typo or design-only edits.\n"}]}, "haiku-skill-2": {"cost_usd": 0.00858, "seconds": 56, "output_tokens": 10152, "files": [{"name": "value-prop-recommendation.md", "text": "# Clinicflow: value proposition for the new site\n\n## Recommendation\n\nUse **\"Intake that's ready before the patient walks in.\"** as the site's primary headline.\n\nSwitch to **\"Cut no-shows by 40%.\"** only if the 80 customers' own data supports a number at least that large, and then cite the source on the page. Don't use \"Your front desk, without the phone tag.\" as the primary headline.\n\n## Why headline 3 is the default\n\n- **It's a claim the product can keep.** Completed intake before arrival is in your control. No-show reduction depends on patient behavior, and reminders only move it part of the way.\n- **It describes what the buyer gets.** It names the moment the owner cares about (the patient arriving ready) without a number that needs defending.\n- **It avoids an unsupported statistic.** \"40%\" has no source in the brief. Performance claims in advertising need evidence, and a precise figure with no source is a compliance risk in EU markets.\n\nHeadline 1 is the strongest hook for owners, since no-shows hit revenue directly. That's why it's the candidate to check against customer data, not drop outright.\n\nHeadline 2 is the weakest. \"Phone tag\" is vague, it doesn't say what the product does, and it may read oddly if the site isn't written for native English speakers.\n\n## Step 1: Check the evidence you already have (about a week, no spend)\n\n1. **No-show data from customers.** Pull before-and-after no-show rates for the customers who have run Clinicflow for at least three months.\n   - If the median reduction is at least 30\u201340% and you can show it, headline 1 becomes viable with a source line.\n   - If not, headline 1 stays out of the primary position.\n2. **Customer interviews or survey (80 customers).** Ask which problem they bought the product to solve, and which of the three lines describes it.\n   - If most say \"my front desk was overwhelmed\" or \"the phone never stopped,\" headline 2 becomes a serious option.\n   - If most say \"patients arrived with forms unfinished,\" headline 3 is confirmed.\n   - Save their exact wording. Their phrasing often beats any headline we write.\n\n## Step 2: Test the headlines where the sample is big enough (about two weeks)\n\nThe agency's plan has three problems at this budget.\n\n**Sample size.** Assuming a EUR 3 cost per click, EUR 2,000 across three arms buys about 220 clicks per arm. Those numbers are illustrative; replace them with real figures from the first days of the test.\n\n| What we measure | Clicks per arm needed to detect the difference | Clicks per arm available | Verdict |\n|---|---|---|---|\n| Click-through rate, 4% \u2192 5% (+25%) | About 6,700 impressions per arm (about 270 clicks) | About 5,500 impressions per arm at 4% CTR | Borderline; only large differences show up |\n| Signup rate, 3% \u2192 6% (2x) | About 750 clicks per arm | About 220 | Not detectable |\n\nThe test can only see large click-through differences. It can't show which headline produces signups, which is the outcome that matters.\n\n**Wrong metric.** Click-through rate measures curiosity. A headline that attracts clicks from the wrong clinics wins the test and loses in sales.\n\n**Rotation setting.** Google's default \"optimize\" ad rotation favors early winners and skews the test. Use even rotation, and keep keywords and ad groups identical across arms.\n\n**Changes to the plan:**\n\n- **Run the headlines by email first.** Send the 4,000 owners three versions, each with one headline as the lead line, about 1,330 per arm. Measure click-through to the site, not opens.\n  - This is cheap and reaches the actual buyers. It can detect a roughly 2x difference in click rate (about 1,140 recipients per arm). Signups from email will be too few to measure, so use email for click-through only.\n- **If Google is still needed, test two headlines, not three.** Drop headline 2 and run headline 1 against headline 3. Two arms at about EUR 1,000 each gives about 330 clicks per arm.\n  - Set the conversion goal to a landing-page signup, and keep click-through as a secondary metric.\n\n## Decision rules\n\n- **Customer data supports 40% (Step 1):** primary = headline 1, with a source line. Still run the email test to confirm headline 3 isn't clearly better.\n- **Customers say \"front desk\" most often:** primary = headline 2, unless email shows headline 3 clearly out-clicks it.\n- **Otherwise:** primary = headline 3.\n- **Email or Google result is inconclusive** (differences within noise): treat as a tie and follow the customer evidence. Don't spend more budget trying to force a winner.\n\n## Open questions\n\n- What is the real cost per click for the target keywords? Search volume for niche terms like \"physiotherapy intake software\" may be too low for two weeks of ads to produce a usable sample.\n- Which language is the site written in? \"Phone tag\" is idiomatic English.\n- Do we have a source for the 40% figure, or is it a target?\n"}]}, "haiku-skill-3": {"cost_usd": 0.01194, "seconds": 61, "output_tokens": 11388, "files": [{"name": "value-proposition-review.md", "text": "# Clinicflow value proposition review (provisional)\n\n## 1. Decision and evidence status\n\n- **Decision:** Which of three headlines to use as the lead value proposition on the new Clinicflow site.\n- **Research question:** Do private physiotherapy clinic owners (and the front-desk staff who run intake) understand each headline as an offer for them, and which outcome do they find worth pursuing?\n- **Desired next action:** Book a demo or start a trial.\n- **Status:** Provisional expert review. No buyer responses exist yet. Nothing below is validated.\n- **Message hypothesis:** A headline that names a concrete, buyer-relevant outcome, with a clear category cue, will earn attention and be understood as intake software for physio clinics.\n- **Success condition (define before testing):** Prospects restate the offer and its category without prompting, connect it to a current priority, and raise doubts that can be answered. Clicks alone do not decide it.\n\n## 2. Buyer context and gaps\n\n| Field | Current evidence | Source/date | Gap or assumption |\n| --- | --- | --- | --- |\n| Role, seniority, industry, size | Clinic owners (email list); private physio clinics | Brief | Front-desk managers are not described. Headline 2 assumes they matter to the purchase. |\n| Buying involvement and budget owner | Not in brief | \u2014 | Unknown whether the owner signs off alone. |\n| Priority, symptoms, desired gain | Not in brief | \u2014 | Assumed: missed appointments cost revenue (Inferred). Unverified. |\n| Trigger and success measure | Not in brief | \u2014 | Unknown. |\n| Buying process and constraints | Not in brief | \u2014 | Unknown. Insurance-check feature suggests a billing/admin concern. |\n| Current approach and alternatives | Phone calls and paper/manual intake implied by \"phone tag\" (Inferred) | \u2014 | Alternatives buyers actually consider are unknown. |\n\nThe 80 customers can fill most of these gaps. Use them for buyer context, not for comprehension scoring. They already know the product, so their reactions will not show what a first-time visitor understands.\n\n## 3. Headline assessment\n\nEach headline is judged against the ordered layers (clarity, relevance, value, differentiation). These are Inferred from the copy alone, not from buyer responses.\n\n| Headline | Clarity | Relevance | Value | Differentiation | Main risk |\n| --- | --- | --- | --- | --- | --- |\n| 1. \"Cut no-shows by 40%.\" | No category cue. Reader can't tell it's software. | Likely high if no-shows cost clinics money (assumption). | Strong, concrete outcome. | None. Any reminder or booking tool could claim it. | **The 40% figure has no source in the brief.** Running an unsupported statistic in paid ads is a compliance and credibility risk. |\n| 2. \"Your front desk, without the phone tag.\" | Partial. \"Front desk\" hints at admin tools, but still no category cue. | Depends on whether the reader runs the front desk or buys for it. Owners on the list may not feel this. | Moderate. The gain is staff time, which is harder to value. | Weak. Any intake or booking tool can claim it. | Speaks to a role that may not hold the budget. |\n| 3. \"Intake that's ready before the patient walks in.\" | Best of the three. \"Intake\" is a category cue for the right reader. | Specific to clinic workflow. | Moderate to strong. States a clear, verifiable result (intake done before arrival). | Weak alone. Ready-before-arrival is a plausible claim for competitors too. | Clinic owners who don't use the word \"intake\" may not connect it to their problem. |\n\nNone of the three names a real alternative or a structural difference. That gap matters more than which headline wins. The differentiation layer is currently unsupported for all three.\n\n**Provisional lean: headline 3**, with headline 1 as the strongest candidate only if the 40% can be substantiated. Reasons:\n- Headline 3 is the only one that is both concrete and free of an unverified number.\n- Headline 1 is the strongest promise, but it depends entirely on a figure the brief does not support.\n- Headline 2 targets a role that may not be the buyer.\n\nThis is a lean from expert reading, not evidence. It should change if buyer responses point elsewhere.\n\n## 4. Revision requirements (apply to whichever headline is chosen)\n\nHeadlines alone do not name the product category. Test and launch each headline with the same subheading so the test measures the promise, not the category cue. Proposed, untested subheading built only from the features in the brief:\n\n> \"Clinicflow: online patient forms, insurance checks and automatic reminders for private physiotherapy clinics.\"\n\n- **Hero headline:** One of the three, per the test result.\n- **Subheading:** As above (Proposed).\n- **Difference:** Not yet defensible. Needed before launch: what Clinicflow does that alternatives don't, or a named alternative (e.g., manual intake, a general booking tool) and the specific reason to choose Clinicflow.\n- **Proof placement:** Any no-show figure must sit beside its source (client data, period, sample). If no source exists, remove the number and state the mechanism instead (e.g., automatic reminders), once that is verified.\n- **CTA:** Demo request, with a stated expectation (e.g., length of call).\n- **Claim status:**\n  - \"40% fewer no-shows\": **Unverified.** Do not use publicly or in ads until a source exists.\n  - \"Automatic reminders,\" \"online forms,\" \"insurance checks\": Feature claims from the brief. Confirm they work as described.\n  - \"Ready before the patient walks in\": Outcome claim. Confirm it with customers.\n\n## 5. Study plan within budget\n\n### Step 0. Verify the 40% claim (cost: staff time)\nAsk whoever wrote headline 1 for the source: which clinics, what period, what baseline, what sample. If it cannot be produced, headline 1 can't run in paid ads as written.\n\n### Step 1. Prospect preference and comprehension test (cost: survey tool, no media spend)\n- **Recruit:** About 30 prospects from the 4,000-contact list. Record role (owner vs front desk). If roles differ, use about 15 per role so each group is readable.\n- **Stimulus:** Each of the three headlines, each with the same subheading, shown in a neutral frame. Preference tests start at about 30 respondents.\n- **Question order (ask before revealing any intended category):**\n  1. For this headline alone: what do you think this company offers? Explain in your own words.\n  2. Who would use this, and in what situation?\n  3. Which part relates to a current priority in your clinic? What feels less relevant?\n  4. What would you use instead today?\n  5. What makes you doubt this claim? What information is missing?\n  6. Across the three, which would you want to learn more about, and why?\n- **Limitations to record:** An email list skews toward engaged contacts. A 30-person panel does not give population percentages or significance. Use it to find misreadings and objections, not to rank conversion.\n\n### Step 2. Customer buyer-context interviews (cost: staff time, uses the 80 customers)\nAsk what problem they bought Clinicflow to solve, what their no-show and front-desk situation was before, what they measured, and which headline they would have responded to. Use this to confirm or reject the outcome each headline promises. Do not count customer reactions as prospect comprehension.\n\n### Step 3. Google Ads only after Steps 0 to 2 (decide budget first)\nThe agency's plan of three headlines for two weeks on EUR 2,000 is underpowered, using standard A/B sample sizes (two-sided alpha 0.05, power 0.8):\n\n| Question | Clicks or impressions needed per headline | Three headlines total |\n| --- | --- | --- |\n| CTR moves from 3% to 4% | about 5,300 impressions | about 16,000 impressions |\n| Demo-request rate moves from 5% to 8% | about 1,060 clicks | about 3,200 clicks |\n\nTo reach about 1,060 clicks per headline across three headlines, you need about 3,200 clicks in total. At an illustrative CPC of EUR 1.00 (assumption; check the real figure in Google Ads Keyword Planner), EUR 2,000 buys about 2,000 clicks, or about 670 per headline. That is short of what a demo-rate difference needs, before any conversion data arrives.\n\nRecommendations for the paid test:\n- **Test two headlines, not three.** Use the two that survived Step 1.\n- **Judge on demo requests, not CTR.** CTR rewards curiosity and bold numbers. Headline 1 would likely win on CTR for that reason alone, which says nothing about whether it's the right promise.\n- **Do not run headline 1 if Step 0 fails.**\n- **Keep ad groups or rotations separate** so headlines get equal traffic. Otherwise rotation settings decide the result.\n\n## 6. Decision rule (declare before testing)\n\n- **Ship headline 3** if prospects restate it correctly without prompting, connect it to a priority, and their doubts can be answered. This is the provisional default.\n- **Ship headline 1** only if Step 0 verifies the 40% figure, prospects connect it to a priority, and the doubts it raises are answered.\n- **Ship headline 2** only if owners (not just front-desk staff) connect it to their purchase.\n- **Revise all three** if prospects misclassify the offer or cannot restate it. That is a clarity failure, and the fix is the subheading and category cue, not a new headline.\n- **Gather more buyer context** if prospects show no priority connection to any headline. The problem itself would then be unknown.\n\n## 7. What this review cannot establish\n\n- Which headline produces more demo requests or revenue. Message feedback is not conversion evidence, and Step 3 is too small to show it.\n- Whether the 40% claim is true (Step 0 is needed for that).\n- Whether Clinicflow has a defensible difference from alternatives. That needs a separate competitor and buyer review.\n\n## 8. Checklist status\n\n- Decision defined: PASS (above)\n- Buyer context usable: FAIL. Priority, budget owner, and alternatives are unknown. Step 2 addresses this.\n- Correct study: PASS for Step 1 (preference test with three viable directions). Paid test is provisional pending Steps 0 to 2.\n- Stimulus traceable: FAIL. No live page; headlines only. Subheading is Proposed.\n- Audience traceable: FAIL until recruitment is defined in Step 1.\n- Questions diagnostic: PASS (open-ended, unaided first).\n- Sample claims restrained: PASS (qualitative use only).\n- Evidence authentic: PASS. No respondent comments or results exist; none are reported here.\n- Order respected: PASS. Clarity assessed before value, and differentiation flagged as unsupported.\n- Doubts addressed: FAIL. No buyer doubts collected yet.\n- Difference defensible: FAIL. No supported difference for any headline.\n- Claims supported: FAIL for headline 1 (40% unsourced). Pass for the others as long as features are confirmed.\n- Revision focused: PASS. Fix is the category cue and subheading, applied to whichever headline wins.\n- Strategy retained: PASS. No feature requests driving copy.\n- Recommendation justified: PASS as provisional lean only.\n- Retest useful: PASS. Paid test is not repeated for cosmetic edits.\n- Measurement boundary clear: PASS. CTR and message feedback are not presented as conversion lift.\n- Maintenance scoped: PASS. Revisit after Step 1 or once the first demo-request data exists.\n"}]}, "opus-1": {"cost_usd": 0.23934, "seconds": 86, "output_tokens": 6969, "files": [{"name": "value-prop-test-plan.md", "text": "# Clinicflow: choosing the value proposition\n\n**Short version:** Don't spend the whole EUR 2,000 on the agency's three-way Google Ads test. At that budget each headline gets roughly 130\u2013330 clicks, which works out to about 4\u201310 demo requests per headline. That's too few to tell the headlines apart. You already have two better sources of evidence, and they cost almost nothing: **80 customers who can tell you why they bought**, and **4,000 clinic owners you can email-test with 4\u201310\u00d7 as many people per headline**. Use those first. Then spend a smaller ad budget to confirm the winner with real search traffic.\n\n---\n\n## 1. What the three messages are really testing\n\nEach headline speaks to a different pain, and possibly to a different person at the clinic:\n\n| # | Headline | Pain it addresses | Who feels it most |\n|---|----------|-------------------|-------------------|\n| 1 | \"Cut no-shows by 40%.\" | Lost revenue from empty slots | Owner |\n| 2 | \"Your front desk, without the phone tag.\" | Admin workload, staff time | Front-desk / practice manager |\n| 3 | \"Intake that's ready before the patient walks in.\" | Smooth first visit, therapist prep | Treating physio, patient |\n\nSo the real question isn't which sentence wins. It's **which pain is strongest for the person who signs the contract.** That's a research question first and a copy question second. Once you know the pain, you can still polish the wording.\n\n## 2. Why the Google Ads plan as proposed won't give a clear answer\n\n**Not enough data.** EUR 2,000 split three ways is about EUR 667 per headline. Here's what that buys at typical B2B software CPCs (please plug in your real CPCs from the agency):\n\n| CPC | Clicks per headline | Demo requests per headline (at 3% landing-page conversion) |\n|-----|---------------------|-------------------------------------------------|\n| EUR 2 | ~333 | ~10 |\n| EUR 3 | ~222 | ~7 |\n| EUR 5 | ~133 | ~4 |\n\nTo reliably detect a lift from 3% to 4.5% conversion (that's already a big 50% relative difference, at 80% power and 95% confidence), you need **about 2,500 visitors per headline**. That's 7\u201319\u00d7 more than the budget buys. With 4\u201310 conversions per arm, the \"winner\" will mostly be noise.\n\n**Other problems:**\n- **CTR is the wrong metric.** If the agency judges on click-through rate, which is the only metric with enough volume, it will reward the most clickable headline, not the one that sells. A specific number like \"40%\" tends to win clicks regardless of buying intent.\n- **Google won't split traffic evenly by default.** Ad rotation has to be set to \"Do not optimize: rotate ads indefinitely,\" and each headline needs its own ad with everything else identical. Otherwise the algorithm picks a winner early, based on very little data.\n- **Search volume may run out first.** \"Physiotherapy practice software\"-type keywords in one country may not have enough searches in two weeks to spend EUR 2,000 at all. Ask the agency for the Keyword Planner estimate before committing.\n- **Searchers aren't just your buyers.** Students, patients and software competitors also click.\n\n**Before you use headline 1 anywhere:** \"Cut no-shows by 40%\" is a specific, measurable claim. In the EU, comparative and quantified advertising claims must be substantiated (Unfair Commercial Practices Directive and national laws such as Germany's UWG). Can you prove it from your own customer data? If not, it can't run as written, no matter how well it tests. Check this first (step 1 below).\n\n## 3. Recommended plan (about 4 weeks, same EUR 2,000)\n\n### Step 1: Check the numbers you already have (week 1, EUR 0)\nPull no-show rates from product data for customers who use automatic reminders, before vs. after they adopted Clinicflow.\n- If you see roughly 40% or better across a reasonable number of clinics \u2192 headline 1 can be substantiated. Keep the method on file.\n- If the real figure is lower, rewrite with the true number (e.g. \"Cut no-shows by 25%\") or drop the number (\"Fewer no-shows, without extra calls\").\n- Also check how often each feature is actually used: forms, insurance checks, reminders. Heavy reminder use backs up message 1. Heavy form use backs up message 3.\n\n### Step 2: Survey and interview existing customers (weeks 1\u20132, about EUR 600)\nSend the survey in Appendix A to all 80 customers, and offer a EUR 15 voucher for completing it. At a 40\u201350% response rate that's about 35\u201340 replies, roughly EUR 600. Then do **6\u20138 short calls (20 min)** with respondents who are owners.\n\nCustomers tell you **why people buy**. The key rule: ask about pains and reasons for buying **before** showing them the three headlines, so the wording doesn't steer their answers.\n\nThe limitation: existing customers already bought, so they over-represent whatever pain your current sales pitch emphasises. That's why step 3 tests with prospects.\n\n### Step 3: Email A/B/C test to the 4,000 clinic owners (week 3, EUR 0)\n**Check consent first.** Under GDPR/ePrivacy, B2B marketing email to people in the EU generally needs opt-in consent or a legitimate existing relationship, and the rules vary by country. Only email contacts you're allowed to email.\n\nDesign:\n- Randomly split the list into 3 groups of about 1,333 people.\n- Each email is identical except the **subject line and the opening headline**, which use messages 1, 2 or 3.\n- Every email has the same CTA, linking to a landing page variant whose headline matches the email.\n- **Primary metric:** demo requests or trial signups per person emailed. **Secondary:** unique clicks per person emailed.\n- **Don't use open rates.** Apple Mail Privacy Protection inflates them, so they're unreliable.\n- Send at the same time to all groups (e.g. Tuesday, 9:00 local time).\n\nWhat this can detect: with about 1,333 recipients per arm, a click rate of 2% vs. 4% shows up reliably (needs about 1,140 per arm). Smaller differences, such as 2% vs. 3%, won't be distinguishable. If nothing clearly wins, treat that as a real result: the headline matters less than you thought, so pick based on step 2.\n\n### Step 4: Confirm the winner with Google Ads (week 4+, about EUR 1,400)\nRun the agency test with **two arms**: the step 2/3 winner vs. the runner-up, or vs. your current homepage headline. Judge them on landing-page conversion, not CTR. Two arms at EUR 1,400 still give each headline slightly more budget (EUR 700) than the original three-way split (EUR 667). This step is mainly a sanity check that the message works on cold search traffic. Don't expect a statistically clean result from it.\n\n### Budget summary\n| Item | EUR |\n|------|-----|\n| Survey incentives (about 40 \u00d7 EUR 15) | 600 |\n| Customer interviews | 0 (your time) |\n| Email test | 0 (existing tool) |\n| Google Ads confirmation, 2 arms | 1,400 |\n| **Total** | **2,000** |\n\n## 4. Decision rules (agree on these before seeing results)\n\n1. **If headline 1 can't be substantiated, it's out** or gets rewritten with the true number.\n2. **If the survey and the email test agree**, pick that message.\n3. **If the email test shows no clear winner**, go with the pain customers rank highest as their reason for buying (Appendix A, Q2\u2013Q3).\n4. **If they disagree**, prefer the email test for the *headline* (prospects' reaction) and use the survey winner as the *subheadline*.\n5. Whatever wins, the page can still mention the other two benefits below the fold. The headline leads; it doesn't have to cover everything.\n\n## 5. Questions to send the agency\n- What CPCs and weekly search volume do you expect for our keywords in [country]? Can we actually spend EUR 2,000 in two weeks?\n- What's the success metric: CTR, or landing-page conversions? How many conversions per arm do you expect?\n- Will you set ad rotation to \"rotate indefinitely\" and keep everything except the headline identical?\n- Would you run a two-arm test after we've narrowed things down with our own email test?\n\n---\n\n## Appendix A: Customer survey (5 minutes)\n\n1. What's your role? (Owner / practice manager / physiotherapist / front desk / other)\n2. Before Clinicflow, what was the **most frustrating** part of patient intake and scheduling? *(open text, asked before any options are shown)*\n3. What was the **main reason** you chose Clinicflow? *(open text)*\n4. Rank these from most to least important to you:\n   - Fewer no-shows / missed appointments\n   - Less time on the phone and admin for front-desk staff\n   - Patient information complete before the first appointment\n   - Insurance checked before the visit\n5. Since using Clinicflow, roughly how has your no-show rate changed? (Gone up / about the same / down a little / down a lot / don't know)\n6. If you recommended Clinicflow to another clinic owner in one sentence, what would you say? *(open text, often the best headline source)*\n7. *(Only now show the three headlines.)* Which of these best describes what Clinicflow does for you? Why?\n8. May we contact you for a 20-minute follow-up call? (Yes / No)\n\n**How to read it:** Q2, Q3 and Q6 are the unprompted signal; code the answers into the three pains. Q4 is the prompted ranking. Q7 is a tiebreaker only, because people pick the wording that sounds nicest, not the one that would have made them buy.\n\n## Appendix B: Email test checklist\n- [ ] Consent and legal basis confirmed for every contact emailed\n- [ ] Random split (not alphabetical or by sign-up date)\n- [ ] Identical sender, send time, body, CTA and design. Only subject and headline differ\n- [ ] A landing page variant per headline, with tracking (UTM `utm_content=vp1|vp2|vp3`)\n- [ ] Count conversions for 7 days after sending\n- [ ] Report: recipients, unique clicks, demo requests per arm. Ignore opens\n"}]}, "opus-2": {"cost_usd": 0.16655, "seconds": 62, "output_tokens": 5301, "files": [{"name": "value-prop-test-plan.md", "text": "# Clinicflow: Choosing the Value Proposition\n\n**Recommendation in one line:** Don't spend the \u20ac2,000 on the agency's two-week Google Ads A/B test. At that budget it can't produce a reliable winner. Instead, use the 80 customers and the 4,000-clinic email list. Together they can answer the question in about the same two weeks for almost no money. Keep most of the budget for a later test that has a single job: confirming the winner.\n\n---\n\n## 1. Why the Google Ads test won't settle it\n\n**There isn't enough traffic.** Searches for \"patient intake software\" from private physio clinic owners are a small niche. The figures below are assumptions, so check them against the agency's own keyword volume and CPC estimates. Even so, they show the scale of the problem:\n\n| | Assumption | Result |\n|---|---|---|\n| Budget | \u20ac2,000 | |\n| Cost per click (B2B health software) | ~\u20ac3\u20136 | **~330\u2013670 clicks total**, or ~110\u2013220 per headline |\n| Click \u2192 demo/trial conversion | ~3% | **~3\u20137 conversions per headline** |\n\nHow much traffic would a real answer take (80% power, 5% significance)?\n- **CTR 4% vs. 5%:** ~6,700 impressions *per headline*. That might be reachable, but CTR shows which headline gets a click. It doesn't show which one sells.\n- **Conversion 3% vs. 4.5%** (a big 50% lift): ~2,500 clicks *per headline*. That's about 10\u00d7 what the budget buys.\n\nWith 3\u20137 conversions per arm, any \"winner\" after two weeks will be noise.\n\n**It also measures the wrong thing.**\n- The people who see search ads already typed something like \"intake software.\" Headline 3 matches that query, so it will probably win on CTR whether or not it's the strongest message. Clinic owners whose main pain is no-shows or phone admin mostly aren't running that search.\n- Google's responsive search ads and automated bidding mix headlines and shift traffic between them. A clean three-way test needs careful setup, such as pinned headlines or separate ad groups with equal budgets. Ask the agency how they would control for this.\n\n## 2. Problems to fix whatever test you run\n\n1. **\"Cut no-shows by 40%\" needs evidence before it goes live.** A specific number in an ad has to be backed up. EU and national rules on misleading commercial practices apply, and health-sector advertising is under extra scrutiny in several EU countries. Where does the 40% come from? Clinicflow's own reminder and attendance data across the 80 customers, comparing before and after onboarding, can confirm it, correct it, or kill it. If the data supports a different number, use that number.\n2. **Each headline speaks to a different buyer and pain:**\n   - (1) No-shows: the **owner's revenue**\n   - (2) Phone tag: the **front-desk workload**\n   - (3) Intake ready: **clinic flow / clinician time and patient experience**\n\n   So the real question is: *which pain do clinic owners rank first, and who signs the contract?* Answer that, and the headline mostly follows.\n3. **\"Proof\" and \"promise\" can be split.** The winning pain can lead the headline, and a validated number can appear as a proof point underneath it. For example, a \"phone tag\" headline with \"\u2026and 40% fewer no-shows\" as the subhead. You don't have to choose only one.\n\n## 3. Recommended plan (2 weeks, ~\u20ac300\u2013500)\n\n### Step A: Validate the claim from product data (days 1\u20133, \u20ac0)\n- Pull no-show rates for customers who have at least ~3 months of data before and after turning on reminders. Report the median and the range, not the best case.\n- Decision: if the median reduction is \u2265 ~30% and holds across most clinics, the number can be used (rounded down honestly). If not, drop the number or replace it.\n\n### Step B: Survey the 80 customers (days 1\u201310, \u20ac0\u2013400)\nShort survey (5 questions, below) plus **5\u20138 follow-up calls of 20 minutes**. A small thank-you, such as a \u20ac25\u201350 voucher, helps response rates. Expect 25\u201340 responses.\n\nWhat you learn: why they actually bought, what they'd miss most, and which headline they'd forward to a peer. These are people who already said yes, so their answers show which message closes deals.\n\n### Step C: Email \"pain poll\" to the 4,000 clinic owners (days 3\u201314, \u20ac0)\n**First, confirm you have GDPR-compliant consent to email this list for marketing/research.** If consent is unclear, check with whoever owns the list before sending.\n\n- One short email with a single question and **three one-click answers**, one for each pain (draft below). The order of the answers is randomized across three equal batches of the list, which cancels out position bias.\n- Each click lands on a short page in the matching message, with a soft CTA (\"See how it works\" / \"Book a 15-min demo\").\n- **Measure:** click share per pain (the main measure) and demo requests per landing page (secondary).\n- Expected volume at a 5\u20138% click rate: **~200\u2013320 responses**. That is enough to detect a clear leader, e.g. 45% vs. 30% share. Don't use open rates: Apple Mail privacy features inflate them.\n\n### Step D: Decide (day 14)\n| Signal | Weight |\n|---|---|\n| Email pain-poll share (Step C) | High: largest sample, made up of prospects |\n| Customer survey: \"main reason you bought\" + \"headline to forward\" (Step B) | High: these people actually bought |\n| Demo requests per landing page (Step C) | Medium: small numbers, but the strongest signal of intent |\n| Claim validation (Step A) | Gate: controls whether headline 1 can be used at all |\n\n**Decision rule (agree on this before seeing the data):** pick the pain that leads in both the email poll and the customer survey. If they disagree, weight what customers say they bought for. They are the proven buyers, and the poll only shows what prospects say they care about.\n\n### Step E: Then use the ad budget (weeks 3\u20136, ~\u20ac1,500)\nRun the winning message against **one** challenger. Two arms instead of three doubles the traffic per arm. Optimize for landing-page conversion (demo requests), not CTR. Treat this as confirming the winner and setting up the channel, not as the main decision.\n\n## 4. Budget\n| Item | Cost |\n|---|---|\n| Product-data analysis | \u20ac0 (internal) |\n| Customer survey thank-yous (~30 \u00d7 ~\u20ac10\u201315, or vouchers for call participants) | ~\u20ac300\u2013400 |\n| Email poll + 3 simple landing pages | \u20ac0\u2013100 |\n| **Reserve for confirmatory ads test** | **~\u20ac1,500** |\n\n---\n\n## Appendix 1: Customer survey (80 customers)\n\n1. What was the main problem that made you look for something like Clinicflow? *(open text)*\n2. Which of these has improved most since you started? *(pick one)*: Fewer no-shows / Less phone and admin work at the front desk / Patients arrive with intake done / Something else: ___\n3. Roughly how has your no-show rate changed? *(Up / Same / Down a little / Down a lot / Don't know)*\n4. If you recommended Clinicflow to another clinic owner, which line would you use? *(randomize order)*\n   - \"Cut no-shows by 40%.\"\n   - \"Your front desk, without the phone tag.\"\n   - \"Intake that's ready before the patient walks in.\"\n5. Who at your clinic was most involved in choosing Clinicflow? *(Owner / Practice manager / Front desk / Physio / Other)*\n\nOptional: \"Can we call you for 20 minutes?\" (yes/no)\n\n## Appendix 2: Pain-poll email draft (4,000 list)\n\n> **Subject:** Quick question about your clinic (one click)\n>\n> Hi {first_name},\n>\n> We're building tools for private physio clinics and want to focus on what actually costs you the most. Which of these would you fix first?\n>\n> **[ No-shows and late cancellations ]**\n> **[ Endless phone calls at the front desk ]**\n> **[ Paperwork and intake eating into appointment time ]**\n>\n> One click is all it takes. Thanks!\n> {sender}, Clinicflow\n\n*(Rotate the order of the three buttons across three equal batches. Use the CTA text above as the poll answers. The headlines themselves appear on the landing pages, one per pain.)*\n\n## Appendix 3: What to tell the agency\n- Ask for their keyword-volume and CPC estimates. If they project \u2265 ~2,500 clicks per arm for \u20ac2,000, the plan can be revisited (that's unlikely in this niche).\n- Ask how they'd prevent Google's automation from skewing traffic toward one headline.\n- Propose bringing them in for the confirmatory two-arm test in weeks 3\u20136, optimized for conversions.\n"}]}, "opus-3": {"cost_usd": 0.2014, "seconds": 84, "output_tokens": 6561, "files": [{"name": "value-prop-test-plan.md", "text": "# Clinicflow: How to choose the homepage value proposition\n\n## Summary\n\n**Don't spend the \u20ac2,000 on a two-week, three-way Google Ads headline test.** At this budget it probably won't produce a result you can trust. It also measures the wrong thing: which line gets a click from whoever happens to search, not which promise makes a clinic owner want to buy.\n\nYou already have two better sources of evidence, and they cost almost nothing:\n\n1. **80 customers.** They can tell you which problem Clinicflow actually solved for them and whether the \"40%\" figure holds up.\n2. **4,000 clinic owners on your email list.** These are your real buyers, and you can split them into test groups.\n\n**Recommended sequence (about 3\u20134 weeks, under \u20ac1,000 spent):**\n\n| Step | What | When | Cost |\n|---|---|---|---|\n| 1 | Check the \"40%\" claim against your own product data | Week 1 | \u20ac0 |\n| 2 | Survey the 80 customers, then interview 6\u20138 of them | Weeks 1\u20132 | ~\u20ac300\u2013400 in incentives |\n| 3 | Head-to-head email test of the top 2 messages on the 4,000 list | Weeks 2\u20133 | \u20ac0 (existing email tool) |\n| 4 | *Optional:* a small Google Ads check of the winner vs. the runner-up | Week 4 | \u2264 \u20ac1,000 |\n| \u2014 | Keep in reserve for landing page iteration | \u2014 | ~\u20ac600+ |\n\n---\n\n## 1. Problems with the proposed Ads test\n\n**a) The budget is too thin to measure conversions.** Illustrative numbers (ask the agency for real Keyword Planner CPCs and volumes for your markets):\n\n- \u20ac2,000 \u00f7 3 headlines \u2248 \u20ac667 per headline\n- At an assumed \u20ac2\u20134 CPC that buys ~170\u2013330 clicks per headline\n- At a typical 2\u20135% demo/trial conversion rate that's **roughly 3\u201315 conversions per headline**\n\nWith that few conversions, the gap between headlines would come down to chance.\n\n**b) Click-through rate (CTR) is detectable only if the gap is big.** To detect 4% vs. 5% CTR at standard confidence (95% confidence, 80% power), you need **~6,700 impressions per headline**. To detect 4% vs. 6%, you need ~1,900. A niche query like \"physiotherapy intake software\" may not produce that many impressions in two weeks. **Ask the agency for estimated impressions before agreeing to the test.**\n\n**c) Google doesn't split traffic evenly.** Responsive search ads and automated bidding favor whichever variant gets early clicks. Without a strictly controlled setup (separate ad groups, pinned headlines, even rotation), you'd be comparing unequal traffic.\n\n**d) Search ads measure the searcher, not the message.** People searching \"patient reminder software\" will click \"Cut no-shows\" because it matches their query. Headline wins on search mostly reflect keyword choice, and that doesn't tell you what to put on the homepage.\n\n**e) Three variants split a small budget three ways.** Narrowing to two first doubles the traffic each one gets.\n\n## 2. Check each message before testing it\n\n| Message | Strength | Risk to check |\n|---|---|---|\n| **1. \"Cut no-shows by 40%.\"** | Concrete and tied to revenue; clinic owners feel no-shows directly. | **Is 40% true and provable?** Under EU consumer and advertising rules, a specific number must be backed by evidence. Where does it come from: your own customer data, one clinic, or an industry figure? It also covers only the reminders feature, not forms or insurance checks. |\n| **2. \"Your front desk, without the phone tag.\"** | Vivid, and speaks to a daily annoyance. | **Many small physio clinics have no front desk.** Solo practitioners may not see themselves in it. It also speaks to the receptionist's pain, while the owner signs the contract. |\n| **3. \"Intake that's ready before the patient walks in.\"** | Describes the product most accurately and covers forms and insurance checks. | Least emotional of the three, with no outcome or number. It may test poorly cold but convert well once someone is already interested. |\n\nSo the real question is which pain your buyers feel most: **lost revenue** (1), **admin workload** (2), or **a smooth first visit** (3). The customer survey can answer that directly.\n\n## 3. Step-by-step plan\n\n### Step 1: Check the 40% claim (week 1, \u20ac0)\n- Pull no-show rates before and after adoption for customers with at least 3 months of data.\n- If the median reduction is close to 40% across a reasonable number of clinics, keep the figure and write down how you calculated it.\n- If not, change the number to what the data supports (e.g. \"Cut no-shows by a third\") or drop it (\"Fewer no-shows, without the follow-up calls\").\n- **If you can't back up the claim, rewrite message 1 before testing it.**\n\n### Step 2: Survey and interview customers (weeks 1\u20132, ~\u20ac300\u2013400)\nSend a 5-minute survey to all 80 customers. A 30\u201350% response rate is realistic, since they already use the product. Questions are in Appendix A.\n\nThen interview 6\u20138 respondents for 20 minutes each, offering about a \u20ac50 voucher or a month's credit. Include at least 2 solo practitioners and 2 multi-therapist clinics, and listen for the words they use themselves. Customers' own phrasing often makes a better headline than any of the three candidates.\n\n**Output:** a ranked list of the three messages, plus any reworded versions that came out of the interviews. **Take the top two forward.**\n\nCustomers are biased toward what the product already did for them, so use the survey to narrow down. The email test in Step 3 makes the actual decision.\n\n### Step 3: Email A/B test (weeks 2\u20133, \u20ac0)\nThis test reaches the actual buyers, costs nothing, and gives far more volume per variant than the Ads budget would.\n\n- **Split:** randomly assign the 4,000 contacts into 2 groups of about 2,000. Keep everything identical except the subject line and opening headline, which carry the value proposition.\n- **Send:** same day and time for both groups.\n- **Primary metric:** click-through to a landing page, ideally one landing page per variant with the matching headline.\n- **Secondary metric:** demo bookings or trial sign-ups from the landing page. They're likely too few to decide on, but they're a useful directional check.\n- **Ignore open rate.** Apple Mail Privacy Protection inflates it, so it isn't reliable.\n\n**What it can detect:** with ~2,000 per group, a 3% vs. 5% click-through difference is detectable (that needs ~1,500 per group). Smaller differences such as 3% vs. 4% are not. **If the gap is small, there's no meaningful winner, so go with the survey result and the stronger claim.**\n\n**Why two variants instead of three:** a three-way split of the list gives ~1,333 per group, which is below what's needed to detect even a 3% \u2192 5% difference.\n\n### Step 4 (optional): A small Ads check (week 4, \u2264 \u20ac1,000)\nUse this only if the agency's estimates show enough search volume. If you run it:\n- Test **winner vs. runner-up only**, in separate ad groups with pinned headlines and even rotation.\n- Send both to matching landing pages and track demo or trial conversions, not just CTR.\n- Treat it as a check on the email result, not a separate decision.\n\n## 4. Decision rules (agree on these before seeing results)\n\n1. **If the 40% claim can't be backed up**, rewrite message 1 or drop it from testing.\n2. **If the email test shows a clear click-through gap** (95% confidence), use the winner.\n3. **If the email gap isn't significant**, use the survey's top-ranked message.\n4. **If the survey shows a split by clinic type** (e.g. solo practitioners pick 3, larger clinics pick 2), use the stronger overall message on the homepage. Use the other on targeted landing pages or in ads aimed at that segment.\n5. **Whichever wins, use the other messages as supporting lines.** A typical structure is headline = outcome, subhead = how it works, for example:\n   > **Cut no-shows by [X]%.**\n   > Online intake, insurance checks and automatic reminders, so every patient is ready before they walk in.\n\n## 5. Questions for the team / agency\n\n- What are the estimated monthly searches and CPCs for the target keywords, by country?\n- How many clinics' data stands behind \"40%\"?\n- Is the email list opted in to marketing (GDPR)? When was it last emailed? (Affects deliverability.)\n- What's the split between solo and multi-therapist clinics among your customers and on the list?\n- Is there one main conversion on the site (demo booking or free trial)? That should be the metric everywhere.\n\n---\n\n## Appendix A: Customer survey (5 minutes)\n\n1. How many therapists work at your clinic? *(Just me / 2\u20134 / 5\u201310 / 10+)*\n2. Do you have dedicated front-desk or reception staff? *(Yes / No / Part-time)*\n3. Before Clinicflow, which of these cost you the most? *(Rank)*\n   - Patients not showing up\n   - Staff time on calls, reminders and paperwork\n   - Patients arriving without forms or insurance details sorted\n4. What's the single biggest thing that has changed since you started using Clinicflow? *(Open text)*\n5. If you had to describe Clinicflow to another clinic owner in one sentence, what would you say? *(Open text; the most useful source of headline wording)*\n6. Which of these would most make you want to try Clinicflow if you'd never heard of it? *(Pick one; randomize order)*\n   - \"Cut no-shows by 40%.\"\n   - \"Your front desk, without the phone tag.\"\n   - \"Intake that's ready before the patient walks in.\"\n7. Why that one? *(Open text)*\n8. May we contact you for a 20-minute follow-up call? *(Yes/No, with a \u20ac50 voucher as thank-you)*\n\n## Appendix B: Email test template\n\n**Subject A:** Cut no-shows at [Clinic name] by [X]%\n**Subject B:** [Runner-up message, adapted to a subject line]\n\n**Body (identical except the headline line):**\n> Hi [First name],\n>\n> **[Headline A / B]**\n>\n> Clinicflow handles online intake forms, insurance checks and automatic appointment reminders for private physio clinics, so your team spends less time chasing and more time treating.\n>\n> [See how it works \u2192] *(links to the variant's matching landing page)*\n\nKeep the sender, timing, length and call to action identical. Only the value proposition changes.\n"}]}, "haiku-1": {"cost_usd": 0.00686, "seconds": 38, "output_tokens": 6912, "files": [{"name": "value-prop-recommendation.md", "text": "# Clinicflow: choosing the lead value proposition\n\n## Recommendation\n\nDo not spend the EUR 2,000 on a two-week, three-way Google Ads headline test as planned. With that budget it cannot reliably tell the headlines apart, so it will likely end \"inconclusive\" or crown a false winner.\n\nInstead, use the cheap assets first and reserve the ad budget for a narrower test:\n\n1. **Survey the 80 customers (about EUR 0).** Ask what problem costs them most, and for any no-show reduction they have measured since starting with Clinicflow. This decides whether the 40% claim is usable.\n2. **Test the headlines in email to the 4,000 owners (about EUR 0 beyond the email tool).** Use click-through as the metric. Send to the top two only if you want more power per arm.\n3. **Spend the ad budget on the top two headlines, measured by click-through rate.** Use it to confirm the email winner, not to pick from three.\n\nProvisional lead: **\"Cut no-shows by 40%.\"** It is the only candidate that names a result owners can measure and pay for. Use it only if the customer survey backs a number of about that size. If it does not, lead with **\"Intake that's ready before the patient walks in.\"** That claim is concrete and provable from the product itself (share of intakes completed before arrival). Treat **\"Your front desk, without the phone tag.\"** as a strong secondary angle for the front-desk staff who influence the purchase.\n\n## Why the agency's plan is underpowered\n\nAssumptions (replace with real account data): CPC EUR 2.00, search CTR 5%, landing-page signup rate 3\u20135%, email open rate 25%, email click rate 2.5%.\n\n- **Budget buys about 1,000 clicks**, roughly 333 per headline across three arms.\n- To detect a 40% relative lift in signup rate at 80% power, you need roughly 2,200\u20133,800 clicks per arm. You have about 333.\n- Only a very large difference (roughly doubling conversion) would show up. Any \"winner\" at this sample size is mostly noise, and stopping early when one headline looks ahead makes it worse.\n- Click-through rate is a better ad metric because impressions are cheaper than clicks. At the assumed numbers, about 6,700 impressions per arm detects a roughly 30% CTR lift, which is more realistic. But CTR measures attractiveness, not whether the headline produces signups or fewer no-shows. Check the winner against landing-page signups afterward.\n- Email to 4,000 owners gives about 1,333 per arm. That detects only large differences (roughly 80% higher click rate). It works as a filter for clearly weak headlines, not as a precise ranking.\n\n## Why the claim needs checking first\n\n\"Cut no-shows by 40%\" is the most persuasive line, but it is also the one that can get you in trouble. If the number is not backed by customer data, it is an unsubstantiated claim in paid advertising. Confirm it with real before-and-after no-show data from customers, and check the wording with counsel before running it. If the true effect is 15%, say so.\n\n## Survey questions for the 80 customers\n\n- What is the biggest daily problem the front desk or intake handles?\n- Since using Clinicflow, how has your no-show rate changed? Do you have numbers?\n- Which of these would make you more likely to recommend us to a peer clinic? (Show all three headlines.)\n- What would you have said to a clinic owner considering us, in your own words?\n\n## Decision rule\n\n- Pick the headline with the strongest customer evidence and the best email click-through, as long as it is not clearly behind on CTR in the ads.\n- Pre-register the metric (CTR for ads, click rate for email) and the decision threshold before launch. Do not stop early.\n- If the survey does not separate the headlines, go with the claim you can prove from the product. That is \"Intake that's ready before the patient walks in.\"\n\n## Open items\n\n- Real CPC, CTR, and signup-rate numbers for the target keywords, to replace the assumptions above.\n- Whether Clinicflow has customer no-show data (before and after) that can substantiate any percentage claim.\n- Agency's proposed headline set and metrics, so the test design can be aligned with this plan.\n"}]}, "haiku-2": {"cost_usd": 0.00675, "seconds": 36, "output_tokens": 6734, "files": [{"name": "value-prop-recommendation.md", "text": "# Clinicflow: which value proposition for the new site\n\n## Recommendation\n\nDon't decide this with a two-week, EUR 2,000 Google Ads test. The budget can't separate three headlines. Use the cheaper sources first, in this order:\n\n1. **Check the no-show number (this week).** Pull no-show rates for the 80 customers before and after they started using Clinicflow. This decides whether headline 1 can be used at all.\n2. **Survey the 80 customers (1\u20132 weeks).** Ask them which of the three statements describes why they bought, and what changed for them. Buyers are the best source for the message that actually sells.\n3. **Test subject lines on the 4,000-owner list (about 1 week).** Split the list three ways, one headline per group, and measure opens and clicks through to a demo page. This is the only test in the brief that has enough volume to read.\n4. **Keep the Google Ads budget for the finalists.** Run the top two headlines only, if the first three steps leave a tie.\n\nWorking default if the data doesn't settle it: **headline 3, \"Intake that's ready before the patient walks in.\"** It names the product's core job, makes a concrete promise, and doesn't rely on a number we haven't verified. Headline 1 becomes the lead only if the customer data supports a figure of that size. Headline 2 is the fallback if owners tell us the front desk workload is the real pain.\n\n## Why not the Google Ads test as briefed\n\nAssumptions below are rough. Replace them with the agency's keyword and CPC data before relying on them.\n\n| Assumed CPC | Total clicks on EUR 2,000 | Clicks per headline | Conversions per headline (at 3%) |\n|---|---|---|---|\n| EUR 2 | ~1,000 | ~333 | ~10 |\n| EUR 3 | ~670 | ~222 | ~7 |\n| EUR 5 | ~400 | ~133 | ~4 |\n\n- **Click-through rate:** detecting 4% vs 5% CTR at 80% power needs about 6,700 impressions per headline. Detecting 4% vs 6% needs about 1,900. Small differences between the three headlines will not show up.\n- **Signups:** detecting 3% vs 5% conversion needs about 1,500 clicks per headline. The budget gives roughly 130\u2013330.\n- **Ad rotation:** headlines inside one responsive ad aren't a clean A/B test. They have to sit in separate ad groups to be compared fairly.\n- **Agency fee:** the brief doesn't say whether it comes out of the EUR 2,000. If it does, the test is smaller still.\n\n## Email test design\n\n- **Sample:** 4,000 owners, about 1,333 per headline. That is enough to detect a 25% vs 30% open rate at 80% power (about 1,250 per arm needed).\n- **Primary metric:** opens. Secondary: clicks through to a demo or signup page.\n- **Consent check first.** Confirm the list was collected with a lawful basis for marketing email. For EU B2B recipients this is often an opt-in requirement, and Germany is especially strict about cold email. Don't send to the list until this is confirmed.\n\n## Survey design\n\n- Show the three statements in random order. Ask: \"Which of these best describes why you chose Clinicflow?\" and \"Which would you have been most likely to click on?\"\n- Add one open question: \"What changed in your clinic after you started?\" Use the answers to write the site copy.\n- Caveat: current customers may differ from prospects. Treat the result as strong input, not a verdict.\n\n## Open issues\n\n1. **The 40% claim is unsupported in the brief.** Headline 1 should not go live, in ads or on the site, unless we have measured data behind it. An unverified number in a consumer-facing claim is a legal and reputational risk in the EU. If the data shows a different figure, use that figure.\n2. **Market and language** are not stated. The copy and the consent rules both depend on this.\n3. **Agency scope.** Ask whether they will run the email and survey work, or whether the ad test is their only proposal. If they insist on the ad test, ask for a plan based on the CPC and conversion data above.\n\n## Decision rule\n\n- If the no-show data shows a reduction the team can defend, lead with headline 1 and use the measured figure, not 40%, unless the data actually shows 40%.\n- Otherwise, if the survey and email tests point to one headline, use it.\n- If they are close, run Google Ads on the top two only, at about EUR 800\u20131,000 each, and treat the result as directional.\n"}]}, "haiku-3": {"cost_usd": 0.00575, "seconds": 31, "output_tokens": 5675, "files": [{"name": "value-prop-recommendation.md", "text": "# Clinicflow: choosing the launch value proposition\n\n## Recommendation\n\nDon't spend the \u20ac2,000 on a two-week Google Ads test of the three headlines. The budget can't separate them, and the test would tell you little. Test the headlines in the 4,000-person email list and survey the 80 customers first.\n\nProvisional lean: headline 1 (\"Cut no-shows by 40%.\") if the customer data supports the number. If it doesn't, headline 2 (\"Your front desk, without the phone tag.\") is the safer lead. Headline 3 is the weakest: it describes a process outcome, not something a clinic owner loses money or time on.\n\n## Why not Google Ads\n\nAssumptions (replace with real numbers from the agency): CPC \u20ac2, landing-page conversion (demo or signup) 3%, \u20ac667 per headline.\n\n- Budget per headline: \u20ac667 \u2192 about 330 clicks \u2192 about 10 conversions per arm.\n- Sample size needed per arm to detect a 3% \u2192 4.5% change (50% lift): about 2,500 clicks. To detect a doubling (3% \u2192 6%): about 750 clicks.\n- So even a doubling in conversion rate would be missed with the budget available. Any \"winner\" after two weeks is mostly noise.\n- Google's default ad rotation (\"optimize for conversions\") shifts traffic toward early leaders, which makes the comparison less fair. Use \"rotate indefinitely\" if you do run it.\n- Paid clicks from cold search traffic are a weak proxy for how clinic owners respond to a message.\n\n## Recommended plan\n\n1. **Survey the 80 customers (this week, no cost).** Show the three headlines in random order and ask which one best describes why they bought and what problem it solves. Also ask an open question: \"What problem were you trying to solve when you signed up?\" Their own words are the best copy source. Treat the results as directional: existing customers are a biased sample and may answer leading questions generously.\n\n2. **Check the 40% claim before using it.** Ask whether any customer data shows a no-show reduction and how it was measured (before/after, same clinic, same period, what counts as a no-show). If there's no solid basis, don't use headline 1 as written. Claims about patient outcomes in health marketing need substantiation, and a number you can't defend is a credibility risk with clinic owners.\n\n3. **Run a three-way email test to the 4,000 owners.** Split the list roughly evenly (about 1,330 per arm). Use each headline as the email subject line and the first line of the body, with identical content otherwise.\n   - Sample size check: about 1,250 per arm detects a change in open rate from 25% to 30%. About 1,140 per arm detects a doubling of click-through from 2% to 4%. This is enough to pick a clear winner, though not small differences.\n   - Measure clicks and demo or trial bookings, not just opens. Opens are inflated by email privacy features and are a weak signal.\n   - Send in a single batch so timing doesn't differ between arms.\n\n4. **Keep most of the \u20ac2,000.** Once the email test has a winner (and the survey agrees), spend a smaller test budget on Google Ads for that headline only, to check that it holds on a second channel. That spend is then a confirmation step, not a comparison.\n\n## Decision rule\n\n- If survey and email favour the same headline, use it.\n- If they disagree, use the email result (larger, more relevant sample) and treat the survey as a tiebreaker for wording.\n- If the email test shows no clear difference, the headlines are roughly equivalent for this audience. Choose the one you can defend most easily (probably headline 2) and move on to improving the product page.\n- Don't use headline 1 unless the 40% figure is substantiated, regardless of how the test comes out.\n\n## Open items\n\n- The agency's CPC, conversion rate, and any historic Google Ads data for physiotherapy software in the target market would change the Google Ads numbers above. Get them before deciding whether to run that test at all.\n- Confirm that the 4,000-owner list is opted in for marketing email in the relevant jurisdictions and that the claims used comply with local advertising rules.\n"}]}}, "verdicts": [{"vs": "opus", "pair": "1", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "haiku+skill", "why": "A gives a provisional headline and tests prospect comprehension while separating customer evidence. B critiques advertising better, but overstates expected evidence, risks exceeding the incentive budget, and mistakes inconclusive results for evidence that headlines matter less."}, {"vs": "opus", "pair": "1", "judge": "gpt", "order": "opus shown first", "winner": "haiku+skill", "why": "B offers a concrete provisional headline and more carefully separates assumptions from evidence. A provides stronger advertising analysis but never recommends a headline and overstates numerical expectations. Both miss the front-desk capability implication and lack complete final decision rules."}, {"vs": "opus", "pair": "1", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "tie", "why": "Both provide clear, actionable, and factually accurate plans using the available resources. They are equally excellent."}, {"vs": "opus", "pair": "1", "judge": "deepseek", "order": "opus shown first", "winner": "tie", "why": "Both provide clear, actionable plans using the resources, accurately assess the messages, and critique the ad test. They are equally excellent."}, {"vs": "opus", "pair": "3", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "haiku+skill", "why": "A recommends a credible provisional headline and prioritizes comprehension over clicks. Both miss phone-tag capability implications. B declares winners from clicks or customer preference, incorrectly says narrowing three variants to two doubles traffic, and understates its maximum planned spending."}, {"vs": "opus", "pair": "3", "judge": "gpt", "order": "opus shown first", "winner": "haiku+skill", "why": "B recommends a provisional headline and prioritizes comprehension, relevance and credibility. A selects winners by clicks or preference and incorrectly says reducing three variants to two doubles traffic. Both miss the broader telephone capabilities implied by headline 2."}, {"vs": "opus", "pair": "3", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "tie", "why": "Both provide clear, actionable plans using the resources, accurately assess messages, and set valid decision rules."}, {"vs": "opus", "pair": "3", "judge": "deepseek", "order": "opus shown first", "winner": "opus", "why": "A provides a more concrete, actionable plan using the available audiences and budget, while B's research plan is less feasible."}, {"vs": "opus", "pair": "2", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "tie", "why": "A gives a usable default but lets clicks and customer preferences dictate the winner. B provides stronger research steps but no provisional headline and incorrectly claims two arms double traffic. Both miss capability overpromises and adequate comprehension checks."}, {"vs": "opus", "pair": "2", "judge": "gpt", "order": "opus shown first", "winner": "opus", "why": "A provides stronger audience research, budgeting, and advertising-test critique. Both miss capability overpromising and rely excessively on preference or clicks. A incorrectly says two arms double traffic; B inconsistently permits the 40% claim and supplies unsupported email-power estimates."}, {"vs": "opus", "pair": "2", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "tie", "why": "Both provide clear, actionable plans using available resources, accurately assess messages, and define decision rules. They are equally excellent."}, {"vs": "opus", "pair": "2", "judge": "deepseek", "order": "opus shown first", "winner": "opus", "why": "A provides a more complete, actionable research plan using all audiences and budget, while B's email test design is vague and underdeveloped."}, {"vs": "haiku", "pair": "1", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "haiku+skill", "why": "A offers stronger comprehension research, separates customer and prospect evidence, and prioritizes demo requests. B ultimately selects by clicks and overstates statistical certainty. Both miss the broader telephone capabilities implied by the front-desk headline."}, {"vs": "haiku", "pair": "1", "judge": "gpt", "order": "haiku shown first", "winner": "haiku+skill", "why": "B separates customer proof from prospect relevance, tests comprehension fairly, and reserves ads for demo requests. A selects primarily on clicks and overstates inevitable underpowering. Both miss the phone-tag headline\u2019s broader capability implications; B\u2019s final decision rules remain incomplete."}, {"vs": "haiku", "pair": "1", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "haiku+skill", "why": "A provides a more thorough, actionable plan with clear decision rules, while B makes arithmetic errors and under-specifies research steps."}, {"vs": "haiku", "pair": "1", "judge": "deepseek", "order": "haiku shown first", "winner": "tie", "why": "Both provide clear, actionable recommendations, accurately assess messages and resources, and propose feasible research plans with sound decision rules."}, {"vs": "haiku", "pair": "2", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "haiku+skill", "why": "A better critiques ad clicks and handles inconclusive results. B makes opens primary and overstates email sufficiency. Both miss capability checks and qualified prospect decision criteria; A inconsistently allows 30\u201340% evidence to support a 40% claim."}, {"vs": "haiku", "pair": "2", "judge": "gpt", "order": "haiku shown first", "winner": "haiku", "why": "A handles the unsupported 40% claim more consistently; B's 30\u201340% viability threshold contradicts its recommendation. Both inadequately assess product capabilities and prospect comprehension, and neither grounds the final decision in qualified buying actions."}, {"vs": "haiku", "pair": "2", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "tie", "why": "Both provide clear, actionable recommendations with accurate, feasible plans and critical assessment of the proposed test."}, {"vs": "haiku", "pair": "2", "judge": "deepseek", "order": "haiku shown first", "winner": "tie", "why": "Both provide clear, actionable recommendations, accurate assessments, and feasible research plans using the given resources."}, {"vs": "haiku", "pair": "3", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "haiku+skill", "why": "A offers a credible provisional intake message and better evidence boundaries, though it misses phone-tag overpromising. B recommends that unsupported promise, overstates test limitations, and wrongly treats inconclusive email results as equivalence. Both sample-size calculations are approximately correct."}, {"vs": "haiku", "pair": "3", "judge": "gpt", "order": "haiku shown first", "winner": "haiku+skill", "why": "B gives a credible provisional recommendation and separates comprehension from conversion evidence. Both miss phone-tag capability overreach. A overstates statistical certainty and equates inconclusive results with equivalence; B lacks concrete budget allocation and a complete conversion-based decision rule."}, {"vs": "haiku", "pair": "3", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "haiku+skill", "why": "A provides a more thorough and nuanced assessment of each message against the product features and a superior research plan."}, {"vs": "haiku", "pair": "3", "judge": "deepseek", "order": "haiku shown first", "winner": "tie", "why": "Both provide clear, actionable recommendations, accurate assessments, and feasible research plans using the available resources."}]}, {"id": "msg-3", "family": "message-testing", "prompt": "We need to pick our positioning statement before a big conference. Options in options.md. How should we test them and which wins?", "input": "options.md", "skill": "laja-wynter-message-testing", "input_text": "Product: Gridline, an energy-management platform for commercial buildings (connects HVAC and meters, shifts load to cheap hours).\nCandidates:\n- \"Lower your building's energy bill by 15% without new hardware.\"\n- \"Turn your building into a flexible asset on the grid.\"\n- \"Hit your 2030 emissions targets with the building you already have.\"\nAudience at the conference: facility managers and sustainability leads at real-estate firms.\nTimeline: 3 weeks. We can reach 300 past demo-requesters by email and get 15 customer calls.\n", "rubric": [{"id": "c1", "criterion": "Ground the recommendation in Gridline\u2019s product and the conference audience.", "full_marks_when": "Connects HVAC and meter integration and shifting load to cheaper hours to the needs of facility managers and sustainability leads at real-estate firms. Explains how the three statements differ in relevance and clarity for those audiences.", "zero_when": "Gives generic positioning advice, invents product capabilities, or ignores either audience."}, {"id": "c2", "criterion": "Provide an executable test within the available time and access.", "full_marks_when": "Provides a concrete three-week sequence for preparing, testing, analyzing, and selecting the conference message using the 300 reachable past demo-requesters and 15 available customer calls. Specifies what participants see, what they are asked, and when the decision is made.", "zero_when": "Offers only abstract advice or relies on more time, participants, or research access than the input provides."}, {"id": "c3", "criterion": "Make comparisons fair and acknowledge what the sample can establish.", "full_marks_when": "Keeps presentation, offer, and call to action consistent across messages; controls assignment or presentation order; and examines audience-role differences where feasible. Recognizes that past demo-requesters and customers are a warm sample and that limited responses may not establish a decisive winner.", "zero_when": "Confounds message effects with different offers or audiences, assumes all 300 people will respond, or presents small-sample results as conclusive evidence for the whole conference audience."}, {"id": "c4", "criterion": "Measure understanding, credibility, and useful action rather than popularity alone.", "full_marks_when": "Supplies usable questions or tasks that reveal what participants think Gridline does, whether the promise matters and is believable, and why they would act. Combines this with a meaningful behavioral measure such as qualified demo interest or replies, and defines how the evidence will determine the winner and resolve mixed results.", "zero_when": "Selects a winner solely from favorite-slogan votes, email opens, or unsupported intuition without checking comprehension and credibility."}, {"id": "c5", "criterion": "Recommend a winner while distinguishing a provisional choice from a tested result.", "full_marks_when": "Names a defensible provisional choice from the three candidates, explains its tradeoffs for the two audience roles, and states what findings would confirm or overturn it. Provides a fallback decision if the test is inconclusive, without pretending results already exist.", "zero_when": "Never recommends a choice, declares an empirically proven winner without data, or recommends a message without explaining why it fits this conference."}, {"id": "c6", "criterion": "Preserve input facts, calculate allocations correctly, and scrutinize unsupported promises.", "full_marks_when": "Accurately uses the three-week timeline, 300 email contacts, 15 calls, 15% savings claim, and 2030 emissions target. If proposing equal allocation across three candidates, states 100 email contacts and five calls per candidate. Treats 15% savings, no new hardware, grid flexibility, and achieving emissions targets as promises requiring substantiation rather than verified capabilities; recognizes that shifting load to cheaper hours does not itself prove lower emissions or grid-service eligibility.", "zero_when": "Misstates input facts, makes allocation or other arithmetic errors, invents test results, or treats unverified savings, hardware, grid, or emissions promises as established facts."}], "runs": {"haiku-skill-1": {"cost_usd": 0.01158, "seconds": 63, "output_tokens": 12465, "files": [{"name": "gridline-positioning-test-plan.md", "text": "# Gridline positioning: provisional review and test plan\n\n**Status: provisional expert review. No buyer responses exist yet.** Every finding below is an inference from the three candidate lines and the product description in options.md. Nothing here is buyer-validated, and none of it claims a conversion effect.\n\n## 1. Decision and evidence status\n\n- **Decision:** Which positioning line leads Gridline's conference materials and homepage hero for facility managers and sustainability leads at real-estate firms.\n- **Research question:** Can facility managers and sustainability leads at real-estate firms, shown each line on a neutral page, say what Gridline does, see a priority of their own in it, and find a credible reason to consider it over what they use now?\n- **Desired next action:** Demo request. Assumed from the conference goal; confirm.\n- **Status:** Provisional expert review, with a proposed preference test and a follow-up message test.\n- **Message hypothesis [Inferred]:** Option 1 (cost) is the clearest and most relevant to a budget owner. Option 3 (emissions) is the most relevant to sustainability leads but carries the most claim risk. Option 2 (flexible asset) is the least clear.\n- **Success condition (fixed before any data is seen):** See the decision gate in section 8.\n\n## 2. Buyer context and gaps\n\noptions.md gives only the audience (facility managers and sustainability leads at real-estate firms). Everything else is unknown.\n\n| Field | Current evidence | Source/date | Gap or assumption |\n| --- | --- | --- | --- |\n| Role and budget owner | Two roles named | options.md | Who owns the energy budget versus the emissions target? Unknown. |\n| Priority and desired gain | Not stated | None | Is cost or emissions the live priority right now? |\n| Trigger and success measure | Not stated | None | What makes a team look for this now? |\n| Buying process | Not stated | None | Who signs off, and what do they need to see first? |\n| Current approach and alternatives | Not stated | None | BMS tuning, utility programs, a consultant, or nothing? Needed for the differentiation check. |\n\nThe 15 existing-customer calls are the fastest way to close these gaps. Run them in week 1, before the stimulus is frozen, so answers can change the copy.\n\n## 3. Test record (planned, not completed)\n\n- **Study type:** Preference test across the three lines, then a message test on the finalist. Three viable directions fit a preference test. The message test is needed to diagnose why the finalist wins or fails.\n- **Stimulus:** Gridline's current homepage body, unchanged, with only the hero headline and subheading swapped for each candidate. Record the capture date and version. This tests first impressions and the hero only. It cannot show whether the body explains the offer in time.\n- **Audience, three streams:**\n  - **Stream B (primary):** Cold recruits from a panel, verified as facility managers or sustainability leads at real-estate firms. The panel is not in options.md, so its cost needs confirmation. Target at least 15 per role, with 30 as the practical floor.\n  - **Stream A (secondary):** The 300 past demo-requesters. They already know Gridline, so this stream will overstate familiarity. Report it separately and never pool it with Stream B.\n  - **Stream C:** The 15 customer calls. These are buyer-context and proof checks, not preference votes. Customers already know the product.\n- **Sample reasoning:** These counts are planning guidance for qualitative and preference work. They do not support percentage claims or significance tests.\n- **Questions and order (show each line in randomized order; ask before revealing the intended category):**\n  1. \"What do you think this company offers? Explain it in your own words.\"\n  2. \"Who would use this, and for what situation?\"\n  3. \"Which part, if any, relates to a priority in your work right now? Which feels less relevant?\"\n  4. \"What result would you expect, and how useful would it be?\"\n  5. \"What would you use instead today? What reason, if any, does this page give to choose this?\"\n  6. \"What makes you doubt a claim? What information is missing?\"\n  7. Rate each line 1 to 5 on: clarity of what it offers, relevance to your priorities, value of the result, difference from what you use now. Anchors: 1 = not at all, 5 = completely. Add a one-sentence reason for each rating.\n  8. After all three lines: \"Which would you most want to learn more about, and why?\"\n- **Evidence location:** One response ID per respondent, stored with role, stream, and line order. Not yet created.\n- **Limitations:** Small panels. Stream A is familiar with the brand. The test is hero-only. There is no behavioral measure.\n\n## 4. Findings and ordered diagnosis [Inferred from text only]\n\n| Layer | Option 1: \"Lower your bill 15%\" | Option 2: \"Flexible asset on the grid\" | Option 3: \"Hit your 2030 targets\" |\n| --- | --- | --- | --- |\n| Clarity | Likely pass. Concrete outcome. The category (\"energy management\") is absent but probably inferred. | Likely fail. \"Flexible asset\" and \"on the grid\" are utility jargon. Buyers must guess what the building earns. | Partial. The priority is clear, but nothing says what Gridline does. |\n| Relevance | Strong for the budget owner. | Unclear for either role. | Strongest for sustainability leads. Weak for facility managers. |\n| Value | Concrete, but the 15% is unverified. | Not stated. | \"Hit your targets\" is a large promise with no mechanism. |\n| Differentiation | \"Without new hardware\" may be the only real difference, and it is unverified. | None stated. | \"The building you already have\" hints at the same no-retrofit point. |\n\n**Predicted earliest failed layer:** Option 2 at clarity. Options 1 and 3 are more likely to fail at proof and value than at comprehension. These are predictions to test, not observed results.\n\n## 5. Claim and sameness audit\n\n| Page claim | Buyer question | Alternative buyers use | Supporting proof | Decision |\n| --- | --- | --- | --- | --- |\n| \"Lower your bill 15%\" | Is this real, and will it apply to my building? | Manual BMS tuning, utility programs, or nothing | None in options.md | Verify the source, sample, period, and whether 15% is a mean or median across which building types. If it can't be verified, use a verified range or a named-building case. |\n| \"Without new hardware\" | Do I need to install anything? | Installing submeters or controls | None in options.md | Verify whether integration uses existing HVAC and meters. If false, drop it. |\n| \"Flexible asset on the grid\" | What does my building earn, and how? | Unknown | None | Define the mechanism, such as utility demand-response payments or time-of-use savings, or drop the line. |\n| \"Hit your 2030 emissions targets\" | Will this get us to our target? | Other decarbonization levers | None | options.md says the product shifts load to cheap hours. That is cost-driven. Cheap hours are not necessarily low-carbon. Verify whether the platform can shift by grid carbon intensity and measure emissions. If not, this line overclaims. \"Hit your targets\" also implies sufficiency, which is hard to support. |\n\n## 6. Proposed revision (for the likely winner; not validated)\n\nIf Option 1 wins, the hero stays as written only once the 15% and hardware claims are verified. The subheading can be drawn from the product description in options.md, which the team already knows to be accurate:\n\n- **Hero:** \"Lower your building's energy bill by 15% without new hardware.\" (Claim status: needs verification.)\n- **Subheading:** \"Gridline connects to your existing HVAC and meters and shifts energy use to lower-cost hours.\" (Supported by options.md. It also adds the category cue that is currently missing.)\n\nOption 3 should not be the hero. It can return as a sustainability-track message only after the carbon mechanism is verified. If the results split by role, do not average them. Decide which audience to lead with, and use a role-specific line in track-specific materials.\n\n## 7. Prioritized changes\n\n1. Verify the 15% figure and the \"without new hardware\" claim. **Owner: unassigned.**\n2. Verify whether the platform reduces emissions before any emissions line is used. **Owner: unassigned.**\n3. Define or drop \"flexible asset on the grid.\"\n4. Run the preference test and the 15 customer calls in weeks 1 and 2.\n5. Add the category term to the finalist's subheading.\n\n## 8. Test sequence and recommendation\n\n**Recommendation: gather context before committing to a line.** Lean provisionally toward Option 1, conditional on verification. Do not treat this lean as a finding.\n\n**Decision gate (set now, before data):**\n\n- **Ship Option 1 as the lead** if, in Stream B, a majority of each role paraphrases the offer correctly and names a priority it addresses, and the 15% and hardware claims are verified or replaced with verified wording. Also require that no unanswered doubt about the headline remains.\n- **Lead with a different option** if another line clearly beats Option 1 on clarity and relevance within both roles, and its claims are verified.\n- **Gather more buyer context** if no line passes clarity in Stream B. Rewrite after the calls, then retest.\n- **Do not use Stream A to decide.** Use it only as a familiar-audience check.\n\n**Timeline (3 weeks from 2026-10-08):**\n\n- **Days 1 to 7 (by 2026-10-15):** Verify claims. Run the 15 customer calls. Recruit the Stream B panel. Email the 300 demo-requesters. Freeze the stimulus.\n- **Days 8 to 12 (by 2026-10-19):** Field the preference test. Analyze by role and stream. If a role has fewer than 15 responses, extend panel recruiting rather than averaging the roles together.\n- **Days 13 to 15 (by 2026-10-22):** Choose the finalist. If the finalist's copy changed substantively, run a short retest of about 15 respondents on the finalist page alone.\n- **Days 16 to 21:** Finalize conference copy. Keep this as a buffer.\n\n**Remaining uncertainty:** This study cannot establish whether any line produces more demo requests. That needs a behavioral test after the conference, or on the live homepage with enough traffic.\n"}]}, "haiku-skill-2": {"cost_usd": 0.0144, "seconds": 82, "output_tokens": 16247, "files": [{"name": "positioning-test-brief.md", "text": "# Gridline positioning test brief\n\n**Status:** Provisional expert review and proposed test plan. No buyer responses exist yet. Nothing below is buyer-validated, and no preference is established by data.\n\nEvidence labels: **Observed** = respondent or documented research data (none yet). **Inferred** = analyst reasoning. **Proposed** = untested copy or study design.\n\n## 1. Decision and evidence status\n\n- **Decision:** Which positioning statement leads Gridline's conference materials (hero, booth, opening talk line).\n- **Research question:** Can facility managers and sustainability leads at real-estate firms correctly explain what Gridline does from each statement, and see why it matters to their current work?\n- **Desired next action:** Demo request or follow-up conversation at the conference.\n- **Status:** Provisional expert review. Proposed retest.\n- **Message hypothesis (Inferred):** Each statement leads with a different promise: (A) bill savings without new hardware; (B) grid flexibility; (C) 2030 emissions targets. The statement that names an outcome the buyer's role owns, in plain language, with a claim the company can verify, should win.\n- **Success condition (fixed before fielding):** See section 5.\n\n## 2. Buyer context and gaps\n\n| Field | Current evidence | Source/date | Gap or assumption |\n| --- | --- | --- | --- |\n| Role, seniority, industry, size | Facility managers and sustainability leads at real-estate firms | options.md | Which role holds the budget or the decision is unknown |\n| Priority and desired gain | Not documented | None | Assumed: bill pressure for facility managers, emissions targets for sustainability leads. Unverified |\n| Trigger and success measure | Not documented | None | Unknown. Covered by customer calls |\n| Buying process and alternatives | Not documented | None | Likely manual scheduling, existing building-management systems, or inaction. Unverified |\n| Product | Connects HVAC and meters; shifts load to cheap hours | options.md | Whether it needs new hardware, and whether it reduces emissions, is unknown |\n\nGuidance: The priority is an assumption. The customer calls (section 7) are the step that tests it before any headline is treated as a strategic promise.\n\n## 3. Test record (planned)\n\n- **Study types:** (a) A prospect survey with a preference component across the three directions. (b) Fifteen customer calls for buyer context and proof questions. Customers are not used for clarity scoring because their product familiarity hides first-visit ambiguity.\n- **Stimulus:** The three headlines alone, each on a plain page with the Gridline logo. The headline-only stimulus is a deliberate limitation (section 6): it tests the lead line, not the whole page.\n- **Audience:** About 300 past demo requesters, screened by role (facility manager or sustainability lead, real-estate firm). Demo requesters are warmer than conference attendees (section 6).\n- **Sample:** The preference-test floor is 30 completes, with 50 preferred. The response rate is unknown. If fewer than 30 complete, report results as directional and do not pick a winner on votes alone. Report each role separately. Expect fewer than 15 per role, and state the denominators.\n- **Rotation:** Randomize the order of the three headlines across respondents to limit order effects.\n\n### Survey sequence (Proposed)\n\n1. Show all three headlines in rotated order, with no product description. For each: \"What do you think this company offers? Explain it in your own words.\" Then: \"Who would use this, and for what situation?\"\n2. Reveal the one-line descriptor: \"Gridline is an energy-management platform for commercial buildings.\" For each headline: \"Which part relates to a current priority in your work? Which feels less relevant?\"\n3. For each headline: \"What result would you expect, and how useful would that be?\" (Rating 1 = not useful, 5 = very useful, with an explanation.)\n4. For each headline: \"What would you use instead? What reason, if any, does this headline give to choose this offer?\"\n5. For each headline: \"What makes you hesitate or doubt the claim? What information is missing?\"\n6. Preference: \"Which would you most want to learn more about, and why?\"\n\nSteps 1 to 5 keep spontaneous interpretation ahead of any category reveal. Question wording must stay neutral. Do not repeat the intended category in the question text.\n\n## 4. Provisional expert review (Inferred)\n\n| Candidate | Likely clarity read | Relevance | Value | Differentiation | Claim risk |\n| --- | --- | --- | --- | --- | --- |\n| A. \"Lower your building's energy bill by 15% without new hardware.\" | Strong. Plain outcome. Category is only implied by \"energy bill\" | Strong for facility managers. Weaker for sustainability leads | Concrete. Depends on the 15% being real | \"Without new hardware\" could be a real difference, if true | High. The 15% and \"no new hardware\" both need verification |\n| B. \"Turn your building into a flexible asset on the grid.\" | Weak. \"Flexible asset on the grid\" is industry jargon. The buyer's gain is not stated | Unclear for facility managers. Plausible for energy-focused leads | Implicit. Buyer may assume they earn money, which the brief does not support | Unknown | Medium. Implies grid-program participation that the brief does not mention |\n| C. \"Hit your 2030 emissions targets with the building you already have.\" | Moderate. Outcome is clear. Category is not named | Strong for sustainability leads who own targets | High stakes, but vague on mechanism | Overlaps with A's \"already have\" | Highest. Shifting load to cheap hours does not by itself cut emissions. Targets depend on many factors |\n\n**Provisional lean (Inferred, not a result):** A is the strongest lead line. C is a credible alternative for sustainability leads, but only if carbon evidence exists. B is weakest, mainly on clarity.\n\n**Earliest failed layer, if the hypothesis is right:**\n- B: clarity (jargon, unstated gain).\n- C: value and proof (mechanism is unsupported).\n- A: differentiation and claim verification (number and hardware claim).\n\n## 5. Decision rule (fix and date before fielding)\n\n- **Primary measure:** Share of respondents whose unaided interpretation matches what Gridline does. Two coders work from a codebook written before fielding. Disagreements are reconciled and reported.\n- **Secondary measures, by role:** Relevance rating and explanation. Doubts and missing information. Preference counts are descriptive only.\n- **Gate:** Choose the statement with the highest correct-interpretation rate among the target roles, **provided its central claim can be verified before the conference**. A claim that fails verification is out, whatever the test shows.\n- **Close or small samples:** If rates overlap, or a role has too few respondents, say so. Choose on verifiable proof and fit with the conference audience. Do not present the result as statistically significant.\n- **Final call:** The team lead makes the decision on the basis of this rule.\n\n## 6. Limitations\n\n- Demo requesters are already interested. Results will likely overstate clarity for cold conference attendees.\n- Headline-only stimulus tests the lead line, not the page. Retest once the hero is built.\n- Role split may leave small groups.\n- Order effects remain even with rotation. Reveal sequence is shared across respondents.\n- Preference responses may favor claims that sound best, not claims the company can defend.\n\n## 7. Proof checks (must happen first, days 1 to 3)\n\n- **A, \"15%\":** Which deployments, what baseline, what period, how measured, and is it an average or a range? If the number is not measured, do not use it on stage. **Proposed fallback:** drop the number and test the stripped line in a later round. This variant is untested.\n- **A, \"without new hardware\":** What does connecting HVAC and meters require (gateways, sub-meters, controller access)? If installation is needed, the claim is false as written.\n- **C, \"2030 emissions targets\":** Does Gridline measure or optimize carbon intensity? If not, C is not defensible as written.\n- **B, \"flexible asset on the grid\":** Does Gridline enroll buildings in grid-services programs that pay? If yes, a concrete payment outcome may be a stronger claim. **Proposed**, untested.\n\n## 8. Timeline (21 days)\n\n- **Days 1 to 3:** Proof checks. Freeze stimuli. Write and date the codebook and decision rule. Confirm the 300-person list and role screen.\n- **Days 2 to 12:** Fifteen customer calls. Cover priorities, triggers, alternatives, which proof convinced them, and how they describe the product to peers. Use the headlines for language reactions and proof questions only.\n- **Days 4 to 14:** Prospect survey open. Reminder around day 8.\n- **Days 14 to 17:** Coding by two coders, reconciliation, results by role.\n- **Day 18:** Decision under section 5.\n- **Days 18 to 21:** Build the hero for the chosen line. If the hero changes substantively, a retest with fresh prospects is worth doing. Otherwise, record it as a remaining uncertainty.\n\n## 9. Recommendation and remaining uncertainty\n\n- **Recommendation:** Do not treat any line as validated yet. Verify proof first (section 7), then field the test. Provisional lead: A, contingent on verification.\n- **Remaining uncertainty:** Cold conference audience versus warm demo requesters. Role-level sample size. Headline-only context. Whether the claims survive verification.\n- **Maintenance trigger:** Re-run a message test after the conference, or when pricing, the product, or the buyer segment changes.\n"}]}, "haiku-skill-3": {"cost_usd": 0.01545, "seconds": 89, "output_tokens": 18174, "files": [{"name": "positioning-test-brief.md", "text": "# Gridline conference positioning: diagnosis and test brief\n\n**Status: provisional expert review and proposed test plan. No buyer data exists yet. Nothing here is validated.**\n\nSource: `options.md`. Three candidate statements, audience of facility managers and sustainability leads at real-estate firms, 3 weeks, access to ~300 past demo-requesters by email and 15 customer calls.\n\n---\n\n## 1. Decision and evidence status\n\n- **Decision:** Which of the three statements Gridline leads with at the conference.\n- **Research question:** Can facility managers and sustainability leads at real-estate firms say what Gridline does after reading each statement, and which one they connect to a current priority?\n- **Desired next action:** Not stated in `options.md`. Assumed to be a demo request. **Confirm.**\n- **Status:** Provisional expert review. Proposed test, not run.\n- **Message hypothesis:** Each statement leads with a different buyer question: cost (A), grid flexibility and revenue (B), emissions targets (C). The best statement is the one that matches a live priority of the person who signs off. We do not yet know which priority that is.\n- **Success condition (declared before any data):** See section 8.\n\n## 2. Buyer context and gaps\n\n| Field | Current evidence | Source | Gap or assumption |\n| --- | --- | --- | --- |\n| Role and industry | Facility managers and sustainability leads at real-estate firms | `options.md` | Company size, portfolio size, and whether both roles are in the same buying group are unknown |\n| Budget owner | None | none | Unknown. This determines which statement's question matters |\n| Priority and symptoms | None | none | Is energy cost, grid participation, or emissions targets the live priority? Unknown |\n| Trigger and success measure | None | none | Unknown. Options imply bill savings, grid revenue, and target attainment |\n| Current approach and alternatives | None named | none | Assumed: manual scheduling, existing building controls, or doing nothing. Unverified |\n| Buying process | None | none | Unknown |\n\n**Action:** Use the 15 customer calls in Week 1 to answer these gaps. Ask which of cost, grid, or emissions drives their budget, what triggered their last energy project, who signs off, and what they would replace. Customers are buyer context here, not message-test respondents, because their familiarity hides first-visit ambiguity.\n\n## 3. Test record (planned)\n\n- **Study type:** Two-part message test. Part 1 is comprehension, between-subjects. Part 2 is preference, all three statements shown side by side. Preference is valid here because there are only three viable directions. It must not be the strategic decision on its own.\n- **Stimulus:** A static mock of the Gridline homepage hero and first section. Only the headline changes. The subheading and body are identical across variants, so differences come from the headline alone. Proposed shared subheading, drawn from product facts in `options.md`:\n  > Gridline is energy-management software for commercial buildings. It connects your HVAC systems and meters and shifts energy use to lower-cost hours.\n  \n  Record the version and capture date when the mock is built.\n- **Audience:** Recruit from the ~300 past demo-requesters. Screen for: works at a real-estate firm; role (facility manager, sustainability lead, other); not a current Gridline customer. Log whether each person has had a demo, since that affects familiarity.\n- **Planned sample:** 15 respondents per headline for Part 1 (45 total). Part 2 uses the same respondents after Part 1. Target 45 completes, 60 if possible. The response rate is unknown, so the 300 invitations do not imply completes. Do not report preference results below 30 respondents.\n- **Part 1 questions, in this order** (the headline is the stimulus, so no category is named by us beforehand):\n  1. What do you think this company offers? Explain it in your own words.\n  2. Who would use this, and for what situation?\n  3. Which part relates to a current priority in your work? What feels less relevant?\n  4. What result would you expect, and how useful would that be?\n  5. What would you use instead? What reason, if any, does this page give to choose this offer?\n  6. What makes you hesitate or doubt a claim? What information is missing?\n- **Part 1 ratings (1 to 5), each followed by \"why\":**\n  - Clarity: \"How clearly do you understand what Gridline does?\" 1 = not at all, 5 = completely\n  - Relevance: \"How much does this relate to a problem you have now?\" 1 = not at all, 5 = a current priority\n  - Value: \"How much would the stated outcome be worth to you?\" 1 = not at all, 5 = a great deal\n  - Differentiation: \"Compared with what you use now, how different is this offer?\" 1 = not different, 5 = clearly different\n- **Part 2 questions:** \"Of these three, which would you most want a vendor to lead with at a conference? Why?\" and \"Which, if any, would you not trust? Why?\"\n- **Evidence location:** Not yet created. Plan a response log with respondent ID, role, variant, verbatim answers, and ratings.\n- **Limitations:** Demo-requesters have seen the product or its pitch, so familiarity may hide ambiguity. With ~15 per variant, and fewer per role, this is a qualitative diagnosis, not a population estimate or a significance test. The email list may over-represent engaged buyers.\n\n## 4. Findings and ordered diagnosis (expert review, untested)\n\nNo respondent data exists. Every entry below is a hypothesis, not a result.\n\n| Option | Clarity risk | Relevance | Value and proof | Differentiation | Hypothesis |\n| --- | --- | --- | --- | --- | --- |\n| **A.** Lower your building's energy bill by 15% without new hardware. | Medium. Names an outcome but no category | High for facility managers who own the cost. Weaker for sustainability leads | Concrete outcome. The 15% and \"without new hardware\" are unsourced | \"No new hardware\" could be a real difference if verified | Strongest concrete value. Main risk is the unsupported number |\n| **B.** Turn your building into a flexible asset on the grid. | High. \"Flexible asset\" and \"on the grid\" are jargon for most facility managers | Unclear. Likely limited to asset, finance, or utility-program roles | Outcome is vague. What the buyer gets and does is not stated | Not established | Probably fails the clarity layer for the primary role |\n| **C.** Hit your 2030 emissions targets with the building you already have. | Medium. Names an outcome but no category | High for sustainability leads with 2030 targets. Unknown for facility managers | High if true. Load-shifting to cheap hours does not automatically cut emissions; the effect depends on hours and grid mix | \"Already have\" echoes the no-hardware claim | Strong fit for sustainability leads. Accuracy risk is the largest of the three |\n\n**Earliest failed layer (expected, not observed):** Clarity. None of the three names the category (\"energy-management software\"). The first fix applies to whichever statement wins.\n\n**Expected segment disagreement (to test):** Facility managers may respond to A. Sustainability leads may respond to C. B may resonate only with asset or finance roles, which is an undefined segment.\n\n**Anticipated friction (inferred from the claims, not observed):**\n- A: \"Is 15% real? Do I have to install anything?\"\n- C: \"Does shifting load actually cut emissions?\"\n- B: \"What do I have to install, and what does the grid operator pay me?\"\n\n## 5. Claim and sameness audit\n\n| Page claim | Buyer question | Named alternative | Supporting proof | Decision |\n| --- | --- | --- | --- | --- |\n| A: 15% bill reduction | \"What's the basis?\" | Not named in `options.md` | None provided | Verify the source, or remove the number |\n| A: without new hardware | \"Do I need to install anything?\" | Not named. Likely sensor or control retrofits | None provided | Verify integration requirements |\n| B: flexible asset on the grid | \"What do I get, and what do I do?\" | Not named | None | Clarify or drop |\n| C: 2030 emissions targets | \"Will this cut carbon?\" | Not named | None provided | Needs an emissions analysis; otherwise remove target language |\n| All: category | \"What is this?\" | Not applicable | Not applicable | Add a category line to whichever statement is chosen |\n\n**Sameness risk:** \"Lower energy costs\" and \"reduce emissions\" are common claims in this market. Apart from \"no new hardware\" (if verified), none of the three names a difference a buyer can check against an alternative. A competitor could make the same claims.\n\n## 6. Proposed revision (applies after the test)\n\n- **Hero headline:** The winning statement, with any unverified number or target claim removed or verified.\n- **Subheading:** Category, then mechanism, then what is required:\n  > Gridline is energy-management software for commercial buildings. It connects your HVAC systems and meters and shifts energy use to lower-cost hours. [Installation requirement, once verified.]\n- **Difference:** Only the \"no new hardware\" claim, and only if verified against a named alternative.\n- **Proof placement:** Next to the claim it supports. Use the product walkthrough, not invented results.\n- **CTA:** Confirm the intended action (assumed: demo request).\n- **Claim status:**\n  - 15%: needs verification\n  - No new hardware: needs verification\n  - 2030 emissions: needs analysis or removal\n  - Category and mechanism: supported by `options.md` product description, needs confirmation by the product team\n\n## 7. Prioritized changes and timeline\n\n| Priority | Change | Why | Timing | Owner |\n| --- | --- | --- | --- | --- |\n| 1 | Verify the 15%, \"no new hardware,\" and the emissions mechanism | Unsupported claims cannot go in front of sustainability leads or facility managers | Week 1, before the stimulus is frozen | Unassigned |\n| 2 | Run 15 customer calls for buyer context | Identifies which priority and budget owner the statement must match | Week 1 | Unassigned |\n| 3 | Build the stimulus with a shared category subheading; only the headline varies | Isolates the headline effect and fixes clarity across variants | Week 1 | Unassigned |\n| 4 | Run the two-part test on ~45 to 60 demo-requesters | Tests clarity, relevance, and preference | Week 2 | Unassigned |\n| 5 | Analyze against the rule in section 8; lock conference copy | Decision by end of Week 3 | Week 3 | Unassigned |\n\nCalendar, assuming today is 2026-10-08: Week 1 ends 2026-10-14, Week 2 ends 2026-10-21, Week 3 ends 2026-10-28. The conference date is unknown. If it is earlier, the schedule compresses from the end.\n\n## 8. Decision rule and recommendation\n\nDeclared before any data is collected:\n\n1. **Coding rule:** A comprehension answer is correct if it names energy-management, building-energy, cost, or emissions software for buildings, or an equivalent accurate category. Write the rubric before analysis. One coder, plus a second coder on a 20% sample.\n2. **Clarity gate:** A headline passes if at least 10 of its 15 respondents give a correct answer.\n3. **Relevance:** Among passing headlines, choose the one that best matches the primary role for the conference. The primary role is set by the Week 1 customer calls. Report facility managers and sustainability leads separately.\n4. **Unsupported claims:** Any claim not verified by the end of Week 2 is removed from the chosen headline before the conference. The test then covers the claim's category, not its exact wording, and this must be disclosed.\n5. **If no statement passes the clarity gate:** Do not choose by preference. Run one more one-week test of the strongest option with the category line in the headline.\n\n**Recommendation (provisional, not a ship decision):** Lean toward A, conditional on three things: the 15% is verified or removed, \"no new hardware\" is verified, and the Week 1 calls confirm cost is the budget driver for facility managers. Keep C as the alternative if the calls show sustainability leads hold the budget and an emissions analysis supports the claim. Do not lead with B on the expert review alone. Its clarity risk for the primary buyer is high.\n\n**Remaining uncertainty:** This study measures interpretation and preference. It cannot show that any statement will raise demo requests. With ~300 emails, a conversion comparison between statements is underpowered. Do not use conference click or demo counts to choose the statement. Track them as descriptive data only.\n\n**Next research:** Customer calls (Week 1), the two-part test (Week 2). After the conference, review sales-call objections for signs of a different priority.\n\n## Checklist status\n\n- Decision defined: **PASS** (desired action to confirm)\n- Buyer context usable: **FAIL, provisional** until Week 1 calls are done\n- Correct study: **PASS**\n- Stimulus traceable: **PENDING** (not built yet)\n- Audience traceable: **PENDING** (planned screen, not yet applied)\n- Questions diagnostic: **PASS** (open questions precede ratings)\n- Sample claims restrained: **PASS**\n- Evidence authentic: **PASS** (no results invented)\n- Order respected: **PENDING**\n- Doubts addressed: **FAIL** (15%, emissions, and installation doubts unanswered)\n- Difference defensible: **FAIL** for B; A's \"no new hardware\" unverified\n- Claims supported: **FAIL** for 15%, 2030 target, and \"no new hardware\"\n- Revision focused: **PENDING**\n- Strategy retained: **PASS** (cost, grid, and emissions trade-offs are explicit)\n- Recommendation justified: **PASS** (no ship recommendation given)\n- Retest useful: **PASS**\n- Measurement boundary clear: **PASS**\n- Maintenance scoped: **PASS** (post-conference review)\n\n## Open questions for the team\n\n1. Where does the \"15%\" come from? Is there measured data?\n2. Does \"without new hardware\" hold for every customer type, or only some?\n3. What is the conference date?\n4. What is the intended next action from the statement: demo request, trial, or something else?\n5. Who signs off on energy projects at the target firms, and how do they weigh cost versus emissions?\n6. Who are the named alternatives buyers compare against?\n"}]}, "opus-1": {"cost_usd": 0.16133, "seconds": 68, "output_tokens": 4929, "files": [{"name": "positioning-scorecard.csv", "text": "participant,role,customer_status,first_line_shown,A_5sec_understood(correct/partial/wrong),B_5sec_understood,C_5sec_understood,A_believable(1-5),B_believable(1-5),C_believable(1-5),would_forward_to_boss(A/B/C),final_pick(A/B/C),why,objections_or_quotes\nP01,,,A,,,,,,,,,,\nP02,,,B,,,,,,,,,,\nP03,,,C,,,,,,,,,,\nP04,,,A,,,,,,,,,,\nP05,,,B,,,,,,,,,,\nP06,,,C,,,,,,,,,,\nP07,,,A,,,,,,,,,,\nP08,,,B,,,,,,,,,,\nP09,,,C,,,,,,,,,,\nP10,,,A,,,,,,,,,,\nP11,,,B,,,,,,,,,,\nP12,,,C,,,,,,,,,,\nP13,,,A,,,,,,,,,,\nP14,,,B,,,,,,,,,,\nP15,,,C,,,,,,,,,,\n"}, {"name": "positioning-test-plan.md", "text": "# Gridline positioning test: plan, prediction, decision rule\n\n**Candidates**\n- **A, Cost:** \"Lower your building's energy bill by 15% without new hardware.\"\n- **B, Grid:** \"Turn your building into a flexible asset on the grid.\"\n- **C, Emissions:** \"Hit your 2030 emissions targets with the building you already have.\"\n\n**Audience:** facility managers (FMs) and sustainability leads at real-estate firms.\n**Resources:** 300 past demo-requesters (email), 15 customer calls, 3 weeks.\n\n---\n\n## 1. My prediction (before any data)\n\nNo test has run yet, so this is a hypothesis to check, not a result.\n\n| | Prediction | Reasoning |\n|---|---|---|\n| **A, Cost** | **Most likely winner overall**, strongest with FMs | Concrete number, the benefit is the buyer's own budget line, and \"without new hardware\" answers the first objection (capex, install disruption). Risk: \"15%\" invites \"prove it\", so we need a case study ready at the booth. |\n| **C, Emissions** | Close second; may **win with sustainability leads** | Ties to a deadline they already own (2030 targets, plus local rules like NYC LL97 and BERDO, and the EU EPBD). \"The building you already have\" carries the same no-capex message as A. |\n| **B, Grid** | **Most likely loser** | \"Flexible asset on the grid\" is utility/DR-industry language. It describes what Gridline *does*, not what the buyer *gets*. FMs may read it as \"the utility controls my HVAC\". |\n\nThe likely real outcome is a **split by role** (A for FMs, C for sustainability leads). In that case, use A as the headline and C as the subhead (see section 5).\n\n---\n\n## 2. What we can and can't learn from this sample\n\n**A three-way email split test won't produce a reliable winner.** 300 contacts gives ~100 per arm. At a typical 3\u20135% click/reply rate, that's 3\u20135 responses per arm. To detect a difference at the usual 95% confidence and 80% power, one arm would need roughly **4\u00d7 the response rate** of another (e.g. 3% vs 12%). Real differences between taglines are usually far smaller. An open-rate test (subject lines) is also unreliable: Apple Mail Privacy Protection inflates open counts.\n\n**What does work at this size:**\n1. **Within-person forced choice.** Each respondent sees all three and picks one. This removes between-group noise. With ~60 responses, a 30/15/15 split is significant (\u03c7\u00b2 \u2248 7.5, p \u2248 0.02).\n2. **Qualitative depth from the 15 calls.** Use them to find out *why* people choose and whether they understand each line. Don't use them to count votes.\n3. **One behavioral signal**, as a tiebreaker only: which line gets people to book a meeting at the conference.\n\n**Bias to remember:** past demo-requesters are already warm and lean toward people who care about cost. Read the results as \"best line for people who already know us\" and give the call findings (especially from cold or less-engaged contacts) extra weight.\n\n---\n\n## 3. The test (3 weeks)\n\n### Week 1: email survey + behavioral split\n- **Clean the list first.** Tag each contact's role (FM / sustainability / other) from CRM or LinkedIn. We have to segment results by role, so this step is required.\n- **Send one email to all 300**, split into 3 random arms of ~100. Arms differ **only** in the headline line used in the opening sentence and the CTA (\"Meet us at [Conference] \u2192 book 15 min\").\n  - Primary behavioral metric: meeting bookings per arm (tiebreaker only, given the power math above).\n- **The same email includes a one-question survey** (all arms see all three lines, **order randomized**):\n  > \"Which of these would make you stop at our booth?\" (pick one) + optional \"Why?\" free text + role dropdown.\n  - Primary decision metric: forced-choice share, overall and by role.\n- Send a reminder on day 4 to people who haven't responded. Target: 50\u201375 survey responses (17\u201325%, realistic for a warm list with a one-click survey).\n\n### Weeks 1\u20132: 15 customer calls (~10 min of each call)\nAim for a mix: ~7 FMs, ~7 sustainability leads, and at least 3 people who did *not* buy or went cold. Rotate which line is shown first.\n\nScript, for each line in turn:\n1. **5-second test.** Show the line, hide it, then ask \"What does this company do?\" (Score: correct / partly / wrong.)\n2. **Restate:** \"In your own words, what's the promise?\"\n3. **Gut reaction:** \"Believable? What would you need to see?\"\n4. **Forward test:** \"Which of these would you forward to your boss / CFO?\" (This tests whether the line helps them sell internally.)\n5. **Final pick + why.**\n\nLog answers in `positioning-scorecard.csv`.\n\n### Week 3: decide and produce\nBooth graphics need printing lead time, so **decide by the end of week 2** (day 14) and use week 3 for production and proof assets.\n\n---\n\n## 4. Decision rule (agreed before looking at the data)\n\n1. **Comprehension gate:** any line that \u2265 1/3 of call participants misread in the 5-second test is dropped, whatever its vote share. (I expect B to fail here.)\n2. **Winner** = the highest forced-choice share among remaining lines, **if** it leads the runner-up by \u2265 10 points overall *and* isn't last in either role segment.\n3. **If roles split** (FMs pick A and sustainability leads pick C, each by \u2265 10 points): run the hybrid in section 5.\n4. **If it's a statistical tie:** use meeting bookings, then the \"forward to boss\" count from the calls.\n5. **Credibility check:** if A wins but \u2265 5 of 15 calls doubt the \"15%\", either soften it to \"up to 15%\" / \"avg. 15% across N buildings\" or back it with a named case study at the booth. Don't drop the number.\n\n---\n\n## 5. Likely final form (if the prediction holds)\n\n> **Headline:** Lower your building's energy bill by 15%, without new hardware.\n> **Subhead:** And hit your 2030 emissions targets with the building you already have.\n\nThe cost line stops FMs walking past the booth. The emissions line gives sustainability leads a reason to stay. \"Flexible grid asset\" moves into the *how it works* explanation for booth conversations, where it fits as a mechanism.\n\n---\n\n## 6. Checklist\n- [ ] Role-tag the 300 contacts\n- [ ] Build 3 email variants + randomized-order survey (one question + why + role)\n- [ ] Back up the 15% claim (which buildings, what baseline) before sending anything\n- [ ] Schedule the 15 calls (include \u2265 3 lost/cold accounts)\n- [ ] Day 10: tally results; Day 14: decide using section 4\n- [ ] Week 3: booth graphics, one-pager, case study for the winning claim\n"}]}, "opus-2": {"cost_usd": 0.16266, "seconds": 116, "output_tokens": 5128, "files": [{"name": "call-scoring.csv", "text": "call_id,date,name,company,role(FM/Sustainability/Other),customer(Y/N),first_shown(A/B/C),5s_understood_A(Y/Partial/N),5s_understood_B(Y/Partial/N),5s_understood_C(Y/Partial/N),rank_1st(A/B/C),rank_last(A/B/C),belief_of_top_pick(1-5),would_forward(A/B/C/none),who_signs_contract,their_own_words,notes\n"}, {"name": "positioning-test-plan.md", "text": "# Gridline positioning test: plan and decision rule\n\n**Status:** Nothing has been tested yet. The \"current lean\" below is a guess based on the audience. It is not a result. The winner gets picked by the decision rule in section 5, once the data comes in.\n\n| # | Statement | Short name |\n|---|-----------|------------|\n| A | \"Lower your building's energy bill by 15% without new hardware.\" | **Savings** |\n| B | \"Turn your building into a flexible asset on the grid.\" | **Grid asset** |\n| C | \"Hit your 2030 emissions targets with the building you already have.\" | **2030 targets** |\n\n---\n\n## 1. What we're actually trying to learn\n\n1. **Which statement makes the conference audience want to stop and talk to us?** This is the main question.\n2. **Does the answer split by role?** My hypothesis is that facility managers will pick Savings and sustainability leads will pick 2030 targets. If that's right, the real decision is which buyer we lead with, and the other message becomes the subline.\n3. **Do people understand it and believe it?** A statement that wins on appeal but gets misread (\"so you sell batteries?\") or doubted (\"15%? sure\") will fail at a booth.\n\n## 2. Why not a plain 3-way email A/B test\n\nSplitting 300 contacts into three groups gives about 100 per arm. At typical B2B rates (around 25% opens, 3\u20135% clicks) that's 3\u20135 clicks per arm. No real difference would be detectable at that size, so the winner would be noise.\n\n**What to do instead:** send one email in which **every recipient sees all three statements** and picks one. Each response then compares all three options directly, which gets a lot more out of a small list. The 15 calls cover question 3 (understanding and belief), which a survey can't answer.\n\n## 3. Test 1: forced-choice email survey (300 past demo-requesters)\n\n**Setup**\n- One email, one question, answered in a single click. Use a separate link per option so nobody has to open a form. Then ask one optional follow-up: \"What's your role?\" (Facilities / Sustainability / Other).\n- **Randomize the order of the statements** across recipients, using 3 or 6 order variants. People pick the first option more often, and randomizing cancels that out.\n- Record the role from the CRM where you have it, so the survey doesn't need to ask.\n- Send on a Tuesday or Wednesday morning. Send one reminder after 4 days to people who haven't responded.\n\n**Draft email**\n\n> Subject: Quick one: which of these would make you stop at our booth?\n>\n> Hi {first_name}, we're at {conference} next month and picking the line for our booth. You asked about Gridline before, so your gut reaction is the most useful input we can get.\n>\n> Which one makes you most want to learn more? (One click, that's it.)\n>\n> \u2192 {statement 1 link}\n> \u2192 {statement 2 link}\n> \u2192 {statement 3 link}\n>\n> Thanks, {sender}\n\nEach link goes to a thank-you page with the optional question \"What's your role?\" plus \"Anything we got wrong?\" as a free-text box.\n\n**Expected volume:** a 10\u201315% response rate means **30\u201345 votes**. That's enough to find a clear winner, such as one statement getting half the votes. It's not enough to separate two statements that are close. Section 5 covers that case.\n\n**Caveat:** past demo-requesters already know us and showed interest. Their views are more favorable and better informed than a cold booth visitor's. Treat the result as directional, and lean on the calls to check whether the statement makes sense to someone new.\n\n## 4. Test 2: 15 customer and prospect calls (about 20 minutes each)\n\n**Who:** aim for about 7 facility managers and about 7 sustainability leads. If possible, include 3\u20135 people who are *not* current customers, because customers already know what we do.\n\n**Script.** Keep the order fixed, and randomize which statement each person sees first.\n\n1. **Five-second test.** Show one statement for 5 seconds, then hide it. Ask: \"What does this company do?\" and \"Who is it for?\" Score it as correct, partly correct, or wrong. Do this for all three statements.\n2. **Rank.** Show all three statements and ask: \"Which would make you walk over to the booth? Which would you ignore?\"\n3. **Belief check.** For the statement they ranked first, ask: \"How believable is that, from 1 to 5? What would you need to see to believe it?\"\n4. **Forward test.** \"Which of these would you forward to your boss or CFO? Why?\" This shows which message helps someone sell the purchase internally.\n5. **Their words.** \"How would *you* describe what we do to a colleague?\" Write down their exact wording, because it's often better than any of the options.\n\n**Scoring sheet:** see `call-scoring.csv`.\n\n## 5. Decision rule (agree on this before looking at any data)\n\nCount the results from both tests together:\n\n1. **Clear winner.** A statement wins if it gets at least 45% of survey votes **and** ranks first in at least 7 of the 15 calls **and** at least 10 of 15 people understood it in the five-second test. Use that statement.\n2. **Role split** (the most likely outcome in my view). Facility managers pick Savings and sustainability leads pick 2030 targets. Lead with the statement chosen by **whichever role signs the contract** at our target accounts. Sales should know this; look at closed deals. Use the other statement as the subline.\n   - Example: **\"Lower your building's energy bill by 15% \u2014 and hit your 2030 targets \u2014 without new hardware.\"** Alternatively, keep the headline short and put the second message on the booth's secondary panel.\n3. **Too close to call.** No statement leads by 10 or more points. Go with the statement that people **understood** best in the five-second test. At a booth, people only glance, so being understood quickly matters more than being liked slightly more.\n4. **Veto.** If a statement gets an average belief score below 3/5, or if 5 or more people misread what we do, it can't be the headline, whatever its vote count.\n\n## 6. Timeline (3 weeks)\n\n| When | What |\n|------|------|\n| Week 1, Mon\u2013Tue | Confirm the decision rule. Finalize the email. Check that the claims are true (see section 7). Start booking calls. |\n| Week 1, Wed | Send the survey. |\n| Week 1, Thu \u2013 Week 2, Fri | Run the 15 calls. |\n| Week 1, Sun | Send the reminder to non-responders. |\n| Week 2, Fri | **Decide.** Tally the results and apply the rule in section 5. |\n| Week 3 | Produce booth graphics, the one-pager, and the badge-scan follow-up email. |\n\nPrint vendors often need about 5 business days. If yours does, **the decision has to be made by the end of week 2.** Don't let the calls run into week 3.\n\n## 7. Check the claims before testing them\n\n- **\"15%\"**: Is this a typical result, a median, or a best case? Can we point to 2\u20133 named or anonymized customer results? Real-estate sustainability leads will ask, and using an unsupported number in front of a room of buyers is risky. If the honest figure is a range, consider \"up to 15%\" or \"10\u201320%\". Then test the wording you'll actually print.\n- **\"Without new hardware\"**: Is this true for every customer? Gridline connects to HVAC and meters. Some buildings need a gateway or new submeters. If so, change the wording to \"without replacing equipment\".\n- **\"2030 targets\"**: Does the shifting of load to cheap hours actually reduce reportable emissions? That depends on how buildings are measured, for example by NYC Local Law 97 or CRREM. If it only cuts cost, Option C claims too much.\n\n## 8. Current lean (a prior, not a result)\n\n- **Grid asset (B) will probably lose.** It's the most distinctive, but facility managers and sustainability leads don't think in terms of \"grid assets\". The phrase describes how Gridline works, not what the customer gets, and it takes more than a glance to make sense of. I'd only keep it as a subline if the calls show surprising pull.\n- **Savings (A) vs. 2030 targets (C) is the real contest,** and it will probably come down to role. A is concrete and has a number, which works well on a booth. C speaks to the sustainability lead's mandate and the fines they face. \"With the building you already have\" also handles the \"do we need capex?\" objection.\n- If I had to choose today without data, I'd pick **A as the headline and C as the subline**. Run the tests anyway. The point is to find out whether the people who sign the contract agree.\n"}, {"name": "survey-tally.csv", "text": "respondent_id,order_variant,vote(A/B/C),role(FM/Sustainability/Other),source_of_role(CRM/self),free_text\n"}]}, "opus-3": {"cost_usd": 0.22509, "seconds": 91, "output_tokens": 7392, "files": [{"name": "call-log.csv", "text": "call_no,date,company,role_segment,customer_status,order,A_playback_correct,A_relevance_1to5,A_first_doubt,B_playback_correct,B_relevance_1to5,B_first_doubt,C_playback_correct,C_relevance_1to5,C_first_doubt,rank_1st,rank_2nd,rank_3rd,forward_to_boss,word_changes,notes\n"}, {"name": "call-script.md", "text": "# Customer call script: positioning test (about 10 minutes of a call)\n\nShow each statement **on its own, on screen or in chat**, rather than reading it aloud. On the booth it will be read, not heard.\n\n**Rotation:** use the order matching the call number (call 1 \u2192 order 1, call 4 \u2192 order 1 again, and so on).\n- Order 1: A, B, C\n- Order 2: B, C, A\n- Order 3: C, A, B\n\nA = \"Lower your building's energy bill by 15% without new hardware.\"\nB = \"Turn your building into a flexible asset on the grid.\"\nC = \"Hit your 2030 emissions targets with the building you already have.\"\n\n## For each statement, one at a time\n\n1. **5-second reaction:** show it for about 5 seconds, then hide it. \"What's that company offering?\"\n   - Record it in their words. Did they get it right? (Y / partly / N)\n2. \"On a scale of 1\u20135, how relevant is this to what you're measured on this year?\"\n3. \"What's the first question or doubt that pops into your head?\"\n   - Listen for credibility doubts (\"15%? really?\") vs confusion (\"what's a flexible asset?\").\n\n## After all three\n\n4. \"Rank them: which would make you stop at a booth?\" (1st, 2nd, 3rd)\n5. \"Which one would you forward to your boss or CFO to justify a meeting?\" (This is often different from #4, and that's worth knowing.)\n6. \"Is there a word in any of them you'd change?\"\n7. Confirm their role: facility/ops, sustainability/ESG, or other.\n\n## Don't\n\n- Don't explain what a statement means before they react.\n- Don't say which one the team prefers.\n- Don't ask \"do you like it?\" People say yes to everything. Ask about rank and action instead.\n"}, {"name": "email-results.csv", "text": "contact_id,segment,arm,clicked,replied,meeting_booked\n"}, {"name": "email-variants.md", "text": "# Email variants\n\nOnly the **subject line** and the **headline line** change between arms. Everything else stays identical, including the length, CTA, sender, and send time.\n\nGive each arm's link its own UTM tag (`utm_content=A|B|C`) so clicks can be attributed. Replies are attributed by arm from the send list.\n\n---\n\n**Subject (arm A):** Lower your building's energy bill by 15%, without new hardware\n**Subject (arm B):** Turn your building into a flexible asset on the grid\n**Subject (arm C):** Hit your 2030 emissions targets with the building you already have\n\n---\n\nHi {first_name},\n\nYou asked about Gridline a while back, so here's what we've been working on.\n\n**{HEADLINE = same statement as the subject line}**\n\nGridline connects to the HVAC and meters you already have and automatically shifts load to the cheapest hours.\n\nWe'll be at {Conference} on {dates}. If it's useful, we've put together a 2-page brief showing how this works in a building like yours.\n\n\u2192 [Get the 2-page brief]({link}?utm_campaign=positioning_test&utm_content={A|B|C})\n\nOr just reply \"brief\" and I'll send it over.\n\n{Sender}\n\n---\n\nNote: the body sentence (\"connects to the HVAC and meters you already have\u2026\") is deliberately neutral. It doesn't repeat any arm's benefit, so the headline does the persuading on its own.\n"}, {"name": "positioning-test-plan.md", "text": "# Gridline positioning test: plan and recommendation\n\n## The short answer\n\n**No winner has been picked yet, because there's no data yet.** My prediction, before testing:\n\n| # | Statement | Prediction |\n|---|---|---|\n| A | \"Lower your building's energy bill by 15% without new hardware.\" | **Most likely winner overall.** It's concrete, has a number, and answers the capex objection. It's the safest choice for a booth banner that people read in 3 seconds. Risk: the \"15%\" must be backed up (see Pre-flight). |\n| B | \"Turn your building into a flexible asset on the grid.\" | **Most likely to lose.** \"Flexible asset\" is utility and grid-operator language. Facility managers don't wake up wanting one, and it doesn't say what they get. |\n| C | \"Hit your 2030 emissions targets with the building you already have.\" | **Strong with sustainability leads, weaker with facility managers.** It could beat A if the conference crowd skews toward sustainability. |\n\nThe real question is probably **A vs C, and which audience the booth is for**. The test below is designed to answer that, not just to name a statement.\n\n## Pre-flight (day 1\u20132)\n\n1. **Can we defend \"15%\"?** Is it a median across customers, a best case, or a model? If a prospect asks \"15% of what, measured how?\", we need a one-line answer. If we can't give one, change A to a range we can back up (\"10\u201320%\") before testing. Otherwise we'd be testing a claim we can't use.\n2. **Segment the 300 contacts by role** (facility/ops vs sustainability/ESG vs other) from CRM job titles. The analysis depends on this.\n3. **Check the print deadline.** Booth graphics often need 7\u201310 days lead time. Work backward from it; it is probably the real deadline, not the conference date.\n4. **Write the decision rule down now** (see below), before any results come in.\n\n## Step 1: customer calls (week 1, 15 calls)\n\nCalls tell us *why* a statement works. The email tells us *how many* people act on it. Doing the calls first lets us drop the weakest option, which gives the email test more statistical power.\n\n- Aim for a mix: about 7 facility/ops and about 7 sustainability. Use current customers *and* lost or stalled prospects if possible. Current customers only would bias toward whatever they already bought.\n- Use the script in `call-script.md`. It covers a cold first reaction, playback in their own words, a forced ranking, and the \"forward to your boss\" test.\n- **Rotate the order** in which the statements are shown (orders are listed in the script), so no option always benefits from going first.\n- Log answers in `call-log.csv`.\n\n**End of week 1 checkpoint:** if one statement is ranked last by at least 9 of 15 people, or nobody can play it back correctly, drop it. I expect this to be B. Then run the email test as a two-way split (150 per arm) instead of three-way (100 per arm).\n\n## Step 2: email test (week 2, up to 300 past demo-requesters)\n\n**Design**\n- Randomize contacts into arms, **stratified by role segment**, so each arm has the same mix of facility and sustainability people.\n- Every arm gets **the same email**. Only the subject line and the opening headline change, and both use the positioning statement. Templates are in `email-variants.md`.\n- Send all arms at the same time, from the same sender.\n- **Primary metric: click-through *or* reply** (anyone who clicks the CTA or replies counts once). Do **not** use open rate. Apple Mail Privacy Protection inflates opens, so open rate is mostly noise.\n- Secondary: demo or meeting booked within 7 days, though counts will be tiny.\n- Read the results after 5 business days with `python3 analyze.py email-results.csv`.\n\n**Be honest about power.** With 150 per arm and a baseline action rate near 10%, the test can only reliably detect large gaps (roughly 10% vs 20%). With 100 per arm, the gap would need to be even larger. Small differences in the email will be inconclusive. That's fine: in that case the calls break the tie. Don't read meaning into 12 clicks vs 9.\n\nAlso note that past demo-requesters are already warm, and partly self-selected by whatever messaging brought them in. They're a reasonable proxy for booth visitors, but not a perfect one.\n\n## Step 3: decide (start of week 3)\n\n**Decision rule (agree on this before results come in):**\n\n1. If the email shows a statistically clear winner (p < 0.10 on the primary metric) **and** the calls don't contradict it, pick that one.\n2. If the email is inconclusive, pick the statement that **won the call ranking** and was played back correctly most often.\n3. If the winner differs by segment (e.g. A wins with facility managers and C wins with sustainability leads), don't force one choice:\n   - Use the statement for the segment that dominates the conference attendee list (ask the organizers for the registration breakdown) as the **banner headline**.\n   - Use the other as the **subhead or second panel**. For example: *\"Lower your building's energy bill by 15% without new hardware. And hit your 2030 targets with the building you already have.\"* The two statements share \"no new hardware / building you already have\", so they combine naturally.\n4. Use week 3 for the remaining 2\u20133 calls to sanity-check the final wording, then send to print.\n\n## Timeline\n\n| When | What |\n|---|---|\n| Wk 1, Mon\u2013Tue | Pre-flight: verify the 15% claim, segment the list, confirm print deadline, agree on the decision rule |\n| Wk 1, Tue\u2013Fri | 12 calls; Friday checkpoint to drop the weakest option |\n| Wk 2, Tue | Email send (2 or 3 arms) |\n| Wk 2, Fri \u2192 Wk 3, Mon | Read email results |\n| Wk 3, Mon\u2013Tue | Decide; 3 confirmation calls on the final wording |\n| Wk 3, by print deadline | Final copy to print |\n\n## Files\n\n- `positioning-test-plan.md`: this plan\n- `call-script.md`: customer call script with rotation orders\n- `call-log.csv`: template for recording call results\n- `email-variants.md`: email copy for each arm\n- `email-results.csv`: template for email results (one row per contact)\n- `analyze.py`: computes action rates and significance overall and by segment\n"}]}, "haiku-1": {"cost_usd": 0.0088, "seconds": 56, "output_tokens": 10619, "files": [{"name": "positioning-test-plan.md", "text": "# Gridline positioning test plan\n\nStatus: plan only, not yet run. Lock the decision rule (section 5) with a date before any results come in.\n\n## 1. Question\n\nWhich statement should Gridline use as its conference positioning, for facility managers and sustainability leads at real-estate firms?\n\n## 2. Options and prior\n\n| | Statement | Speaks most to | Main risk |\n|---|---|---|---|\n| A | Lower your building's energy bill by 15% without new hardware. | Facility managers (cost, no capex) | The 15% needs a source. It will be challenged on stage or at the booth. |\n| B | Turn your building into a flexible asset on the grid. | Nobody in the audience directly. Closer to utilities and asset finance. | Jargon. The owner's benefit is never stated. |\n| C | Hit your 2030 emissions targets with the building you already have. | Sustainability leads (targets) | Load shifting is a cost lever. It does not by itself cut emissions and can raise them on some grids, so C promises something the mechanism may not deliver. |\n\nPrior, to be tested: A wins for facility managers, C is the strongest for sustainability leads, and B is last. That prior assumes facility managers are a large share of buyers. Confirm the audience split with sales.\n\n## 3. Pre-checks before anything goes out\n\n- **15% source.** Confirm how it was measured: which buildings, what baseline, what period, how many sites. If it comes from one pilot, either say so in the copy or soften the claim. Do not use it unsubstantiated in front of a conference audience.\n- **\"Without new hardware.\"** Confirm this holds for every integration type Gridline supports.\n- **C's emissions claim.** Confirm what Gridline can actually show about emissions. If it cannot show it, drop C.\n\n## 4. Tests\n\n### Test 1: Email to past demo-requesters (about 300)\n\n- Stratify the list by role (facility management, sustainability, other), then randomize into three arms of about 100. Keep the role mix equal across arms.\n- Same send time, same subject line, same body. Only the first line changes to the statement under test.\n- Primary metric: (replies + clicks on the booking link) / delivered, per arm.\n- Sample size: at about 100 per arm, you can reliably detect only large gaps (roughly 10% vs 27%). Smaller differences will look like ties. Treat this as a filter, not a verdict.\n- Bias: these people already asked for a demo, so they are warm. This measures what the line does for existing interest, not for a cold conference audience.\n- Email measures action, not understanding. That is why Test 2 exists.\n\n### Test 2: Short calls with 15 people (about 20 minutes each)\n\n- Recruit about 7 facility managers or operations leads, 5 sustainability leads, and 3 others. Prefer prospects or non-customers. Customers will be generous.\n- Script:\n  1. (2 min) \"We're choosing a headline for a conference. I'm not selling anything.\"\n  2. (3 min) Their situation: role, portfolio size, what they report upward (cost, emissions, or both).\n  3. (5 min) Comprehension. Show each statement on its own card, in rotated order. For each, ask: \"What does this company offer, in your words?\" \"Who is this for?\" \"What would you need to believe for this to be true?\"\n  4. (3 min) Ranking: \"Which would make you stop at a booth? Which would you not want your name associated with?\"\n  5. (2 min) Credibility, for A and C only: \"What would you need to see to believe this?\"\n  6. Record answers verbatim.\n- Rotate the card order. There are six possible orders of three, so use each about two or three times across 15 calls.\n- Score per call: comprehension correct (Y/N) for each statement, first-place pick, and any credibility objections.\n\n## 5. Decision rule (lock and date before data)\n\n1. **Clarity gate.** Drop any statement that fewer than 12 of 15 call respondents restate correctly. A misread disqualifies a statement regardless of clicks.\n2. **Calls decide.** Among statements that pass the gate, the one with the most first-place rankings wins. Break ties by the email rate.\n3. **Email is a check.** If the call winner is clearly last in email, re-examine before committing. If it leads or ties, that confirms the choice.\n4. **No clear winner.** Default to A, and use C's language as a secondary line in talks aimed at sustainability leads. This default is a judgment call. Change it if sales says sustainability leads are the bigger buyer group.\n\n## 6. Timeline (3 weeks)\n\n- **Days 1\u20133:** Pre-checks (section 3). Finalize copy. Lock and date the decision rule. Start recruiting the 15 calls.\n- **Day 4:** Send the three email arms on the same day at the same time. Send one identical reminder on day 7 to non-responders in every arm.\n- **Days 4\u201312:** Run calls, about two per day.\n- **Day 14:** Read out both tests and decide.\n- **Days 15\u201321:** Build conference materials around the winner. Keep this as buffer.\n\n## 7. Email drafts\n\nSubject (all arms): Following up on your Gridline demo\n\nBody (all arms, after the first line):\n\n> Hi {first name},\n>\n> Gridline connects to your building's HVAC and meters and shifts load to cheaper hours.\n>\n> Reply with a time that works for a 20-minute walkthrough.\n>\n> {name}\n\nFirst line by arm:\n\n- **A:** Lower your building's energy bill by 15% without new hardware.\n- **B:** Turn your building into a flexible asset on the grid.\n- **C:** Hit your 2030 emissions targets with the building you already have.\n\n## 8. Open questions\n\n- Source and methodology for the 15% figure.\n- Share of conference buyers who are facility managers versus sustainability leads.\n- Conference date, so the day 14 decision can be checked against the production deadline.\n- Whether the 15 calls can include non-customer prospects.\n"}]}, "haiku-2": {"cost_usd": 0.00579, "seconds": 32, "output_tokens": 5840, "files": [{"name": "positioning-test-plan.md", "text": "# Gridline positioning test plan\n\nGoal: pick one positioning statement for the conference in 3 weeks, using the 300 past demo-requester emails and 15 customer calls.\n\n## The three options\n\n- **A (savings):** \"Lower your building's energy bill by 15% without new hardware.\"\n- **B (grid asset):** \"Turn your building into a flexible asset on the grid.\"\n- **C (emissions):** \"Hit your 2030 emissions targets with the building you already have.\"\n\n## Prior (to be tested, not a result)\n\n- **A is the most likely winner for facility managers.** It names a budget outcome they own, and \"without new hardware\" removes the main objection. It is only safe if the 15% can be backed by data (see gate below).\n- **C is the strongest candidate for sustainability leads**, who own the 2030 target. It is the runner-up, and the call split by role will show whether it should lead for that segment.\n- **B is the weakest for this audience.** \"Flexible asset on the grid\" is utility-industry language. It may land with grid or utility contacts, but facility managers and sustainability leads are unlikely to act on it.\n\n## Gate before testing: substantiation\n\nEach claim has to survive scrutiny at the booth. Settle these in days 1-3. An option that fails its gate is out, whatever the tests say.\n\n- **A:** Where does 15% come from? Document the sample size, baseline, period, and whether it is a median or a best case. If it is a best case, reword to \"up to 15%\" or \"a median of X% across N buildings\" before testing. Do not test the unqualified version if it can't be defended.\n- **C:** \"Hit your targets\" implies a guarantee. Reword to a contribution claim (e.g., \"Cut the emissions you can control in the building you already have\") unless the company can show measured reductions.\n- **B:** Confirm the grid-services revenue or demand-response claims, if any, are current.\n\n## Test 1: Email split (days 4-14)\n\n- Send one email to all 300 contacts with the same body. Only the headline and the first line vary across the three options.\n- Randomize contacts into three arms of about 100 each. Send all arms the same day, so timing does not confound the result.\n- Track clicks and demo requests per arm with separate UTM tags.\n- Segment by role if the CRM has it.\n\n**Primary metric:** demo requests per arm.\n**Secondary:** click-through rate.\n\n**Power caveat:** With roughly 100 contacts per arm and typical B2B click and request rates (a few percent), each arm will produce only a handful of clicks. The email test cannot reliably separate the options. Treat it as a sanity check: it shows whether any option is badly failing to get attention, and it breaks ties only if one arm is clearly ahead. Do not decide on it alone.\n\n## Test 2: Customer calls (days 5-14)\n\nRun 15 calls, about 1-2 per day. Target a mix of roles: about 8 facility managers or operators and about 7 sustainability leads. Include some prospects if possible, since existing customers already bought on a different message and may be biased.\n\n**Method:** Within-subject. Show all three options on separate cards, rotating the order across calls so each option appears first in about five calls.\n\n**Script (keep it to about 10 minutes):**\n\n1. Read card 1. Ask: \"In one sentence, what do you think this company sells?\" Record the answer.\n2. Ask: \"How much does this matter to your job, 1 (not at all) to 5 (a top priority)?\"\n3. Ask: \"How believable is this, 1 (not at all) to 5 (completely)? What would you need to see to believe it?\"\n4. Repeat steps 1-3 for cards 2 and 3.\n5. Ask: \"Which of these would make you stop at a booth or ask for a meeting? Why?\"\n6. Ask: \"Who else on your team would care about this one?\"\n\n**Record per call:** comprehension (the one-sentence answer matches the product, yes or no), relevance score, credibility score, final pick, and the reason given.\n\n## Scoring and decision rule\n\nApply these in order:\n\n1. **Gate:** Drop any option that fails substantiation.\n2. **Comprehension:** Drop any option where more than a third of calls describe the product incorrectly. This is the most likely failure for B.\n3. **Calls:** Pick the option chosen first most often across the 15 calls. Report the split by role. If the top pick differs between roles, lead with the one for the larger share of the conference audience and use the other as a secondary line.\n4. **Tiebreaker:** Use email demo-request rate, then average relevance score.\n\nIf no option wins clearly on the calls, pick A if it passes the gate. It is the most concrete and the easiest to defend at the booth.\n\n## Timeline\n\n- **Days 1-3:** Substantiate claims. Finalize the three cards and email variants. Write the call script. Book calls.\n- **Day 4:** Send the email split.\n- **Days 5-14:** Run calls. Track email results daily.\n- **Days 15-16:** Score and decide.\n- **Days 17-18:** Lock copy. Update the deck, booth banner, and talk slides.\n- **Days 19-21:** Buffer for print and final review.\n\n## Results template\n\n| Option | Gate passed | Comprehension (% correct) | Avg relevance | Avg credibility | First pick (of 15) | Email clicks | Email demo requests |\n|---|---|---|---|---|---|---|---|\n| A | | | | | | | |\n| B | | | | | | | |\n| C | | | | | | | |\n\n## Open questions for the team\n\n- Is the 15% figure from measured data? What is the sample?\n- Does the CRM have job title or role for the 300 contacts?\n- Who is on the 15-call list, and are any of them prospects?\n"}]}, "haiku-3": {"cost_usd": 0.00643, "seconds": 35, "output_tokens": 6379, "files": [{"name": "positioning_test_plan.md", "text": "# Gridline positioning test plan\n\n## Recommendation (prior, to be confirmed by the test)\n\nLead with **Option 1: \"Lower your building's energy bill by 15% without new hardware.\"**\nUse **Option 3** (\"Hit your 2030 emissions targets...\") as the supporting line or a second booth banner for sustainability leads. Drop **Option 2** for this audience.\n\nOption 1 is the most likely winner because it is concrete, names a cost outcome that facility managers own, and is understandable in about five seconds. Option 2 uses grid-industry language (\"flexible asset\") that this audience probably does not use. Option 3 is strong for sustainability leads but depends on the building owner's emissions targets, which facility managers may not own.\n\nThis is a prior, not a result. The tests below decide it.\n\n## Blockers to check before any option goes on stage\n\n- **The 15% claim.** Find out what the pilot data shows: average, median, or best case, and over what period and how many buildings. If there is no measured baseline, the claim is not ready for a conference, regardless of which option wins. A fallback is a sourced range or a \"up to\" qualifier, but that needs legal or marketing sign-off.\n- **\"Without new hardware.\"** Gridline connects HVAC and meters. Confirm whether any deployment needs a new meter, a gateway, or a controls upgrade. If some do, the phrase is false for those buildings and needs a qualifier.\n\n## Test design (3 weeks)\n\n### Week 1: Calls (15 conversations) \u2014 primary decision input for comprehension\n\nShow all three options to each person, in a randomized order per call (rotate so each option appears first about five times).\n\nAsk:\n1. \"Read these three. What do you think Gridline does?\" (Record their words, not yes/no.)\n2. \"Which one would make you want to hear more? Why?\"\n3. \"Which one is least relevant to your job? Why?\"\n4. \"What would you need to see to believe the first number?\" (Only if they pick Option 1.)\n\nRecord for each option: first-pick count, \"restated correctly\" count, and any confusion (e.g., \"flexible asset\" read as a financial asset).\n\nCaveat: 15 calls gives direction, not statistical proof. Count it as a strong signal only if one option is clearly first and is also restated correctly by most people.\n\nIf the 15 calls are with existing customers, comprehension will be biased by product knowledge. Note that in the results or recruit non-customers for part of the sample.\n\n### Weeks 1\u20132: Email test (300 past demo-requesters) \u2014 secondary, tie-break\n\n- Randomly split into three groups of 100. Stratify by firm and role if possible so each group has a similar mix of facility managers and sustainability leads.\n- Same email body in all three. The only change is the option as the subject line and opening line.\n- Send all three in one batch on the same day and time to limit timing effects.\n- **Primary metric: replies or meetings booked.** Not open rate, which is inflated by email privacy features.\n- **Secondary metric: clicks** to a short landing page or a demo link.\n\n**Power warning.** With about 100 per group, the test can only detect large differences. For a baseline reply rate of 5%, a difference is detectable only if one option reaches about 17% (roughly a 12-point gap). Typical differences between headlines will be smaller than that. A \"no clear winner\" result is likely and not a failure. In that case, the calls decide.\n\n### Week 3: Decide, then build booth copy\n\n## Decision rule (write it down before results come in)\n\n1. If one option wins the calls on first picks and \"restated correctly,\" and is not behind on replies, choose it.\n2. If the email test shows a large gap (roughly 10+ points) in the other direction, stop and check why before choosing. Look for list or segment bias first.\n3. If results are split or within noise, choose Option 1 and use Option 3 as the supporting line. Option 2 is dropped unless calls show strong demand from grid or utility-facing attendees.\n4. Do not choose any option that fails the blockers above, until the claim is verified or qualified.\n\n## Open questions\n\n- What is the conference date? That sets how much of the 3 weeks is usable.\n- Who holds the budget at the target firms: facility managers, sustainability leads, or asset managers? If it is mostly asset managers, Option 1's cost framing still likely works, but the call script should probe for that role.\n- Are the 15 calls with customers or prospects?\n- Is the 15% figure verified, and over what sample?\n"}]}}, "verdicts": [{"vs": "opus", "pair": "2", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "haiku+skill", "why": "A better grounds the audience tradeoffs, handles warm-sample limitations, and scrutinizes all promises. Both lack meaningful behavioral measurement. B overstates small-sample decisiveness, uses customer familiarity to assess comprehension, and recommends combining unverified savings and emissions promises."}, {"vs": "opus", "pair": "2", "judge": "gpt", "order": "opus shown first", "winner": "haiku+skill", "why": "B better handles warm-sample limitations, comprehension, and unsupported grid and emissions promises. A overstates what sparse votes establish and suggests combining unverified claims. Both lack a meaningful behavioral measure of qualified interest."}, {"vs": "opus", "pair": "2", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "tie", "why": "Both deliverables fully meet all criteria with rigorous, executable plans grounded in the input."}, {"vs": "opus", "pair": "2", "judge": "deepseek", "order": "opus shown first", "winner": "tie", "why": "Both provide excellent, factually grounded, executable plans with clear decision rules and audience focus."}, {"vs": "opus", "pair": "3", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "tie", "why": "A scrutinizes promises but omits behavioral testing, assignment controls, and a timely failure fallback. B offers stronger comparisons and action measures, but leaves hardware and emissions promises unsupported and inconsistently applies a 15-call checkpoint after 12 calls."}, {"vs": "opus", "pair": "3", "judge": "gpt", "order": "opus shown first", "winner": "opus", "why": "A offers stronger controlled comparisons, behavioral evidence, and fallback selection. Its call-count inconsistencies and unchecked hardware/emissions promises weaken it. B scrutinizes claims better but lacks behavioral testing, assignment controls, and a feasible inconclusive-result fallback."}, {"vs": "opus", "pair": "3", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "tie", "why": "Both provide grounded, executable, fair tests with strong measurement and clear provisional choices."}, {"vs": "opus", "pair": "3", "judge": "deepseek", "order": "opus shown first", "winner": "opus", "why": "A provides a concrete, executable test plan using all resources correctly, while B's plan is abstract and underpowered."}, {"vs": "opus", "pair": "1", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "haiku+skill", "why": "A better grounds the recommendation and scrutinizes unsupported promises, but depends on unavailable panel recruitment and lacks behavioral measurement. B offers a clearer schedule but adds unavailable noncustomers, prioritizes preference votes, and endorses unsubstantiated savings and emissions claims."}, {"vs": "opus", "pair": "1", "judge": "gpt", "order": "opus shown first", "winner": "haiku+skill", "why": "B better grounds the recommendation and scrutinizes unsupported promises, especially emissions. Its major flaw is dependence on unavailable panel recruits and no behavioral measure. A offers practical testing but prioritizes popularity and endorses unsupported emissions and softened savings claims."}, {"vs": "opus", "pair": "1", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "tie", "why": "Both plans are grounded, executable, fair, measure beyond popularity, recommend a provisional winner, and respect input facts."}, {"vs": "opus", "pair": "1", "judge": "deepseek", "order": "opus shown first", "winner": "opus", "why": "A provides a concrete, executable test plan using the given resources; B's plan relies on an unapproved external panel."}, {"vs": "haiku", "pair": "2", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "haiku+skill", "why": "A better connects product mechanics to audience needs, controls comparisons, acknowledges warm-sample limits, and scrutinizes all promises. Its main gap is behavioral measurement. B supplies demo-request tracking but underchecks hardware and emissions claims and relies heavily on customer preferences."}, {"vs": "haiku", "pair": "2", "judge": "gpt", "order": "haiku shown first", "winner": "haiku+skill", "why": "B better scrutinizes hardware, grid and emissions promises, acknowledges warm-sample limits, and prioritizes comprehension over preference. A includes behavioral testing but ultimately favors votes and incompletely verifies claims. B lacks a meaningful behavioral measure."}, {"vs": "haiku", "pair": "2", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "tie", "why": "Both provide excellent, executable plans grounded in product and audience, with fair comparisons and proper caution. No decisive difference."}, {"vs": "haiku", "pair": "2", "judge": "deepseek", "order": "haiku shown first", "winner": "tie", "why": "Both plans are excellent, fully meet all criteria, and are equally rigorous and actionable."}, {"vs": "haiku", "pair": "3", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "haiku", "why": "B offers an executable allocation, behavioral measures, and a clear fallback winner. A better grounds product fit and scrutinizes emissions, but omits assignment controls and behavioral selection evidence; its failed-clarity fallback risks exceeding three weeks. B insufficiently scrutinizes emissions promises."}, {"vs": "haiku", "pair": "3", "judge": "gpt", "order": "haiku shown first", "winner": "haiku", "why": "A offers a more executable, controlled test, behavioral evidence, and an inconclusive-results fallback. Its emissions and grid claims need scrutiny. B better explains product fit but omits assignment controls and behavioral measurement, with an uncertain fallback timeline."}, {"vs": "haiku", "pair": "3", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "haiku+skill", "why": "A's test plan is more rigorous, with better controls, diagnostic questions, and a clear decision rule."}, {"vs": "haiku", "pair": "3", "judge": "deepseek", "order": "haiku shown first", "winner": "tie", "why": "Both provide grounded, executable, fair tests measuring comprehension and action, with provisional recommendations and correct arithmetic."}, {"vs": "haiku", "pair": "1", "judge": "gpt", "order": "Haiku+skill shown first", "winner": "haiku", "why": "B offers a feasible email experiment, behavioral evidence, and a fallback choice. A makes unavailable cold-panel recruitment essential and excludes available emails from selection. B still requests unavailable prospects, overweights rankings, and insufficiently scrutinizes grid-flexibility promises."}, {"vs": "haiku", "pair": "1", "judge": "gpt", "order": "haiku shown first", "winner": "haiku", "why": "A offers a workable conference decision using mostly available access, but overweights preference votes and substitutes prospects for customer calls. B scrutinizes claims better, yet depends on unavailable panel recruits, lacks behavioral measurement, and offers no timely inconclusive fallback."}, {"vs": "haiku", "pair": "1", "judge": "deepseek", "order": "Haiku+skill shown first", "winner": "tie", "why": "Both plans are excellent, grounded, executable, and rigorous. They differ in approach but fully meet all criteria."}, {"vs": "haiku", "pair": "1", "judge": "deepseek", "order": "haiku shown first", "winner": "haiku", "why": "A provides a concrete, executable test plan using the given resources, while B's plan is less specific and introduces an unapproved panel."}]}]}