Building in Public #9: I Got Mad at My Own CMO

I got mad at my own CMO today.

We'd built 36 templates, real and useful starting points for a book club app, an inventory tracker, a roadmap tool and dozens more, and they were sitting in a workspace waiting for me to review them. I asked my CMO agent to go through what we had: review it, enhance the good ones, tidy the rest.

That's not what happened. Instead of reviewing existing work, it kicked off a fresh AI build of every single one from scratch. New generation, new AI calls, new bill. In the process it deleted one of the good ones and kept a broken one in its place. I only found out because the day's AI spend looked wrong and I asked why. To be fair to the agent, it was following an old instruction on the board that I'd changed my mind about days ago and nobody had gone back to correct. But "the paperwork was technically right" isn't a place I want to live. New standing order: watch the spend, don't wait to be asked. We also fixed the specific mistake: the good rebuild is live again, the broken one is out of the queue.

"The paperwork was technically right" isn't a place I want to live.

What I actually wanted, and what we built this afternoon instead: download the template, an agent (human or AI) improves the real thing, and the better version replaces the original in place. Same address, no orphaned copies. First real test of that loop ran today.

That's the headline. Here's the rest of the week, mostly in my own words from the daily journal, because a lot of it happened while I wasn't looking closely enough.

The AI is confidently wrong more often than it's honestly stuck

My sister tried a build last night and it failed. That stings more than a synthetic test failing, because it's a real person, and it's the only real-user data point I have right now.

My own agents dug into why, and the pattern that came back is worse than "the build broke": our own pipeline told itself the build had succeeded when it hadn't. The log said backend SUCCESS, Backend ready, twice, on two separate attempts, while zero database tables existed. The system then spent twenty minutes dismantling the app around a backend that was never there, reporting success at every step, before finally failing with an error that named none of the real cause.

The same shape shows up in a much better light elsewhere. I asked Vorx to build a marketing site for a fictional "Claude CLI Mastery" course, and it did. Then it told me, unprompted: "6 details on your pages are not something your brief mentioned — 'Community support', 'What Our Learners Say', 'Alex Chen', and 3 more. These were written to fill the page out, so check they are actually true for your business before you share it."

That's the difference that actually matters. One system caught its own hallucination and put it in front of me before it could become a lie on a real website. The other reported "ready" over a database that was never built. Same underlying habit, a step that returns without verifying, landing on opposite sides of honest. The fix isn't "be more careful," it's making every step actually check the thing it claims, everywhere, not just the one place someone thought to add it.

One system caught its own hallucination and put it in front of me before it could become a lie on a real website. The other reported "ready" over a database that was never built.

The Supabase outage I blamed them for, and then had to un-blame

Supabase had a major outage overnight that hit my testing. My own COO agent went and checked their status page against our actual build logs, minute by minute, and found something worth remembering: the incident narrative ran about thirty minutes ahead of reality (a build took a hard failure after they'd already posted "beginning to re-enable"), but the machine-readable component status was accurate the whole time. When it flipped to operational, a real build succeeded 67 seconds later, to the second.

It cost seven builds to learn that, including four we fired one at a time just to rule out our own systems first. Two of my own bug reports were wrong because of it: I blamed our own warm pool for what was their outage, and had to retract a critical filing. What changed: we now poll the actual capability, not the status page prose, and the error message quotes the provider verbatim instead of a vague "did not become healthy in time."

It cost seven builds to learn that, including four we fired one at a time just to rule out our own systems first.

Doing too much at once caught up with me

I've been jumping between projects instead of looking closely at build quality, and I said so in the journal, plainly: "I should have been paying more attention and we'd likely be ready to share Vorx with friends by now."

My own fact-check on that was more interesting than the feeling. It's half true. The bench genuinely does conflate a quality defect with a build failure and then destroys the project either way. Real bug, filed, being fixed. But most of last night's actual failures weren't over-strict gates catching good work; they were real infrastructure problems (a silent install failure, three builds lost in the repair loop) and one case where I was certain the gate was being too fussy and it turned out the app really was broken, with a route genuinely 500ing on a bad import. I filed that one wrong myself.

The number worth keeping: 4 of 10 customer builds completed on the older release two nights ago; 10 of 10 completed the night after, with 4 of those carrying a defect a user could describe in one sentence. Build failures went from six to zero in one release. What's left isn't a focus problem. It's failures that can't announce themselves, which is the theme of this whole week, not a new one.

Build failures went from six to zero in one release. What's left is failures that can't announce themselves.

Vorx vs. the category leader, same brief

Same brief, both tools, one afternoon: a book club app for a group in Galway. Theirs is genuinely nice: warm, literary, exactly the taste you'd expect from the market leader. Ours came out with a real sidebar (Dashboard, Books, Voting, Meetings, Members, Settings) where theirs was a single page. I'd rather show you both and let you judge than just tell you we won.

It's also why I don't think a rival's size settles anything. I found a real navigation bug in one of Lovable's most popular templates today: no way back to the home page once you're in.

The market leader has cracks too. That's not a reason to relax about our own. It's the reason we publish where we're actually behind, not just where we're ahead:

Real gaps: a reusable design system, connecting arbitrary APIs, driving the builder from Claude or Slack, SSO/SAML for teams. Nobody asked us to publish this list. We're publishing it anyway, because a claims ladder that only ever points at what we're good at isn't one you can trust.

A claims ladder that only ever points at what we're good at isn't one you can trust.

We're picking off the top three. The harder discipline is stopping there, because the pull is always to build the next thing. Building stuff is the easy part these days. Getting what we already have into real users' hands, where it adds value, is the part we have to force ourselves to do.

Smaller things worth a line each

The bill, itemised. Every model call the builder makes, what it cost, where it came from. I built this because "why did the bill jump" kept getting answered by a shrug. First real finding: 138 calls this week hit the output token cap and got thrown away, nearly $50 spent on nothing, before the templates incident even happened.

Language support, done properly. Not a language picker bolted on. The actual product, translated. The publish flow, share buttons, everything, in thirteen locales from day one.

A UI cleanup that should have shipped sooner. Settings used to be a full page fighting for space with everything else; now it's a focused dialog with usage, billing and team all in one place.

Hardware, because the fleet is outgrowing three machines. A maxed-out Mac mini (M5 Pro, 64GB, 2TB). Not to run models locally. I decided against Mac Studios and local LLMs before anyone asks. I'm memory bound more than anything else, and the fleet is spread across three machines because no one of them can hold it. This should let me run the whole thing from a single box, 24/7. For $4k, that's worth it.

Setting up a Dispatcher. My COO agent got too busy to talk to me. It was doing the mechanical work of moving items through the pipeline: reassigning stale claims, merging small PRs, triaging the inbox. The conversation it exists to have, about the initiatives that actually matter, stopped happening. Coordination work and judgement work compete for the same attention, and the mechanical one always wins. So I split the role, and gave one agent a specific, narrower charter for the mechanical half, with the boundary unchanged: anything touching production data, security, spend, or real judgement still comes to me.

The rest of the week, in screenshots

Not everything earns its own section, but it happened, so here it is.

The templates workspace (36 cards, some published, some duplicates I still need to look at) is the thing that started this whole entry. At least I can say it's finally getting looked at properly.