Building in Public #13: The Blind Taste Test
By Peter Coppinger
This one is a few days late: we had a great weekend in London.
Anyhow, here's the week.
The blind taste test
We built the same two apps on Lovable, Base44 and Vorx: a CRM for a small studio and an app for running an allotment. Then four AI judges who had never seen any of them ranked them blind.
They agreed on every app:
The blind taste test · 4 judges, unanimous on each app
Studio CRM
Lovable
Vorx
Base44
Allotment app
Lovable
Base44
Vorx
To be fair to Lovable, it won both, and it deserved to. On a checklist of the 16 features the two briefs asked for, Lovable had 16, Base44 had 15 and we had 11. Time from request to a ready app was 5 to 8 minutes on Lovable, 4 to 5 on Base44 and 6 to 8 on Vorx.
Our apps look comparable, but our home page has always lagged behind on design, and that's what people judge. Inside the app there were misses too: the sales pipeline was a bar chart instead of a board, the invoices had no status, the payments were labelled by an ID instead of a name, and an "Overdue" tile said 0 right beside a bar that said 3.
It was a small test, two briefs, and the judges could partly tell the entries apart because our screenshots were crisper than theirs. Lovable would still have won.
Which cells moved
Last week I published the Gap Map, Vorx against Lovable and Base44 on the same test project, and said I'd show you which cells moved.
The Gap Map · week 2 · 4 October
Vorx against Lovable and Base44. Open the details for every row, last week → this week.
Builds: making a new app
2/4
level with or ahead of the best two
Updates: changing an app you already have
2/5
level with or ahead of the best two
Details
Builds: making a new app
| Lovable | Base44 | Vorx: last wk → now | Status | |
|---|---|---|---|---|
| Time to a working app | 8–9 min | 4–8 min | 5½ min → about 4½–5 min | Level |
| Accessibility and phone layout | – | – | 0 failures → not re-measured | Ahead |
| First screen looks like your business (blind design rank, 1 = best) | 1.33 | 3.0 | 2.0 → 1.67 | Behind |
| Kinds of screens it can make | wide | wide | + shop, cart → + day-view calendar, occupancy grid | Behind |
Updates: changing an app you already have
| Lovable | Base44 | Vorx: last wk → now | Status | |
|---|---|---|---|---|
| App still works after 20 edits in a row | yes | yes | broke on edit 19 → all 20 ran, every page loads, 15 of 20 right (one run) | Behind |
| Tries the change before saying “done” | 0 of 20 | 9 of 20 | 1 of 2 → not re-measured | Behind |
| Says plainly what it didn’t check | 10 of 20 | – | fix in review → not re-measured | Behind |
| Time for a tiny edit | ~1 min | ~1½ min | 95 s → about 33 s live, about 1 min on our bench | Level |
| Reply to a tiny edit | short | short | 155 characters, not live → about 150–230, live | Level |
Lovable and Base44 weren’t re-measured this week except on design. Vorx build rows: rounds r323/r324, 30 September. Update rows: round 933 (3 October), rounds 992 and 997 (4–5 October) and production on 4 October. The 20-edit run used our low-cost settings.
The cell that moved most is design: on the next blind round we went from last of the three to second.
But after a week of marginal improvements, I decided we weren't going in the right direction. So I tried a new approach with engine 003. After two days I could see it wasn't going to get us there either, so now we're on engine 004.
After a week of marginal improvements, I decided we weren't going in the right direction... so now we're on engine 004.
Four build engines · the numbers only
No single test ran all four, so each number says which test it’s from: 12 business apps, or one job board app checked on 10 points. A dash means not measured. All bench builds.
Engine 001
The first pipeline
Customers’ engine until 29 September
Key issue: Too slow, about 6 minutes for a build.
Engine 002
The optimised pipeline
Customers’ engine now
Key issue: Designs too samey, and not as good as the competition.
Engine 003
The kit pipeline
Tried for two days, 3–4 October
Key issue: Slower than 002 and scored less than half.
Engine 004
The new hope…
From 5 October
Key issue: One day old, and not yet proven on production.
My bar for the job board: 8 or more out of 10, in under 4 minutes, on production. 004 is one day old: seven runs, each version better than the last. *The 10 checks got stricter on 5 October, after 002’s runs, so 002 hasn’t been scored against them yet.

I'm excited that 004 just might be the right approach. It's AI that builds the app with tools, a lot closer to how the best builders work. On its first day, its best run scored 8 out of 10 in just over 4 minutes. Its first runs on the live site haven't passed yet. One was interrupted by one of our own releases, and the other linked to a page that didn't exist and took about 5½ minutes. That's what this week is for.
Side project: an iPhone app for about $40
On the side, I used a spare laptop and some spare AI credits to build Vorx an iPhone app.
The main navigation and side menu are native, and the screens inside are web views. It’s the same approach we took with the Success.co app, except back then I paid a buddy a few thousand to build it. Claude did this one for about $40, plus some of my time and taste.
You can view your apps full screen, which is sweet. This weekend I gave the project view a cleaner design, added swiping between the chat and your app, and gave it a nicer icon than the white square it started with.
It’s turning into a really nice app, and I think customers will get loads of value from it. It’s lovely seeing Vorx as an icon on my phone.








Where that leaves me
I'll be honest, I'm a bit frustrated. I've gone down a few alleys this past while that looked promising and ultimately had to try another approach. I'd planned a 28 September release and we missed it. That was probably hopelessly optimistic, but that's my MO. And my own manual testing turned up a good few bugs and things I want polished before anyone else sees them.
My goal hasn't changed: better than the best.
This week that's all that matters. Get the 004 engine building at the level of the best two builders, and keep improving updates, which have come a long way.
Back to it.
Experience still counts
Even with AI doing the grunt work, you still need to know what good looks like.
I just found that we’d implemented caching on templates in a really stupid, slow, pointless way, instead of using a date-last-modified token in Redis. An amateur builder wouldn’t even know to add caching, let alone how to do it right.
It shows we can’t blindly trust AI. Everything must be validated.
P.S. This one on r/vibecoding gave me a giggle this week. If you've ever watched an AI chew through your token budget, you'll get it.
Stay updated: one founder, an army of AI agents, building this in public.
Get each new diary entry by email: the wins, the failures, and the one number we're chasing. No spam, unsubscribe any time.
Shipped this week 28 Sep – 4 Oct
- Feature: Delete my account is now a deliberate step: a Danger zone, the facts in the dialog, and your email typed to confirm.
- Feature: Settings → Embedding: choose which sites may show your published app in a frame.
- Feature: connect Figma from Settings → Integrations (beta).
- Feature: chat search now finds the text of every message in a turn.
- Feature: a 13th template category: Local business & services.
- Feature: rename your workspace and old links still redirect to it.
- Bug fix: a failed preview load no longer replaces a good one on screen.
- Bug fix: an idle preview tab no longer wakes its sleeping server.
- Bug fix: the Publish button can no longer spin forever.
- Bug fix: only an editor can change a project, and only its owner can change who sees it.
- Bug fix: no more "Revert" to the version already on screen.
- Bug fix: the sign-up form no longer jumps when the web address check appears.
- Bug fix: building cards are readable in every colour theme.
Comments