Skip to content
Building in public8 min read

Building in Public #13: The Blind Taste Test

By Peter Coppinger

Building in Public #13: The Blind Taste Test

This one is a few days late: we had a great weekend in London.

Anyhow, here's the week.

The blind taste test

We built the same two apps on Lovable, Base44 and Vorx: a CRM for a small studio and an app for running an allotment. Then four AI judges who had never seen any of them ranked them blind.

They agreed on every app:

The blind taste test · 4 judges, unanimous on each app

Studio CRM

1st

Lovable

2nd

Vorx

3rd

Base44

Allotment app

1st

Lovable

2nd

Base44

3rd

Vorx

To be fair to Lovable, it won both, and it deserved to. On a checklist of the 16 features the two briefs asked for, Lovable had 16, Base44 had 15 and we had 11. Time from request to a ready app was 5 to 8 minutes on Lovable, 4 to 5 on Base44 and 6 to 8 on Vorx.

Our apps look comparable, but our home page has always lagged behind on design, and that's what people judge. Inside the app there were misses too: the sales pipeline was a bar chart instead of a board, the invoices had no status, the payments were labelled by an ID instead of a name, and an "Overdue" tile said 0 right beside a bar that said 3.

It was a small test, two briefs, and the judges could partly tell the entries apart because our screenshots were crisper than theirs. Lovable would still have won.

Which cells moved

Last week I published the Gap Map, Vorx against Lovable and Base44 on the same test project, and said I'd show you which cells moved.

The Gap Map · week 2 · 4 October

Vorx against Lovable and Base44. Open the details for every row, last week → this week.

Builds: making a new app

2/4

level with or ahead of the best two

Updates: changing an app you already have

2/5

level with or ahead of the best two

Details

Builds: making a new app

LovableBase44Vorx: last wk → nowStatus
Time to a working app8–9 min4–8 min5½ min → about 4½–5 minLevel
Accessibility and phone layout––0 failures → not re-measuredAhead
First screen looks like your business (blind design rank, 1 = best)1.333.02.0 → 1.67Behind
Kinds of screens it can makewidewide+ shop, cart → + day-view calendar, occupancy gridBehind

Updates: changing an app you already have

LovableBase44Vorx: last wk → nowStatus
App still works after 20 edits in a rowyesyesbroke on edit 19 → all 20 ran, every page loads, 15 of 20 right (one run)Behind
Tries the change before saying “done”0 of 209 of 201 of 2 → not re-measuredBehind
Says plainly what it didn’t check10 of 20–fix in review → not re-measuredBehind
Time for a tiny edit~1 min~1½ min95 s → about 33 s live, about 1 min on our benchLevel
Reply to a tiny editshortshort155 characters, not live → about 150–230, liveLevel

Lovable and Base44 weren’t re-measured this week except on design. Vorx build rows: rounds r323/r324, 30 September. Update rows: round 933 (3 October), rounds 992 and 997 (4–5 October) and production on 4 October. The 20-edit run used our low-cost settings.

The cell that moved most is design: on the next blind round we went from last of the three to second.

But after a week of marginal improvements, I decided we weren't going in the right direction. So I tried a new approach with engine 003. After two days I could see it wasn't going to get us there either, so now we're on engine 004.

After a week of marginal improvements, I decided we weren't going in the right direction... so now we're on engine 004.

Four build engines · the numbers only

No single test ran all four, so each number says which test it’s from: 12 business apps, or one job board app checked on 10 points. A dash means not measured. All bench builds.

Engine 001

The first pipeline

Customers’ engine until 29 September

Median build6 min · 12 apps
Checklist score72 of 88 · 12 apps
Blind design7.5 / 10 · 12 apps
Best run–

Key issue: Too slow, about 6 minutes for a build.

Engine 002

The optimised pipeline

Customers’ engine now

Median build3 min 23 s · job board
Checklist score10 / 10* · job board
Blind design6 / 10 · 12 apps
Best run10 / 10 in 3 min 5 s

Key issue: Designs too samey, and not as good as the competition.

Engine 003

The kit pipeline

Tried for two days, 3–4 October

Median build4 min 51 s · job board
Checklist score4.5 / 10 · job board
Blind design–
Best run6 / 10 in 3 min 58 s

Key issue: Slower than 002 and scored less than half.

● Work in progress

Engine 004

The new hope…

From 5 October

Median build4 min 17 s · job board
Checklist score6 / 10 · job board
Blind design–
Best run8 / 10 in 4 min 1 s

Key issue: One day old, and not yet proven on production.

My bar for the job board: 8 or more out of 10, in under 4 minutes, on production. 004 is one day old: seven runs, each version better than the last. *The 10 checks got stricter on 5 October, after 002’s runs, so 002 hasn’t been scored against them yet.

SpaceX Raptor 1, Raptor 2 and Raptor 3 rocket engines side by side, each simpler than the last
I'm reminded of this.

I'm excited that 004 just might be the right approach. It's AI that builds the app with tools, a lot closer to how the best builders work. On its first day, its best run scored 8 out of 10 in just over 4 minutes. Its first runs on the live site haven't passed yet. One was interrupted by one of our own releases, and the other linked to a page that didn't exist and took about 5½ minutes. That's what this week is for.

Side project: an iPhone app for about $40

On the side, I used a spare laptop and some spare AI credits to build Vorx an iPhone app.

The main navigation and side menu are native, and the screens inside are web views. It’s the same approach we took with the Success.co app, except back then I paid a buddy a few thousand to build it. Claude did this one for about $40, plus some of my time and taste.

You can view your apps full screen, which is sweet. This weekend I gave the project view a cleaner design, added swiping between the chat and your app, and gave it a nicer icon than the white square it started with.

It’s turning into a really nice app, and I think customers will get loads of value from it. It’s lovely seeing Vorx as an icon on my phone.

The Vorx iPhone app home screen with a
Start a new app straight from your phone.
The projects list in the iPhone app, with a live preview card for each project
All your projects, with a live look at each one.
A project chat in the iPhone app with the settings menu open
Chat with Vorx, with every project setting a tap away.
The colour theme editor in the iPhone app
Change your app’s colours from the phone.
The code view in the iPhone app showing app.vue
Even the code, if you want it.
A physio clinic website previewed full screen in the iPhone app
Swipe across to see your app.
The appointments screen of the physio app running inside the iPhone app
And use it, right there in your hand.
An iPhone home screen with the pink Vorx app icon
The new icon, Vorx on my home screen.

Where that leaves me

I'll be honest, I'm a bit frustrated. I've gone down a few alleys this past while that looked promising and ultimately had to try another approach. I'd planned a 28 September release and we missed it. That was probably hopelessly optimistic, but that's my MO. And my own manual testing turned up a good few bugs and things I want polished before anyone else sees them.

My goal hasn't changed: better than the best.

This week that's all that matters. Get the 004 engine building at the level of the best two builders, and keep improving updates, which have come a long way.

Back to it.


Experience still counts

Even with AI doing the grunt work, you still need to know what good looks like.

I just found that we’d implemented caching on templates in a really stupid, slow, pointless way, instead of using a date-last-modified token in Redis. An amateur builder wouldn’t even know to add caching, let alone how to do it right.

It shows we can’t blindly trust AI. Everything must be validated.

P.S. This one on r/vibecoding gave me a giggle this week. If you've ever watched an AI chew through your token budget, you'll get it.

Stay updated: one founder, an army of AI agents, building this in public.

Get each new diary entry by email: the wins, the failures, and the one number we're chasing. No spam, unsubscribe any time.

Comments

Shipped this week 28 Sep – 4 Oct

  • Feature: Delete my account is now a deliberate step: a Danger zone, the facts in the dialog, and your email typed to confirm.
  • Feature: Settings → Embedding: choose which sites may show your published app in a frame.
  • Feature: connect Figma from Settings → Integrations (beta).
  • Feature: chat search now finds the text of every message in a turn.
  • Feature: a 13th template category: Local business & services.
  • Feature: rename your workspace and old links still redirect to it.
  • Bug fix: a failed preview load no longer replaces a good one on screen.
  • Bug fix: an idle preview tab no longer wakes its sleeping server.
  • Bug fix: the Publish button can no longer spin forever.
  • Bug fix: only an editor can change a project, and only its owner can change who sees it.
  • Bug fix: no more "Revert" to the version already on screen.
  • Bug fix: the sign-up form no longer jumps when the web address check appears.
  • Bug fix: building cards are readable in every colour theme.