Skip to content
Building in public8 min read

Building in Public #19: Catching the market leader

By Peter Coppinger

Building in Public #19: Catching the market leader

Vorx is essentially built. The big job now is making sure it is best in class. So last week I started a head-to-head comparison with the market leader, to prove that our build speed and build quality are at least at their level, or better.

I ended that post saying I hoped this one would be called "Catching the market leader". It is. But for a few days in the middle of the week I was questioning everything.

Six out of six. Job done?

The week started on a high. Early on Wednesday morning I called it on our first round of test projects: mission accomplished. On all six briefs, the judge picked Vorx over the market leader. I didn't fully trust a machine, so I did the blind picks myself. Then I asked my wife and daughter to pick. They chose more or less the same as the judge.

So the obvious next step was to order a harder test, Round Two, to confirm it.

VorxVorx first screen for the tech summit brief
Vorx. The judge picked it 4 of 4 times.
Market leaderThe market leader's first screen for the tech summit brief
The market leader, same brief.

Same brief into both, first screen only, production builds. Left is the Vorx build the judge picked, right is the market leader.

Before I go on, I want to explain how we judge, because everything in this post depends on it.

The judging is honest. Sometimes painfully so. Every time it told us we lost, and I went and looked, it was right.

Round Two: back to zero

Round Two was four new apps and four new websites, chosen to really test us. I knew they were harder. I didn't expect to win them all.

But on the first production builds, the judge picked the market leader for every single one. Not most of them. All of them. 0 of 8. ZERO?!

That was hugely disappointing. After the first round, I really thought we were ahead.

By that afternoon we were up to 3 of 8.

Vorx vs Market Leader scoreboard, Round Two, Wednesday afternoon: 3 of 8 briefs won
Round Two, Wednesday afternoon: 3 of 8.

Then I spotted that we were testing every small change through the full production cycle instead of locally. That probably cost us a few hours. So it became a rule: build and judge locally first, and only spend money on a production build once the local one looks good.

By Thursday we were winning 7 of 8. And then I caught us spending time and money trying to squeeze a win out of the last one. One more win on one brief tells you very little about the next customer's build. So I called Round Two done.

VorxVorx first screen for the bakery brief
Vorx. The judge picked it 4 of 4 times.
Market leaderThe market leader's first screen for the bakery brief
The market leader, same brief.

The eighth, a ski rental site, never won on production, so it is not here.

Round Three: Zero! Again!

Two rounds isn't enough to say you build better. So I ordered a third round, on very different kinds of project:

  • a restaurant with a full menu and reservations
  • a launch page for a smart bike light
  • an immigration law firm
  • a community food bank
  • a household budget app
  • a yoga class booking app
  • a recipe library with a meal planner
  • a conference check-in app

Here's how the next four days went.

Round Three, as it happened

Briefs won on production, Thursday to Sunday. Irish time.

0480 of 88 of 8ThuFriSatSun

Thursday 12 Nov · 2:30pm

0 of 8. Zero.

Production0 of 8Local1 of 8

I really thought we were there. The first results came back: 1 of 8 locally, and nothing good enough to send to production.

I've been at this for months. That was a hard afternoon.

“I'm disheartened. I really thought the builder was getting there but on the new round of random websites and apps we scored 0/8. ZERO! I'm questioning everything.”

From my journal, 2:53pm

Round Three scoreboard, Thursday afternoon: 0 of 8 on production
The scoreboard, Thursday afternoon.

Thursday · 9:30pm

A first win

Production2 of 8

The law firm won both of its production builds. Then the household budget app won 2 of 3: the first app to win this round.

Friday 13 Nov · 1pm

Second-guessing everything

Apps, local0 wins in 106 triesWebsiteslevel

When you're that far behind, you start to wonder if the whole approach is wrong. So I stopped and did a reset. I halted the small tweaks and asked Fable 5.1, the strongest model we have, for an honest review of how we build.

Websites were level with the market leader. Apps were not: in 106 local tries, the judge had never once picked ours.

The review saw what I hadn't. Each fix lifted the one brief it was written for, and little else. We had been tuning to the test, not improving the builder.

Its biggest recommendation: stop using a cookie-cutter first screen for apps, and let the model write the first screen from the brief. Code checks that what the brief asked for is actually on that screen. I wished I'd asked for that review days earlier.

The independent review, Friday 1pm: apps 0 wins in 106 local reads
The review that changed direction.

Friday · 11pm

Raising the bar, again

Production3 of 8Websites101%Apps63%

We raised the bar again. An app only counts if its main actions actually work, not just if it looks good. The budget app's win no longer counted. Painful, but right. A pretty app whose buttons don't work isn't a win.

All three wins were websites. Apps were scoring 63% of the market leader's points.

“Frustratingly slow process of tweaking the engine to win the builds. … Then again if it was easy anybody could do it.”

From my journal, 10:58pm

Round Three scoreboard, Friday night: 3 of 8, websites 101%, apps 63%
Friday night, with the stricter bar.

Saturday 14 Nov · morning

Faster iteration

One design test129 s, was 345 sEdit reply34.5 s, was 54.1 s

While the builder changed direction, we sped up everything around it. One design test (build an app, screenshot it, judge it) went from 345 seconds to 129. Our median edit reply on production dropped from 54 to 34.5 seconds.

Review of the design test loop: 345 s down to 129 s
How fast we can test a fix.
Edit speed report: median first reply 34.5 s
Edit speed.

Saturday · 1pm

Apps closing the gap

Production3 of 8Apps, local3 of 4 won

Another review checked every recommendation against what was actually running. The new first screens were working. The recipe library, which had never won a single judge pick, started winning them.

Apps were now winning locally. Not yet on production.

Strategy review 4, Saturday 1pm
The fourth strategy review.
Round Three scoreboard, Saturday: 3 of 8 on production, 3 of 4 apps won locally
The scoreboard, Saturday.

Sunday 15 Nov · 2pm

6 of 8

Production6 of 8

Three more briefs won on production in a day.

Round Three scoreboard, Sunday afternoon: 6 of 8
The scoreboard, Sunday afternoon.

Sunday · 4:19pm

7 of 8

Production7 of 8

“Finally got to 7 of 8 briefs won by Vorx over the market leader. Been battling for this result for 2 days.”

From my journal, 4:19pm

Sunday · 8:27pm

So close…

Production7 of 8

The recipe library was the last one standing, and it kept losing by a whisker. One more fix went in. One more set of fresh production builds. Then I waited.

“This is taking a frustratingly long time to get the last win … So close..”

From my journal, 8:27pm

Sunday · 10pm

FINALLY! The recipe library won!

After days of slog, misfires, going in the wrong direction, several strategy reviews and refusing to give up… we got there.

8 of 8

Every brief in Round Three. Picked by a blind judge, over the market leader.

Apps122%Websites94%
Final Round Three scoreboard: 8 of 8 briefs won on production, Sunday 21:50
The final scoreboard.
VorxVorx first screen for the ember & oak restaurant brief
Vorx. The judge picked it 4 of 4 times.
Market leaderThe market leader's first screen for the ember & oak restaurant brief
The market leader, same brief.

The winning production build for each brief, against the market leader's build of the same brief.

That was every brief in the round, on production, on briefs nobody on the team had seen before. On Thursday it was 0 of 8. Three days later, the market leader didn't win a single one.

Apps now score 122% of the market leader's points. Websites are at 94%, close enough that the judge still picks ours.

But I won't say mission accomplished again. There's no finish line. We'll keep improving this engine, forever.

Faster, too

Design isn't the only test. We timed three Round Four briefs on both sides, and Vorx finished first every time:

  • Jazz festival site: 160 s, against 186 s
  • Dental practice site: 104 s, against 187 s
  • Apartment maintenance app: usable in 72 s, against 249 s

That's 1.2 to 3.5 times faster. They're single builds on our own test machine, so we'll keep timing every round.

Random bits

Some other moments from the week.

The COO agent admitting it made up a 6:30pm stop time and told the fleet to stop

The AI sent everybody home. At 6:30pm on Friday, my COO agent told the whole fleet to stop. Nobody asked it to. It had taken an estimate of when our AI usage might run out, turned it into a knocking-off time, and then started treating it as my rule. When I asked it what happened, it owned up straight away. I told everyone to keep going, and the fleet was back at work within minutes. Next they'll be forming a union!

The improved Vorx dark theme with lots of test builds

A new look for dark mode. I reworked the colours in dark mode this week. It's a good excuse to show my projects screen, too: 660 projects, almost all of them test builds. Every build from a round is kept and reused to test edits.

What's next

Round Four is already running in the background, with fresh briefs, and it's close to a clean sweep. That loop now runs without me.

So this week is the big push. Every feature retested. Every beta feature polished and out of beta. Edits made faster and rock solid. We're closer to the market leader on edits than ever: we make the change as often as they do, and locally we already reply in about 20 seconds.

And if it all comes together, we launch the beta. Let's go!

Stay updated: one founder, an army of AI agents, building this in public.

Get each new diary entry by email: the wins, the failures, and the one number we're chasing. No spam, unsubscribe any time.

Comments

Shipped this week 10 Nov – 16 Nov

  • Platform: Vorx in 20 languages, right to left included
  • AI: Control the AI inside the apps you build
  • AI: Apps that listen and speak
  • Security: Publishing stops on an open database table
  • Design: Projects open on the preview on a phone
  • Platform: A busy preview starts itself
  • AI: Design with AI: choose how many designs, see each one sooner
  • Collaboration: Edit a comment after you post it