Building a CRO Experimentation Program From Zero
Stood up an A/B experimentation program from nothing, ran 30 experiments in three months at a 33% win rate, and compounded the winners into a 25% lift in seller lead conversion.
- Product
- Managed marketplace for refurbished e-bikes
- Model
- Buys, refurbishes, and resells e-bikes across three European markets
- Timeframe
- 3 months
Results


The situation
A growth-stage marketplace with no experimentation capability, where the binding constraint was supply rather than demand.


What we did
- Framed a conversion brief as a supply problem and descoped the demand storefront
- Implemented A/B experimentation tooling (GrowthBook) alongside a component library, so tests could be built and shipped without bespoke engineering each time
- Wrote the operating model the team runs
- Experiment build guide and results guide
- Pre-sprint ticket audit cadence
- A definition of ready, so nothing enters a sprint half-specified
- Set supply-side guardrails distinct from the demand funnel's, and held that line when seller-flow tests were judged on buyer metrics
- Ran a competitive teardown across car, electronics, and bike resale marketplaces to source the hypothesis backlog
- Reconciled two conflicting offer-acceptance rates into one number for the CEO
- Held one significance bar for calling a test, so a flat result got recorded as flat rather than argued into a win




Most teams that want a testing program buy a tool and stall, because the tool was never the constraint. The constraint is a backlog worth testing, a build path that does not need bespoke engineering per test, and an agreed bar for what counts as a win. Those three got built alongside the tooling, which is why the cadence held at four experiments every two weeks rather than spiking and dying. Thirty experiments ran in the first three months and eighteen concluded with a called result: six wins, four clear losses, eight that moved nothing. A third of concluded tests winning is roughly double what most programs report, and the four losses are the reason to trust the six wins. Each winner lifted seller lead conversion between 2.2% and 7.0% on its own surface, and because they ran in sequence, each measured against the baseline the previous winner had already set, they compound to about 25% across the three months. Two further tests shipped on their own merits while reading flat, which is a different decision and is recorded as one.
We help consumer apps and products reach their next phase of growth.
Related notes
More case studies
A CRO Program That Consistently Beats Benchmarks
Building B2B Product-Led Growth From Zero
Nearly Tripling Paid LTV:CAC in Four Months
Scaling Ad Spend 3x and Reaching Profitability in Seven Months
Growth notes, in your inbox
New writing on subscription growth and product-led strategy when I have something worth saying. No spam.