Otter Growth
    Advisory
    Case studies
    Managed marketplace for refurbished e-bikes

    Building a CRO Experimentation Program From Zero

    Stood up an A/B experimentation program from nothing, ran 30 experiments in three months at a 33% win rate, and compounded the winners into a 25% lift in seller lead conversion.

    Product
    Managed marketplace for refurbished e-bikes
    Model
    Buys, refurbishes, and resells e-bikes across three European markets
    Timeframe
    3 months

    Results

    +25%
    Compounded lift in lead conversion over three months
    33%
    Experiment win rate, 6 wins of 18 concluded
    30
    Experiments run in the first three months
    13 experiments read out in 12 weeks, a 33% win rate against a 20% target

    The situation

    A growth-stage marketplace with no experimentation capability, where the binding constraint was supply rather than demand.

    A seller funnel losing 85% before a lead, and two dashboards that disagreed on the rest

    What we did

      • Framed a conversion brief as a supply problem and descoped the demand storefront
      • Implemented A/B experimentation tooling (GrowthBook) alongside a component library, so tests could be built and shipped without bespoke engineering each time
      • Wrote the operating model the team runs
        • Experiment build guide and results guide
        • Pre-sprint ticket audit cadence
        • A definition of ready, so nothing enters a sprint half-specified
      • Set supply-side guardrails distinct from the demand funnel's, and held that line when seller-flow tests were judged on buyer metrics
      • Ran a competitive teardown across car, electronics, and bike resale marketplaces to source the hypothesis backlog
      • Reconciled two conflicting offer-acceptance rates into one number for the CEO
      • Held one significance bar for calling a test, so a flat result got recorded as flat rather than argued into a win
    Mapped the seven-step seller flow and marked where the sellers left
    Built the operating model the team runs without us

    Most teams that want a testing program buy a tool and stall, because the tool was never the constraint. The constraint is a backlog worth testing, a build path that does not need bespoke engineering per test, and an agreed bar for what counts as a win. Those three got built alongside the tooling, which is why the cadence held at four experiments every two weeks rather than spiking and dying. Thirty experiments ran in the first three months and eighteen concluded with a called result: six wins, four clear losses, eight that moved nothing. A third of concluded tests winning is roughly double what most programs report, and the four losses are the reason to trust the six wins. Each winner lifted seller lead conversion between 2.2% and 7.0% on its own surface, and because they ran in sequence, each measured against the baseline the previous winner had already set, they compound to about 25% across the three months. Two further tests shipped on their own merits while reading flat, which is a different decision and is recorded as one.

    Want results like this?

    We help consumer apps and products reach their next phase of growth.

    Related notes

    More case studies

    Growth notes, in your inbox

    New writing on subscription growth and product-led strategy when I have something worth saying. No spam.

    Otter Growth AdvisoryProduct growth for consumer apps and products