Case study · E-commerce · ML & Data← All work

Bidding and marketing ML that added $40M a year in business volume

I worked on the paid-acquisition models of a European e-commerce unicorn: I improved the Google Shopping bidding algorithm, built the Google keyword campaigns system, and shipped a delivery-time model that lifted conversion.

Fig. 1ManoMano · Shopping bidding model, randomized A/B test

Controlbaseline

New model · +$30M/yr+7%

additional business volume
+$40M/yr
from the Shopping bidding model, in a randomized A/B test
+7%
conversion rate from realistic delivery times
+3%
ROI on the cost of the work
>60×

“As Data Science manager at ManoMano, I had the chance to manage Bryce for several years. [...] He successfully worked on complex and strategic data projects for the company. Focused on user impact and production, Bryce was always a driving force, bringing innovative yet pragmatic solutions across the entire value chain, from data collection to production, through Data Engineering and Data Science. Bryce has a rare multidisciplinary profile that gives him a precise understanding of the challenges of each data team (Engineering, Analytics, Science, MLOps, ...), and therefore lets him work with each of them.”

Romain Ayres

Romain Ayres

Head of Data Science at ManoMano

The problem

ManoMano is a DIY marketplace and one of Europe’s e-commerce unicorns. Google Shopping is its main source of traffic, with automated campaigns spending up to $1M a day, so a small gain in how that money is bid is worth millions a year.

What I built

Google Shopping bidding

The bidding model was already at v3 when I joined: a LightGBM ensemble trained on over a billion rows and 500 features, one row per day, product and device. I worked on it with a Senior Data Scientist. Its feature engine computed moving averages of past revenue in memory and crashed training whenever the data grew. I rewrote it to compute features in time chunks, so memory stays constant as the data grows, then added features (ensembles, new time windows, mixed granularities) after Data Analysis showed it would be impactful and fine-tuned the model.

Our work made the model spend more and earn more, enough to beat Target ROAS (tROAS), Google’s own bidding algorithm. In 2020, a randomized A/B test in France measured a statistically significant 7% uplift against tROAS, about $30M of extra business volume a year.

Keyword campaigns

I built the system that runs ManoMano’s Google keyword campaigns end to end. It finds keywords from Google’s APIs, ManoMano’s own search data and the business teams’ input, scores each one daily with a LightGBM revenue model, and sets budgets and parameters through the Google Ads API. New keywords are picked up every day as new products go on sale. The campaign spent about $1M a month, profitably.

Delivery-time estimates

An internal causal analysis showed delivery time drives conversion: a product that looks slow to arrive doesn’t get ordered. Comparing the promised delivery times with tracking history showed they were too pessimistic. I built a model that estimates delivery time from that history and put it into production. Delivery times dropped for 60% of products, and an A/B test measured a 3% uplift in conversion rate.

Production ML for the team

I also industrialized most of ManoMano’s ML models and coached the data scientists on DevOps and Software Engineering practices.

Results

  • About +$40M a year of business volume from the bidding and marketing ML.
  • A wider impact across teams: as the bridge between data science and data engineering, I helped most of ManoMano’s 10 ML models reach production, well beyond the projects I owned.
  • ROI above 60× on the cost of the work.

More work.

PropTech · AI Engineering

Matera: AI assistant searching 6M+ docs

Built an internal AI assistant from scratch: a chatbot on top of a knowledge search engine that retrieves the information employees need to do their work, across 6M+ documents.

Fig. 2Matera · questions asked per month since launch
×60

100h/daysaved across Matera's property managers

Fintech · Data Platform

Alma: Data platform

Led the build of Alma's data platform from scratch. From cloud infra to business dashboards.

Fig. 3Alma · what the platform connects

Sources

What it feeds

  • Postgres
  • HubSpot
  • Zendesk
  • Stripe
  • Adyen
  • Google Sheets
BigQuery · dbt
  • Dashboards
  • Fraud model
  • HubSpot
  • Zendesk
  • Customer.io

Postgres via CDC: freshness 24 h → 5 min

$50M/yrfraud savings enabled by the platform

Have a similar problem?

Bring the workflow and what's blocking it. In 20 minutes, you'll know whether AI can solve it, what to build first, and whether I'm the right person for it.

20 minutes · you leave with a clear next step

Book a free call