Case study · Financial Services · AI Engineering← All work

A RAG system rebuilt from scratch, from 50% to 100% on the baseline eval

Opio's due-diligence platform answered half of its baseline questions correctly. I rewrote its RAG system from scratch, took it to 100%, and worked across the rest of the product: ingestion, the app and deployments.

Fig. 1Opio · the client's own 60-question eval

Before · 30 of 60 right

After · 60 of 60 right
accuracy on the 60-question baseline eval
50% → 100%

“I had the pleasure of working with Bryce for several months, and I can confidently say he's one of the most reliable and technically skilled developers I've worked with.

From the very start, he was immediately operational and quickly became a key contributor across multiple areas: frontend, backend, agentic systems, and CI/CD. His technical breadth is impressive, and he approaches every task with the same focus and enthusiasm, no matter the scope.

He's also an exceptional worker, dedicated, consistent, and someone you can absolutely rely on during intense periods or critical deadlines. When he's engaged in a project he enjoys, his commitment and motivation are truly remarkable.

On top of that, he's a great person to have on the team, kind, positive, and always easy to collaborate with.

In short, Bryce combines technical excellence, reliability, and great team spirit. I can't recommend him highly enough.”

Olivier Chance

Olivier Chance

CTO at Opio

The problem

Opio is a fintech start-up that had just raised $4M. Its platform helps transaction-services teams run financial due diligence: AI workflows read a company’s accounting data and deal documents and produce structured insights, each linked to its source.

On the baseline use case, the retrieval system behind those workflows answered only half of the client’s question set correctly. Opio needed it to be right, and as their CTO was busy working for an important client they needed an engineer who could also work across the rest of the product.

What I built

An eval system that runs on its own. Opio’s CEO had written a 60-question test set, but it was never run automatically. I built the eval system that runs it and scores the answers, so every change to the retrieval system is measured.

A RAG system rewritten from scratch. Chunking, embeddings, the retriever and the metadata system, all rebuilt to best practices.

Document ingestion that keeps up. I worked on the data engineering workflows so the document ingestion pipeline - which was needed for the retrieval system to work - runs incrementally and fast.

A feature that helped win a deal. I contributed to the full-stack app, including a critical feature that helped Opio land an important lead: it automatically maps French FEC accounting files (the standard export of a company’s general ledger) to business categories, so the cash flow can be read category by category.

Safe, automated deployments. I put industry-standard DevOps practices in place, so deployments became automated, fast and safe.

How we knew it worked

The rewrite was measured against Opio’s baseline use case: the CEO’s 60-question set, run by the eval system I built. Accuracy went from 50% to 100%.

A perfect score on 60 questions is a start, not a proof. The next step is a larger test set and new eval sets, to show the system holds on questions it hasn’t seen.

Results

  • Accuracy on the baseline eval went from 50% to 100%, and the system is in production.
  • The 60-question set runs automatically, so every change is measured.
  • Document ingestion is incremental and fast.
  • Deployments are automated, fast and safe.

Who runs it now

Opio had just raised its round, and I was the first engineer to help the CTO build the product. Once Opio’s internal team was ready, I handed everything over to them.

More work.

E-commerce · ML & Data

ManoMano: E⁠-⁠commerce optimization

Worked on the Google Shopping bidding algorithm and built the keyword-campaign system, alongside a delivery-time model, for a European e-commerce unicorn.

Fig. 2ManoMano · Shopping bidding model, randomized A/B test

Controlbaseline

New model · +$30M/yr+7%

+$40M/yradditional business volume from the bidding & marketing ML

PropTech · AI Engineering

Matera: AI assistant searching 6M+ docs

Built an internal AI assistant from scratch: a chatbot on top of a knowledge search engine that retrieves the information employees need to do their work, across 6M+ documents.

Fig. 3Matera · questions asked per month since launch
×60

100h/daysaved across Matera's property managers

Have a similar problem?

Bring the workflow and what's blocking it. In 20 minutes, you'll know whether AI can solve it, what to build first, and whether I'm the right person for it.

20 minutes · you leave with a clear next step

Book a free call