AI / Eval Engineer

Our client is a small, fully remote team building technology that puts better clinical information in the hands of healthcare professionals — at a scale that belies the size of the team behind it. They've grown fast, they operate lean by design, and they're now building out the founding team that will shape the next decade of the company.

If you're the kind of person who can't leave a broken system alone, who goes three layers deeper than the obvious answer, who finds the pattern inside the failure — this was written for you.

What You'll Work On

• Own the System. Every layer is yours — model behavior, retrieval, ranking, the smallest parameters. You'll figure out which ones actually move the needle, and move them.
• Build the Feedback Loop. Measure what matters. Design the frameworks that tell you not just when the system fails, but why — and what it means for the next iteration.
• Run the Experiments. Break things on purpose. Test the edges. Come back with something better. Rinse, repeat, ship.
• Validate in the Real World. Work with leading healthcare institutions to stress-test performance across specialties, clinicians and clinical settings. See what holds up when it really counts.
• Work with the Founders. No middle layers. You'll work directly with the people building this — on the problems that matter most.

What You Bring

• Deep technical grounding in LLMs, search, or similar systems
• An obsession with understanding failures, not just fixing them
• The ability to move fast and think carefully at the same time
• Comfort in ambiguity — you make your own path when there isn't one
• High standards that don't slip when the pressure is on

Details Based in San Francisco. Remote considered for the right person, with relocation support available. Competitive salary, meaningful equity, full-time.

Interested? Apply and I'll be in touch if there's a fit. We work with a number of exciting clients across the industry — so even if this one isn't quite right, it is worth the chat!