Machine Learning Engineer

• Three or more years building machine learning systems that other people relied on
• Fluent Python and the modern ML stack: PyTorch or JAX, Hugging Face, experiment tracking
• Practical experience evaluating LLMs beyond a single aggregate score
• Able to explain a technical trade-off to a client who is not an engineer

Nice to have

• Worked under FedRAMP, HIPAA, or an equivalent compliance regime
• Published or presented on evaluation methodology
• Experience with retrieval systems on private corpora

Powered by JazzHR

Theory AI

• Fully paid medical, dental and vision
• 401(k) with employer contribution
• Unlimited paid time off with a four-week minimum
• $3,000 a year for conferences, courses and certifications
• Home office budget and a choice of hardware
• This is a Remote (work from home) position.

Most of the machine learning work that reaches production is not modelling. It is knowing which failures matter, building the evidence that shows whether they still happen, and being able to defend that evidence to someone whose job is to find the hole in it. You will own model development and evaluation on client engagements, largely in regulated settings: federal health, prime contractors, enterprises with data that cannot leave their boundary. That means working against real constraints on residency and access rather than a clean benchmark, and writing up what you found in a form a review board can read. This role suits someone who has shipped a model that other people depended on, and who has opinions about why aggregate accuracy is a poor summary of anything. What you will do

• Design and run evaluations for client models, including refusal behaviour and the failure modes a review board will test
• Build reproducible training and fine-tuning pipelines against regulated data
• Turn ambiguous client questions into measurable criteria before any modelling starts
• Write up methodology and findings for technical and non-technical audiences
• Review the ground-truth work our annotation teams produce, and feed problems back into the guidelines