Senior AI Engineer

Position Overview: We are seeking a Senior AI Engineer to build and support production AI solutions involving document intelligence, LLMs, retrieval, evaluation, and agentic AI. This is a hands-on engineering role for someone who has built, operated, measured, and improved AI systems in production, not someone whose experience is primarily research, notebooks, or proof-of-concepts.

The client operates a live document-intelligence platform: multimodal large language models extract structured records from high volumes of unstructured documents, hybrid retrieval reconciles those records against large reference catalogs, and human reviewers adjudicate low-confidence output through internal applications.

We are seeking one senior individual contributor to own the applied-AI and data-science layer end to end - extraction modeling, retrieval quality, and the evaluation and experimentation infrastructure that governs both.

• Expert Python: 5+ years of production Python experience, including services the candidate personally operated; experience with async development, typing, Pydantic, packaging, and profiling. * Production AI: Hands-on experience with both production document extraction and retrieval at scale. * AI Evaluation: Personally built an evaluation framework/harness for a non-deterministic system and used the results to make a meaningful production decision. * Measurement & Statistics: Strong understanding of hypothesis testing, confidence intervals, sample sizing, precision/recall/F1, accuracy, variance, and model calibration. * Dataset & Error Analysis: Experience with sampling, annotation, dataset versioning, label quality/agreement, and identifying performance issues hidden within specific data populations. * Production Engineering: Experience with event-driven architectures, idempotency, deduplication, replay/backfill, schema/API evolution, and safe rollout/rollback practices.
• Regulated Data: Experience appropriately handling sensitive/personal data, including classification, minimization, redaction, retention, least-privilege access, and secrets management. * Agentic AI / AI orchestration experience in production. * Experience managing LLM cost, token usage, and caching. * Online experimentation such as A/B testing, shadow deployments, or similar comparison approaches. * Drift detection and traditional ML experience. * Fine-tuning, adapter training, distillation, or systematic prompt optimization. * Large-scale analytical/data experience. * Experience with FERPA, HIPAA, GDPR, PCI-DSS, or SOX environments. * GCP and Google ADK experience would be a strong plus. * ML experience is beneficial. * Higher education experience is a plus, but not required.