AI Platform Engineer, Applied AI

• Design and build evaluation infrastructure to measure AI system speed, accuracy, and quality across both development and live production environments.
• Develop observability tools, dashboards, and monitoring systems that track performance metrics and surface actionable insights week over week.
• Diagnose quality gaps in AI systems, identifying root causes across retrieval, ranking, prompting, embeddings, and agent configurations.
• Prototype, test, and validate improvements to RAG pipelines, including chunking strategies, retrieval techniques, and re-ranking approaches.
• Run structured experiments and A/B tests to guide model selection and configuration decisions using data-driven methodologies.
• Analyze real-world user interactions with AI features to identify optimization opportunities and inform future product enhancements.
• Collaborate closely with engineering teams to ensure changes measurably improve AI system quality and reliability.
• Stay current with cutting-edge AI research, tools, and best practices, translating advancements into practical production improvements.

Requirements:6+ years of experience in software engineering, machine learning, or data science, with a strong foundation in search, retrieval, NLP, recommendation systems, or information extraction.Proven experience building and evaluating AI systems in production, including RAG pipelines, LLM-powered applications, and both offline and model-based evaluation frameworks.Strong proficiency in Python for experimentation, prototyping, and data analysis.Willingness to work with and integrate into Ruby on Rails production systems.Experience building infrastructure and tooling such as evaluation pipelines, dashboards, and data processing systems.Deep understanding of RAG architecture, embeddings, context management, and retrieval optimization techniques.Strong experimentation mindset with the ability to design, measure, and iterate quickly.Excellent analytical skills with the ability to interpret data, identify trends, and communicate findings clearly.Comfort working in a fast-paced, ambiguous environment that values autonomy, accountability, and collaboration.Professional-level English proficiency (CEFR C2 / ILR Level 5).Bonus: Experience with AI evaluation frameworks such as Braintrust, LangSmith, or similar tools.

$130,000 - $140,000 Vollzeit NEUHOME-OFFICE, Benefits:Competitive salary range of $130,000 - $140,000 USD per year, with annual compensation reviews.Equity participation in a high-growth startup environment.Fully remote work - operate from anywhere in the world.35 days of paid time off annually, plus paid sabbatical after 5 years.Comprehensive medical coverage (100% coverage for employees and families in applicable regions) or medical reimbursement options.Parental leave for growing families.Home office stipend to support your remote setup.Learning & development stipend for continuous professional growth.Annual bonus eligibility for applicable roles.Biannual, fully paid global company retreats in inspiring international locations.Why Apply Through Jobgether?We use an AI-powered matching process to ensure your application is reviewed quickly, objectively, and fairly against the role's core requirements. Our system identifies the top-fitting candidates, and this shortlist is then shared directly with the hiring company. The final decision and next steps (interviews, assessments) are managed by their internal team.We appreciate your interest and wish you the best! Why Apply Through Jobgether? Data Privacy Notice: By submitting your application, you acknowledge that Jobgether will process your personal data to evaluate your candidacy and share relevant information with the hiring employer. This processing is based on legitimate interest and pre-contractual measures under applicable data protection laws (including GDPR). You may exercise your rights (access, rectification, erasure, objection) at any time.