AI Engineer RAG Platform

Voor Rabobank zoeken wij een AI Engineer RAG Platform. Do you want to improve a strategic AI platform through data-driven experimentation and measurable model improvements? Your role As an AI Engineer specialising in RAG platforms, you continuously improve the quality, accuracy and effectiveness of an internal Retrieval-Augmented Generation (RAG) platform. You work at the intersection of information retrieval, data science and software engineering. Your focus is improving the performance of the RAG system itself-not building AI applications or platform infrastructure. The platform serves thousands of users and supports multiple AI-powered products. You investigate where and why the system underperforms, design experiments to validate improvements and translate your findings into measurable results in production. You contribute throughout the improvement lifecycle, from analysing failures and developing hypotheses to evaluating, implementing and monitoring solutions. You use user feedback, retrieval metrics, evaluation datasets and production telemetry to improve answer quality and user trust. You work closely with software engineers, platform engineers, data scientists and product managers, and drive the continuous improvement of the platform’s RAG capabilities. What you’ll work on

• Investigate why certain user questions produce lower-quality answers and identify underlying retrieval or grounding issues.
• Improve automated evaluation frameworks to measure retrieval quality, answer quality and user satisfaction.
• Run large-scale experiments to assess the impact of different chunking, embedding and retrieval strategies.
• Analyse user interactions and system performance data in Databricks to identify opportunities for improvement.
• Build feedback loops that identify weak responses and feed insights into the development process.
• Define and monitor quality metrics so that releases lead to measurable improvements.

Your responsibilities

• Improve retrieval performance, answer quality and user experience through experimentation, analysis and optimisation.
• Develop automated frameworks to measure RAG quality, identify regressions and provide actionable insights.
• Turn data analysis and experiment results into scalable, maintainable engineering solutions with measurable business value.

Your team You’ll join a multidisciplinary team of experienced engineers building and operating critical AI capabilities. The RAG platform is a production product that must continuously improve. You’ll combine scientific rigour with engineering excellence, challenge assumptions, validate ideas with data and help raise the quality bar. Your profile You enjoy solving complex problems and focus on measurable outcomes. You’re comfortable discussing statistical significance, retrieval metrics and system architecture. You take ownership of results, and you’re naturally curious, analytical and motivated to improve complex systems., * Je beheerst het Engels goed, zowel mondeling als schriftelijk.

• Je hebt ervaring met RAG.
• Je hebt ervaring met Large Language Models (LLM’s).
• Je beschikt over Python-vaardigheden.
• Je beschikt over Databricks-vaardigheden.
• Je hebt ervaring met evaluatieframeworks voor GenAI-systemen.
• Je hebt ervaring met embeddingmodellen.
• Je hebt ervaring met promptoptimalisatie.
• Je hebt ervaring met datapipelines en workflowautomatisering.
• Je hebt ervaring met CI/CD-praktijken.
• Je hebt ervaring met Azure.
• Je hebt een masterdiploma of een gelijkwaardig academisch niveau in Data Science, Artificial Intelligence, Computer Science, Mathematics, Statistics of een verwant vakgebied.
• Je hebt minimaal 3 jaar relevante ervaring in machine learning, toegepaste AI of data science.
• Je hebt aantoonbare ervaring met het verbeteren van productiewaardige RAG-systemen of geavanceerde AI-oplossingen op basis van retrieval.
• Je hebt ervaring met het ontwerpen van evaluatiemethodologieën en kwaliteitsmetingen.
• Je hebt ruime ervaring met het omzetten van analyses naar geautomatiseerde productieoplossingen.
• Je hebt ervaring met grote datasets en complexe gedistribueerde omgevingen.
• Je kunt bevindingen en aanbevelingen duidelijk overbrengen aan zowel technische als niet-technische stakeholders.

Functie-wensen

• Het is een pre als je ervaring hebt met het finetunen van taalmodellen.
• Het is een pre als je ervaring hebt met rankings- en retrievalsystemen.
• Het is een pre als je ervaring hebt met AI-observability en monitoring.
• Het is een pre als je ervaring hebt met grootschalige AI-platformen in productie.
• Het is een pre als je kennis hebt van synthetische evaluatiedatasets en het opstellen van benchmarks.

• Strong experience with Retrieval-Augmented Generation (RAG) and large language models (LLMs).
• Proficiency in Python and experience with Databricks.
• Experience with evaluation frameworks for generative AI systems, embedding models and prompt optimisation.
• Knowledge of data pipelines, workflow automation, CI/CD practices and Azure.
• A master’s degree or equivalent academic background in data science, artificial intelligence, computer science, mathematics, statistics or a related field.
• At least three years of relevant experience in machine learning, applied AI or data science.
• Demonstrable experience improving production-grade RAG systems or advanced retrieval-based AI solutions.
• Experience designing evaluation methodologies and quality metrics.
• A strong track record of turning analysis into automated production solutions.
• Experience working with large datasets and complex distributed environments.
• The ability to communicate findings and recommendations to technical and non-technical stakeholders.
• Fluency in written and spoken English.

It’s an advantage if you have

• Experience fine-tuning language models.
• Experience with ranking and retrieval systems.
• Experience with AI observability and monitoring.
• Experience working with large-scale production AI platforms.
• Knowledge of synthetic evaluation datasets and benchmark creation.