Principal, Lead Engineer AI
Role details
Job location
Tech stack
Job description
The Principal, AI/ML Engineer leads the design, delivery and continuous improvement of AI and ML solutions within their area of responsibility, aligning engineering activities to the Bank's broader AI strategy and long-term objectives. The role combines strong technical leadership with a clear focus on delivery outcomes, ensuring solutions are scalable, reliable and aligned to business value.
The Principal operates with a clear understanding of the wider technology landscape, contributing to strategic direction while taking accountability for translating this into practical, high-quality engineering outcomes across squads.
-
Leads the definition and execution of AI/ML engineering direction across one or more squads, ensuring alignment with the Bank's AI strategic vision, as set out by the AI Capability Lead, and architectural principles.
-
Accountable for shaping and evolving technical approaches for AI/ML systems within their area, ensuring solutions are consistent with agreed patterns, standards and platforms.
-
Contributes to the development and adoption of reference architectures for key AI components, including LLMs, vector search and inference services, ensuring solutions are scalable and maintainable.
-
Leads technical discovery activities such as proofs of concept, feasibility assessments and vendor evaluations, translating findings into clear recommendations and delivery plans.
-
Supports design reviews and technical checkpoints for AI initiatives, ensuring risks are identified early and responsible AI considerations such as fairness, privacy and safety are incorporated into delivery.
-
Guides squads in adopting best practices in observability, incident management and service-level design to ensure reliable and resilient AI/ML services.
-
Works closely with Product Owners and Platform teams to prioritise and sequence backlogs, balancing delivery of business value with ongoing improvement and technical sustainability.
-
Mentors engineers within and across squads, contributing to the development of capability in data and ML engineering and fostering knowledge sharing.
-
Engages with senior stakeholders to communicate technical direction, delivery progress and key risks in a clear and structured manner.
-
Maintains an understanding of commodity AI solutions and ensures appropriate security and governance guardrails are applied when introducing new capabilities.
Software Design and Development
-
Leads the design and delivery of AI/ML solutions within their scope, ensuring high standards of performance, scalability and maintainability.
-
Designs and oversees implementation of end-to-end ML workflows, including data pipelines, feature engineering, model development and deployment, working closely with AI architecture roles to ensure alignment with enterprise patterns and standards.
-
Oversees the end-to-end AI/ML lifecycle within their scope, from prototyping through to deployment and optimisation.
-
Ensures that solutions consider performance, interpretability and appropriate use of AI, in line with organisational standards and expectations.
-
Promotes the adoption of modern AI techniques, including machine learning, natural language processing and advanced analytics, where appropriate to business needs.
-
Champions effective software engineering practices, including modular design, reuse of components and adherence to coding standards.
-
Identifies and drives opportunities to enhance existing solutions or introduce innovation through the application of emerging technologies.
Quality Assurance
-
Defines and embeds appropriate testing and validation approaches for AI/ML solutions, including model evaluation and performance testing.
-
Ensures that monitoring and benchmarking practices are in place so that models perform reliably and consistently in production.
-
Supports the implementation of processes to identify and address model drift, data issues and potential bias.
-
Promotes a culture of quality within squads, ensuring solutions meet business requirements and relevant regulatory standards.
-
Represents the team in internal and selected external technical forums where appropriate, contributing to knowledge sharing and continuous improvement.
Operations, Maintenance, Support and Documentation
-
Leads the implementation of MLOps practices within their area, supporting efficient model deployment, monitoring and lifecycle management.
-
Ensures that CI/CD practices are applied to AI/ML workflows to improve delivery speed and reliability.
-
Oversees the ongoing performance of production AI systems within their scope, addressing issues related to scalability and stability.
-
Ensures that appropriate documentation is in place for models, datasets and key technical decisions, supporting maintainability and transparency.
Data and Architecture
-
Contributes to the evolution of AI/ML architecture within their area, ensuring solutions are scalable, efficient and aligned with enterprise direction.
-
Guides the optimisation of data pipelines, feature stores and model serving approaches to support effective AI delivery.
-
Supports the evaluation and adoption of cloud-based AI/ML services, ensuring choices are aligned with technical and business requirements.
-
Ensures that ethical AI principles and security considerations are embedded in solution design and implementation.
Requirements
-
Holds a degree in Computer Science, Machine Learning, or a related technical field, or equivalent industry experience, with a strong focus on AI and ML systems.
-
Brings significant hands-on experience developing and delivering production-grade AI/ML solutions within cloud environments, ideally Azure.
-
Proven ability to design, implement and support resilient AI/ML solutions in production environments, ensuring reliability and scalability.
-
Demonstrates strong practical experience with Azure-based AI services, including Azure OpenAI, AI Search and AI Studio.
-
Well-versed in modern machine learning approaches, MLOps practices and cloud-based AI architectures.
-
Proficient in Python, common machine learning frameworks, distributed processing concepts and core MLOps practices.
-
Experience designing APIs, building microservices and implementing end-to-end ML pipelines on cloud platforms.
-
Demonstrates experience of cloud platforms such as Azure and AWS, with experience supporting and maintaining cloud-based infrastructure.
-
Hands-on experience working with:
-
LLM-based solutions, including Retrieval-Augmented Generation techniques and prompt engineering approaches
-
Data processing frameworks and platforms, including batch and streaming pipelines
-
Practical understanding of MLOps processes, including model lifecycle management, deployment approaches, monitoring and performance optimisation.
-
Experience supporting model serving, feature engineering and solution optimisation to meet performance and accuracy requirements.
-
Clear understanding of AI ethics, model governance and explainability principles, and their application in delivery.
-
Ability to contribute to technical direction and support alignment across teams, working effectively with Product Owners, architects and engineering leads.
-
Strong understanding of cloud security fundamentals, compliance considerations and cost awareness when delivering AI/ML solutions.
-
Fluent in spoken and written English, with an ability to work effectively across diverse and multicultural teams.
-
Able to communicate clearly with both technical and non-technical stakeholders, tailoring messages to the audience.
-
Confident in making informed technical decisions within their scope, considering delivery constraints, risks and longer-term implications.
-
Experience contributing to the adoption of coding standards, CI/CD practices and quality approaches within teams.
-
Experience contributing to technical documentation, knowledge sharing and internal communities of practice.
-
Demonstrates strong team leadership behaviours, supporting and mentoring engineers and contributing to a positive and collaborative engineering culture.
-
Experience working with data engineering processes, model training workflows and real-time or near real-time AI solutions.
-
Understanding of data governance practices and regulatory considerations relevant to AI/ML delivery.