Senior Software Developer (Python AI/ML)

This post sits within the Senior Software Engineer job family and provides specialist expertise in Python and AI/ML engineering day-to-day.

In this role, you will design, build, maintain and support robust software solutions that underpin our digital products and internal services. You will be responsible for developing and operating scalable data pipelines, APIs and cloud-native infrastructure, and for applying AI/ML techniques, including OCR, large language models and computer vision, to automate processes and improve efficiency.

Working across the full delivery lifecycle, you will contribute to discovery, design, implementation, testing, deployment and ongoing support. You will collaborate closely with multidisciplinary teams, ensuring solutions are secure, reliable, maintainable and aligned to architectural and engineering standards, while continuously improving performance and user outcomes., * Design, build, and operate scalable ETL and data pipelines handling structured and unstructured data for AI/ML workloads.

• Develop and maintain robust API services, including FastAPI, RESTful APIs, WebSockets, model-serving endpoints, integrating AI/ML capabilities with existing digital platforms.
• Implement authentication/authorisation using JWT, OAuth 2.0, API keys, and maintain API versioning and documentation.
• Deploy and operate cloud-native infrastructure using AWS Lambda, S3, RDS/Aurora, SQS, IAM, CloudWatch, with infrastructure-as-code tools: CDK, Terraform, CloudFormation.
• Containerize applications using Docker, orchestrate with Kubernetes (EKS/ECS), and maintain automated CI/CD pipelines.
• Implement monitoring and observability using CloudWatch, Grafana, telemetry frameworks, including experiment tracking tools like MLflow and Weights & Biases.
• Research, prototype, and implement AI/ML solutions using Transformers/Hugging Face, PyTorch, OpenCV, PIL/Pillow, YOLO, including LoRA/QLoRA fine-tuning, RLHF, and multi-modal AI/ML systems.
• Collaborate with team members to optimize platform and AI/ML workflow performance, reliability, and scalability.
• Ensure compliance with security, accessibility, performance, and operational standards.
• Participate in agile ceremonies, contribute to team knowledge-sharing, and support process improvements.
• Support disaster recovery procedures and maintain high-availability, resilient system standards, * Python 3.9+, object-oriented programming, async/await, decorators, context managers, structured logging, pytest, performance optimization.
• Data processing: Pandas, NumPy, SQL, SQLAlchemy/psycopg2, ETL orchestration (Apache Airflow, Dagster, Temporal.io).
• AI/ML frameworks: Transformers/Hugging Face, PyTorch, OpenCV, PIL/Pillow, YOLO; model fine-tuning (LoRA/QLoRA), RLHF, experiment tracking (MLflow, Weights & Biases).
• Web/API development: FastAPI, RESTful APIs, WebSockets, authentication/authorisation (JWT, OAuth 2.0, API keys), API versioning, documentation, model-serving endpoints.
• Cloud & DevOps: AWS Lambda, S3, RDS/Aurora, SQS, IAM, CloudWatch; infrastructure as code with CDK, Terraform, CloudFormation; Docker, Kubernetes (EKS/ECS); CI/CD pipelines.
• Monitoring & Observability: CloudWatch, Grafana, telemetry frameworks for production systems.
• System Design: Event-driven and microservices architectures, high availability, resilient systems, multi-modal AI/ML systems.
• Professional software engineering practices: Git workflows, unit/integration testing, code review, agile delivery (Scrum/Kanban).

Essential Experience

• Developing production-grade AI/ML and data platforms, ensuring reliability, maintainability, and performance for public sector services.
• Designing, building, and operating scalable ETL/data pipelines handling structured and unstructured data.
• Delivering secure, cloud-native AI solutions, integrating with existing infrastructure, managing lifecycle via IaC.
• Developing, supporting, and integrating APIs and microservices, including AI/ML model-serving endpoints.
• Deploying and operating containerized applications in production, with automated CI/CD and environment management.
• Implementing monitoring, alerting, and incident response processes for production systems, including AI/ML services.
• Applying professional software engineering practices collaboratively in multidisciplinary teams to deliver services iteratively.

Behaviours

Working Together

Works collaboratively within multidisciplinary teams, partnering with engineers, architects, and stakeholders to translate business and technical requirements into practical AI/ML solutions, sharing knowledge and contributing to collective delivery outcomes.

Managing a Quality Service

Takes responsibility for the design, delivery, and ongoing operation of secure, resilient, and high-performing AI/ML platforms, implementing monitoring, incident management, and continuous improvement to ensure services consistently meet user needs and agreed service levels.

Delivering at Pace

Plans and delivers complex AI/ML technical work across multiple priorities, using automation, CI/CD, and agile practices to release reliable platform capabilities quickly while managing risk and maintaining service stability., We'll assess you against these behaviours during the selection process:

• Delivering at Pace
• Managing a Quality Service
• Working Together, * A CV outlining your career history and how you meet the Essential criteria and the Technical Experience (max 4 pages).
• Complete the application questions that are related to the technical aspects of this position

Please note:

• If we receive a high volume of applications, we may complete an initial sift on Technical Experience.
• We reserve the right to invite candidates to participate in a telephone interview prior to being further assessed.
• Applications that are not accompanied by CVs will not be scored.
• We would strongly recommend that your statement is written in the STAR format (Situation, Task, Action & Result) and suggest preparing your answers using software such as MS Word or Google Docs, and then uploading the file.
• We strongly advise you review our policy on responsible use of AI in the application process. RoS may check answers with an AI detection tool and will contact you for a pre-screening call to verify your responses.
• Applications and appointments are subject to a strict merit-based assessment process, in line with the Civil Service Recruitment Principles.

Stage two – Virtual assessment

If your application is successful, you'll be invited to an online interview where you'll be assessed on the following essential criteria:

• Working Together (success profiles behaviour)
• Technical Experience

Candidates who are successful at this stage will progress to the final stage assessment in merit order, in line with the UK Civil Service recruitment principles.

Stage three – In person assessment

If successful at first stage assessment, you will be invited to an in-person interview which will include the following:

• Behaviour and experience interview
• Technical exercise assessment

Behaviour based interview questions will be given to candidates 15 minutes before the start of the interview to allow candidates to prepare in advance. You will be able to take short notes with you into the interview from your preparation., * UK nationals

• nationals of the Republic of Ireland
• nationals of Commonwealth countries who have the right to work in the UK
• nationals of the EU, Switzerland, Norway, Iceland or Liechtenstein and family members of those nationalities with settled or pre-settled status under the European Union Settlement Scheme (EUSS)
• nationals of the EU, Switzerland, Norway, Iceland or Liechtenstein and family members of those nationalities who have made a valid application for settled or pre-settled status under the European Union Settlement Scheme (EUSS)
• individuals with limited leave to remain or indefinite leave to remain who were eligible to apply for EUSS on or before 31 December 2020
• Turkish nationals, and certain family members of Turkish nationals, who have accrued the right to work in the Civil Service

Do you have experience in Terraform?, Successful candidates must undergo a criminal record check. People working with government assets must complete baseline personnel security standard (opens in new window) checks.

Alongside your salary of £60,291, Registers of Scotland contributes £17,466 towards you being a member of the Civil Service Defined Benefit Pension scheme. Find out what benefits a Civil Service Pension provides.

• Learning and development tailored to your role
• An environment with flexible working options
• A culture encouraging inclusion and diversity
• A Civil Service pension with an employer contribution of 28.97%