Senior Data Engineer (Informatica)

We're seeking an SC cleared Senior Data Engineer who have Informatica skill to design, build, and operate data solutions that power mission critical analytics in a complex public sector environment. You'll lead on scalable pipelines (PySpark on Amazon EMR), modernise Legacy estates, and mentor engineers-turning raw data into reliable, secure, and actionable intelligence for stakeholders.

What you'll do Engineer production grade data pipelines on AWS (EMR, S3, Lambda), using PySpark/Python and SQL, with a focus on performance, resilience, testing, and observability. Migrate and modernise Legacy workloads (eg, ETL jobs and reporting feeds) onto cloud native services, creating reusable components and shared frameworks. Support reporting & MI use cases, including transformations and data models that feed downstream tools (eg, Power BI). Own CI/CD and version control practices (eg, Git/GitLab), review code, and enforce engineering standards. Coach and mentor engineers, provide technical guidance/code reviews, and contribute to architectural decisions across squads. Work in Agile delivery, collaborating across product, data, and platform teams using Jira/Confluence; translate requirements into robust engineering tasks. Embed security and compliance by design, aligning with BPSS/SC constraints and department data handling policies., AWS Certified Cloud Practitioner (or higher), Azure AI Fundamentals (awareness of ML/AI services). SFIA Level 4 (Enable) alignment Autonomy: Works under general direction; plans own work; designs and implements PySpark jobs on EMR, modernising Legacy code with minimal supervision. Influence: Shapes standards through code reviews and mentoring; influences delivery outcomes across teams. Complexity: Handles substantial, multifaceted engineering tasks (eg, migration to new MI platform; data quality resolution; estimating effort). Business skills: Communicates effectively with stakeholders; aligns data products to reporting/decision making needs; contributes to Agile ceremonies.

Hands on expertise in AWS & Spark: Amazon EMR, S3, Lambda; strong PySpark/Python and SQL for large scale batch processing. Data engineering at scale in government or similarly complex domains, including performance tuning and data quality management. CI/CD & DevOps: pipelines and IaC (eg, Terraform), automated testing, and release governance. Version control & collaboration: Git/GitLab, code review, branching strategies, and trunk/PR workflows. APIs & integration: building/consuming data services to move and expose data safely and reliably. Agile ways of working with Jira/Confluence; clear stakeholder communication and concise technical documentation. Security clearance: BPSS (minimum) and SC cleared or SC clearable for UK government work.

Desirable Data warehousing & modeling (eg, Redshift; dimensional modeling; dbt). Basic Power BI familiarity to partner with BI developers and validate end to end data flows. AWS ecosystem depth (Athena, Redshift, EC2, CloudWatch, IAM) and event driven patterns.