AWS Data Engineer

As a Senior Data Engineer, you will lead the design, build, and operation of scalable, secure, and high-performance data platforms. You’ll own end-to-end data pipelines, streaming platforms, and enterprise data stores, ensuring they meet business requirements, DDaT standards, and non-functional requirements. You will also provide technical leadership, mentor junior engineers, and contribute to the continuous improvement of engineering standards., * Design and deliver robust data architectures covering data ingestion, streaming, storage, processing, and consumption.

• Architect, develop and support real-time data pipelines using Apache Kafka.
• Design and optimise relational databases using AWS Aurora (Postgres).
• Build and maintain enterprise-grade data warehouse solutions and analytical data models.
• Deploy, manage and optimise containerised data services using Docker and Kubernetes.
• Design and implement scalable API-based data ingestion solutions integrating operational systems, SaaS platforms, and partner applications.
• Define conceptual, logical and physical data models while ensuring data quality, governance, and lineage.
• Collaborate with development and integration teams to define API contracts, schemas, versioning strategies, and resilient integration patterns.
• Optimise SQL performance, pipeline throughput, reliability, and scalability.
• Implement monitoring, alerting, and operational support for data platforms.
• Mentor junior engineers and provide technical leadership across the engineering team.
• Produce and maintain high-quality technical documentation aligned with DDaT standards.

• Expert SQL development and performance tuning
• Strong hands-on experience with Apache Kafka
• AWS Aurora (Postgres)
• Enterprise Data Warehousing
• Dimensional and Transactional Data Modelling
• Docker
• Kubernetes
• REST API Integration
• JSON and Schema Validation
• API resilience including retries, timeouts and idempotency
• Strong analytical, troubleshooting and problem-solving skills

Desirable Skills

• Experience with API Gateways or Integration Platforms
• OpenAPI / Swagger
• OAuth2 and token-based authentication
• Event-driven architectures combining APIs and Kafka
• Data observability and monitoring tools
• Experience working within Public Sector or regulated environments
• Knowledge of data governance, security and privacy principles
• Exposure to analytics and reporting platforms

Experience Required

• Significant experience in Senior Data Engineering roles
• Proven experience delivering scalable cloud-native data platforms
• Strong background in enterprise data architecture and engineering
• Experience leading technical delivery and mentoring engineers
• Experience working within Agile multidisciplinary teams
• Experience working within UK delivery environments

Ways of Working

• Agile, collaborative and multidisciplinary environment
• Strong ownership mindset with a focus on continuous improvement
• Active contribution to engineering standards and communities of practice

Security Clearance

Candidates must be eligible to obtain BPSS or SC Clearance, depending on client requirements., * AWS and Azure Cloud Platforms

• AWS Athena & Big Data Services
• AWS Cloud
• Data Engineering & Analysis
• Kafka and Message Queues
• PostgreSQL (Database)

• Modern technology stack including Kafka, AWS Aurora, Docker and Kubernetes
• Collaborative engineering culture with opportunities to influence technical decisions
• Initial 3-4 month contract with a strong likelihood of extension
• Hybrid working arrangement