AWS Cloud & IDMC Data Engineer

Unitech
London, UK
about 2 months ago

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Experience level
Expert
Experience required
10 years minimum
Working hours
Regular working hours
Job source

Tech stack

Airflow Amazon Web Services Amazon S3 Application Integration Architecture Cloud Computing Cloud Database Information Engineering Data Governance Extract Transform Load (ETL) Data Retention Linux DevOps
+30 more
Disaster Recovery Distributed Computing Environment Information Lifecycle Management Data Intelligence Python (Programming Language) Meta-Data Management Cloud Services Standard Sql Shell Script SQL Databases Enterprise Data Management Data Logging Cloud Platform System Data Ingestion Informatica Powercenter System Availability Grafana State Machines Containerization Pyspark Semi-structured Data Kubernetes Infrastructure Automation Frameworks AWS Glue Enterprise Integration Functional Programming Cloudwatch Terraform Data Pipelines Service Stack

Job description

  • Design and implement scalable enterprise data and archival platforms using AWS and Informatica. * Build and maintain batch and near real-time data pipelines using AWS Glue, Lambda, PySpark, and IDMC services. * Develop data ingestion, transformation, validation, and archival workflows. * Lead technical architecture discussions and establish reusable engineering standards. * Support large-scale enterprise data migration and archival initiatives. * Improve platform reliability, scalability, observability, and operational support processes. * Integrate structured and semi-structured data from multiple enterprise systems. * Collaborate with infrastructure, security, network, DBA, and application teams. * Drive automation and Infrastructure as Code practices using Terraform and CI/CD pipelines. * Optimize data pipeline, storage, and query workload performance. * Mentor engineering teams and influence best practices across delivery programs. * Support governance, compliance, audit, retention, and data lifecycle management requirements. * Define operational processes including monitoring, recovery, and support readiness.

Requirements

Cloud and Data Engineering * Strong AWS experience (Glue, Lambda, S3, Athena, Step Functions, CloudWatch). * Strong Python and PySpark development experience. * Experience building scalable ETL and ELT pipelines. * Strong SQL and data modeling skills. * Experience with distributed data processing frameworks., * Strong experience with Informatica Intelligent Data Management Cloud (IDMC). * Experience with Cloud Data Integration, Mass Ingestion, Application Integration, Data Quality, and Cloud Data Governance. * Experience designing and supporting enterprise archival and data retention solutions using IDMC. * Experience integrating IDMC with AWS services and enterprise data platforms. * Understanding of metadata management, lineage, and governance. Platform and DevOps: * Experience with CI/CD pipelines and automation. * Infrastructure as Code using Terraform or similar tooling. * Experience with Linux and shell scripting. * Containerization and Kubernetes experience preferred. * Monitoring and logging experience with tools such as Grafana, ELK, or CloudWatch.Leadership and Delivery: * Strong stakeholder management and communication skills. * Experience leading technical delivery across multiple teams. * Strong troubleshooting and problem-solving skills. * Experience managing platform dependencies, operational risks, and delivery milestones. * Ability to define engineering standards and improve delivery processes. * Experience working in enterprise governance and controlled environments.Preferred Experience: * Enterprise archival and data retention platforms. * Banking, healthcare, telecom, or other regulated industries. * Data governance and compliance frameworks. * Large-scale platform modernization initiatives. * High-availability and disaster recovery environments. * Supporting operational and production platforms. Typical Technology Stack: AWS Glue, Lambda, S3, Athena, Step Functions, Python, PySpark, SQL, Informatica IDMC, Terraform, Airflow, Kubernetes, Linux.Ideal Candidate Profile * Ten or more years in data engineering and enterprise data platforms. * Strong mix of architecture, hands-on engineering, and operational support. * Experience designing scalable cloud-native data solutions. * Proven experience delivering enterprise integration and archival solutions. * Comfortable working in complex multi-team enterprise environments. * Ownership mindset with strong focus on reliability, scalability, and delivery quality.

About the company

Company Description UNITECH provides high-quality IT consulting and resourcing services to organizations seeking reliable technology expertise. The company focuses on delivering practical, scalable solutions that align with clients’ technical and business objectives. UNITECH partners with clients across industries to support digital transformation, cloud adoption, and data-driven decision-making. Professionals joining UNITECH can expect to work with modern technologies, diverse projects, and collaborative teams. The organization values technical excellence, professionalism, and a commitment to delivering measurable results.

Apply for this position

This job is hosted externally. Click below to view the full posting and apply.

Apply on apply4u.co.uk

Good distractions

Talks and stories from around this role — technically off-topic, practically not.

2:15 min

Empowering domain teams with an open data platform

Sandhya Menon Sandhya Menon · WWC Europe 2026

52 sec

Running persistent Linux environments directly on Windows

Ben Breard Ben Breard · WWC 2025

2:17 min

Mapping the maturity roadmap for scaled devops adoption

Dominik Krichbaum Dominik Krichbaum · WWC Europe 2026

3:03 min

Career evolution in data engineering and AI platforms

Maria Apazoglou · Coffee With Developers

3:05 min

Audience questions on AI agents and pipeline vectorization

Joy Joy · WWC 2024

3:18 min

Scaling global network engineering through DevOps culture

Stuart Clark · LIVE

Videos

See all

Related articles

See all