> Markdown version of [/jobs/ext/337949-aws-cloud-idmc-data-engineer](https://www.wearedevelopers.com/jobs/ext/337949-aws-cloud-idmc-data-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # AWS Cloud & IDMC Data Engineer - **Company:** Unitech - **Location:** London, UK - **Experience:** Expert - **Contract:** Permanent contract - **Skills:** Airflow, Amazon Web Services, Amazon S3, Application Integration Architecture, Cloud Computing, Cloud Database, Information Engineering, Data Governance, Extract Transform Load (ETL), Data Retention, Linux, DevOps, Disaster Recovery, Distributed Computing Environment, Information Lifecycle Management, Data Intelligence, Python (Programming Language), Meta-Data Management, Cloud Services, Standard Sql, Shell Script, SQL Databases, Enterprise Data Management, Data Logging, Cloud Platform System, Data Ingestion, Informatica Powercenter, System Availability, Grafana, State Machines, Containerization, Pyspark, Semi-structured Data, Kubernetes, Infrastructure Automation Frameworks, AWS Glue, Enterprise Integration, Functional Programming, Cloudwatch, Terraform, Data Pipelines, Service Stack - **Published:** June 15, 2026 - **Apply:** https://www.apply4u.co.uk/jobs/x/39019478/ ## About the Role Cloud and Data Engineering * Strong AWS experience (Glue, Lambda, S3, Athena, Step Functions, CloudWatch). * Strong Python and PySpark development experience. * Experience building scalable ETL and ELT pipelines. * Strong SQL and data modeling skills. * Experience with distributed data processing frameworks., * Strong experience with Informatica Intelligent Data Management Cloud (IDMC). * Experience with Cloud Data Integration, Mass Ingestion, Application Integration, Data Quality, and Cloud Data Governance. * Experience designing and supporting enterprise archival and data retention solutions using IDMC. * Experience integrating IDMC with AWS services and enterprise data platforms. * Understanding of metadata management, lineage, and governance. Platform and DevOps: * Experience with CI/CD pipelines and automation. * Infrastructure as Code using Terraform or similar tooling. * Experience with Linux and shell scripting. * Containerization and Kubernetes experience preferred. * Monitoring and logging experience with tools such as Grafana, ELK, or CloudWatch.Leadership and Delivery: * Strong stakeholder management and communication skills. * Experience leading technical delivery across multiple teams. * Strong troubleshooting and problem-solving skills. * Experience managing platform dependencies, operational risks, and delivery milestones. * Ability to define engineering standards and improve delivery processes. * Experience working in enterprise governance and controlled environments.Preferred Experience: * Enterprise archival and data retention platforms. * Banking, healthcare, telecom, or other regulated industries. * Data governance and compliance frameworks. * Large-scale platform modernization initiatives. * High-availability and disaster recovery environments. * Supporting operational and production platforms. Typical Technology Stack: AWS Glue, Lambda, S3, Athena, Step Functions, Python, PySpark, SQL, Informatica IDMC, Terraform, Airflow, Kubernetes, Linux.Ideal Candidate Profile * Ten or more years in data engineering and enterprise data platforms. * Strong mix of architecture, hands-on engineering, and operational support. * Experience designing scalable cloud-native data solutions. * Proven experience delivering enterprise integration and archival solutions. * Comfortable working in complex multi-team enterprise environments. * Ownership mindset with strong focus on reliability, scalability, and delivery quality. ## Description * Design and implement scalable enterprise data and archival platforms using AWS and Informatica. * Build and maintain batch and near real-time data pipelines using AWS Glue, Lambda, PySpark, and IDMC services. * Develop data ingestion, transformation, validation, and archival workflows. * Lead technical architecture discussions and establish reusable engineering standards. * Support large-scale enterprise data migration and archival initiatives. * Improve platform reliability, scalability, observability, and operational support processes. * Integrate structured and semi-structured data from multiple enterprise systems. * Collaborate with infrastructure, security, network, DBA, and application teams. * Drive automation and Infrastructure as Code practices using Terraform and CI/CD pipelines. * Optimize data pipeline, storage, and query workload performance. * Mentor engineering teams and influence best practices across delivery programs. * Support governance, compliance, audit, retention, and data lifecycle management requirements. * Define operational processes including monitoring, recovery, and support readiness. ## Related Videos - [From Messy Queries to Scalable Systems - How Data Engineering actually works](https://www.wearedevelopers.com/videos/100203-from-messy-queries-to-scalable-systems-how-data-engineering-actually-works) - [From DevOps to Scaled DevOps: How We’re Rebuilding Continuous Delivery as a Platform](https://www.wearedevelopers.com/videos/100018-from-devops-to-scaled-devops-how-we-re-rebuilding-continuous-delivery-as-a-platform) - [Docker network without Docker](https://www.wearedevelopers.com/videos/1418-docker-network-without-docker) - [Enjoying SQL data pipelines with dbt](https://www.wearedevelopers.com/videos/823-enjoying-sql-data-pipelines-with-dbt) - [#90DaysOfDevOps - The DevOps Learning Journey](https://www.wearedevelopers.com/videos/548-90daysofdevops-the-devops-learning-journey) - [AI Model Management Life Circles: ML Ops For Generative AI Models From Research to Deployment](https://www.wearedevelopers.com/videos/1152-ai-model-management-life-circles-ml-ops-for-generative-ai-models-from-research-to-deployment) ## Related Articles - [Making Data Warehouses Fast: A Developer’s Story](https://www.wearedevelopers.com/magazine/107-making-data-warehouses-fast-a-developer-s-story) - [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers) - [What Are The Top Skills Required For Azure Developers?](https://www.wearedevelopers.com/magazine/77-what-are-the-top-skills-required-for-azure-developers) - [Coffee with Developers - Maria Apazoglou - Making AI understandable for all in production](https://www.wearedevelopers.com/magazine/475-coffee-with-developers-maria-apazoglou-making-ai-understandable-for-all-in-production) - [7 Cloud Computing Trends Coming in 2025 for Developers](https://www.wearedevelopers.com/magazine/412-7-cloud-computing-trends-coming-in-2025-for-developers) - [Best US AI Conferences for CTOs in 2026: Build vs. Buy, Vendor Evaluation, and Peer Intelligence](https://www.wearedevelopers.com/magazine/736-best-us-ai-conferences-for-ctos-in-2026-build-vs-buy-vendor-evaluation-and-peer-intelligence)