Cloud Data Engineer
Role details
Job location
Tech stack
Job description
We are seeking an experienced Cloud Data Engineer with expertise in AWS, Snowflake, dbt, PySpark, and SQL to design, build, and maintain scalable cloud-based data platforms and analytics solutions., * Design, develop, and maintain scalable data pipelines using AWS cloud services.
- Build and manage ETL/ELT workflows for data ingestion, transformation, validation, and reporting.
- Develop and maintain dbt models, tests, snapshots, macros, and documentation.
- Create high-performance data transformation solutions using PySpark and SQL.
- Design and optimize Snowflake data warehouse solutions.
- Integrate data from multiple sources including databases, APIs, cloud storage, and third-party systems.
- Implement data quality, reconciliation, audit controls, and exception handling.
- Optimize data processing jobs for performance, scalability, and cost efficiency.
- Develop CI/CD pipelines and automate deployment processes.
- Collaborate with architects, business analysts, QA teams, and stakeholders.
- Ensure data security, governance, compliance, and access controls.
- Support production deployment, monitoring, incident management, and troubleshooting.
- Prepare technical documentation and operational runbooks., * Banking
- Financial Services
- Payments
- Cards
- Loans & Mortgages
- Risk & Compliance
- Regulatory Reporting
- AML
- Fraud Analytics
- Customer Data Platforms
- Financial Reporting
- Data Governance
We are a Disability Confident Employer:
Capgemini is proud to be a Disability Confident Employer (Level 2) under the UK Government's Disability Confident scheme. As part of our commitment to inclusive recruitment, we will offer an interview to all candidates who:
- Declare they have a disability, and
- Meet the minimum essential criteria for the role.
Requirements
The ideal candidate should have strong experience in cloud data engineering, ETL/ELT development, data transformation, data warehousing, and performance optimization. Experience working in regulated industries such as Banking, Financial Services, Insurance, or Healthcare is preferred., Cloud & Data Engineering
- Strong hands-on experience with AWS cloud platform.
- Experience in designing cloud-based data platforms.
- Strong ETL/ELT development experience. Experience building enterprise-grade data pipelines.
Snowflake
- Strong experience with Snowflake Data Warehouse.
- Experience in Snowflake performance tuning.
- Knowledge of Snowpipe, Streams, Tasks, and Stored Procedures. Experience implementing RBAC and security controls.
DBT
- Hands-on experience with dbt Core or dbt Cloud.
- Development of reusable models, macros, tests, and snapshots.
- Data lineage, documentation, and dependency management. Integration with CI/CD pipelines.
PySpark & Programming
- Strong PySpark development expertise.
- Experience processing large-scale datasets.
- Code optimization and performance tuning.
- Data transformation, enrichment, aggregations, and joins.
SQL
- Advanced SQL skills.
- Stored procedures, CTEs, window functions, and query optimization.
- Data warehousing and dimensional modelling concepts.
AWS Skills Expected
Candidate should have experience with one or more of the following AWS services:
- Amazon S3
- AWS Glue
- Amazon EMR
- AWS Lambda
- AWS Step Functions
- Amazon Redshift
- Amazon Athena
- Amazon RDS / Aurora
- AWS IAM
- AWS KMS
- Amazon CloudWatch
- AWS Secrets Manager
Data Modelling Skills
- Star Schema
- Snowflake Schema
- Dimensional Modeling
- Data Vault
- Data Warehouse Design
- Data Lake Architecture
Data Formats
Experience working with:
- Parquet
- Avro
- JSON
- CSV
- ORC
CI/CD & DevOps
- Git
- GitHub
- GitLab
- Bitbucket
- Jenkins
- GitHub Actions
- GitLab CI/CD
- AWS DevOps Tools
Domain Experience (Preferred)