Cloud Data Engineer

Capgemini Sogeti
yesterday

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Working hours
Regular working hours
Languages
English

Job location

Remote

Tech stack

API
Amazon Web Services (AWS)
Amazon Web Services (AWS)
Business Analytics Applications
Data analysis
Big Data
Cloud Computing
Cloud Database
Cloud Storage
Program Optimization
Computer Programming
Databases
Continuous Integration
Customer Data Management
Information Engineering
Data Governance
ETL
Data Transformation
Data Security
Data Vault Modeling
Data Warehousing
DevOps
Dimensional Modeling
Github
Identity and Access Management
JSON
Key Management
Performance Tuning
Query Optimization
Role-Based Access Control
Standard Sql
Software Deployment
SQL Stored Procedures
SQL Databases
Parquet
Data Processing
Cloud Platform System
Macros
Data Ingestion
Sql Optimization
Snowflake
AWS Lambda
Gitlab
GIT
Amazon Web Services (AWS)
Data Lake
PySpark
Gitlab-ci
Data Lineage
Avro
Amazon Web Services (AWS)
Star Schema
Bitbucket
Cloudwatch
Data Pipelines
Amazon Web Services (AWS)
Jenkins
Redshift

Job description

We are seeking an experienced Cloud Data Engineer with expertise in AWS, Snowflake, dbt, PySpark, and SQL to design, build, and maintain scalable cloud-based data platforms and analytics solutions., * Design, develop, and maintain scalable data pipelines using AWS cloud services.

  • Build and manage ETL/ELT workflows for data ingestion, transformation, validation, and reporting.
  • Develop and maintain dbt models, tests, snapshots, macros, and documentation.
  • Create high-performance data transformation solutions using PySpark and SQL.
  • Design and optimize Snowflake data warehouse solutions.
  • Integrate data from multiple sources including databases, APIs, cloud storage, and third-party systems.
  • Implement data quality, reconciliation, audit controls, and exception handling.
  • Optimize data processing jobs for performance, scalability, and cost efficiency.
  • Develop CI/CD pipelines and automate deployment processes.
  • Collaborate with architects, business analysts, QA teams, and stakeholders.
  • Ensure data security, governance, compliance, and access controls.
  • Support production deployment, monitoring, incident management, and troubleshooting.
  • Prepare technical documentation and operational runbooks., * Banking
  • Financial Services
  • Payments
  • Cards
  • Loans & Mortgages
  • Risk & Compliance
  • Regulatory Reporting
  • AML
  • Fraud Analytics
  • Customer Data Platforms
  • Financial Reporting
  • Data Governance

We are a Disability Confident Employer:

Capgemini is proud to be a Disability Confident Employer (Level 2) under the UK Government's Disability Confident scheme. As part of our commitment to inclusive recruitment, we will offer an interview to all candidates who:

  • Declare they have a disability, and
  • Meet the minimum essential criteria for the role.

Requirements

The ideal candidate should have strong experience in cloud data engineering, ETL/ELT development, data transformation, data warehousing, and performance optimization. Experience working in regulated industries such as Banking, Financial Services, Insurance, or Healthcare is preferred., Cloud & Data Engineering

  • Strong hands-on experience with AWS cloud platform.
  • Experience in designing cloud-based data platforms.
  • Strong ETL/ELT development experience. Experience building enterprise-grade data pipelines.

Snowflake

  • Strong experience with Snowflake Data Warehouse.
  • Experience in Snowflake performance tuning.
  • Knowledge of Snowpipe, Streams, Tasks, and Stored Procedures. Experience implementing RBAC and security controls.

DBT

  • Hands-on experience with dbt Core or dbt Cloud.
  • Development of reusable models, macros, tests, and snapshots.
  • Data lineage, documentation, and dependency management. Integration with CI/CD pipelines.

PySpark & Programming

  • Strong PySpark development expertise.
  • Experience processing large-scale datasets.
  • Code optimization and performance tuning.
  • Data transformation, enrichment, aggregations, and joins.

SQL

  • Advanced SQL skills.
  • Stored procedures, CTEs, window functions, and query optimization.
  • Data warehousing and dimensional modelling concepts.

AWS Skills Expected

Candidate should have experience with one or more of the following AWS services:

  • Amazon S3
  • AWS Glue
  • Amazon EMR
  • AWS Lambda
  • AWS Step Functions
  • Amazon Redshift
  • Amazon Athena
  • Amazon RDS / Aurora
  • AWS IAM
  • AWS KMS
  • Amazon CloudWatch
  • AWS Secrets Manager

Data Modelling Skills

  • Star Schema
  • Snowflake Schema
  • Dimensional Modeling
  • Data Vault
  • Data Warehouse Design
  • Data Lake Architecture

Data Formats

Experience working with:

  • Parquet
  • Avro
  • JSON
  • CSV
  • ORC

CI/CD & DevOps

  • Git
  • GitHub
  • GitLab
  • Bitbucket
  • Jenkins
  • GitHub Actions
  • GitLab CI/CD
  • AWS DevOps Tools

Domain Experience (Preferred)

About the company

* You'd be joining an accredited Great Place to work for Wellbeing in 2024. Employee wellbeing is vitally important to us as an organisation. We see a healthy and happy workforce a critical component for us to achieve our organisational ambitions. To help support wellbeing we have trained 'Mental Health Champions' across each of our business areas, and we have invested in wellbeing apps such as Thrive and Peppy. * You will be empowered to explore, innovate, and progress. You will benefit from Capgemini's 'learning for life' mindset, meaning you will have countless training and development opportunities from thinktanks to hackathons, and access to 250,000 courses with numerous external certifications from AWS, Microsoft, Harvard ManageMentor, Cybersecurity qualifications and much more. * You will be joining one of the World's Most Ethical Companies®, as recognised by Ethisphere® for 13 consecutive years. We live our values by making ethical business choices every day. Working ethically is at the centre of our culture at Capgemini, meaning you will be helping to create a future we can all be proud of. Why you should consider Capgemini: Growing clients' businesses while building a more sustainable, more inclusive future is a tough ask. When you join Capgemini, you'll join a thriving company and become part of a collective of free-thinkers, entrepreneurs and industry experts. We find new ways technology can help us reimagine what's possible. It's why, together, we seek out opportunities that will transform the world's leading businesses, and it's how you'll gain the experiences and connections you need to shape your future. By learning from each other every day, sharing knowledge, and always pushing yourself to do better, you'll build the skills you want. You'll use your skills to help our clients leverage technology to innovate and grow their business. So, it might not always be easy, but making the world a better place rarely is. About Capgemini: Capgemini is an AI-powered global business and technology transformation partner, delivering tangible business value. We imagine the future of organisations and make it real with AI, technology and people. With our strong heritage of nearly 60 years, we are a responsible and diverse group of 420,000 team members in more than 50 countries. We deliver end-to-end services and solutions with our deep industry expertise and strong partner ecosystem, leveraging our capabilities across strategy, technology, design, engineering and business operations. The Group reported 2024 global revenues of €22.1 billion. Make it real | www.capgemini.com   You must create an Indeed account before continuing to the company website to apply

Apply for this position