Senior Data Engineer

Capgemini
New York, NY, United States
18 days ago

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Experience level
Expert
Compensation
$85,000.0 - $95,000.0
Working hours
Regular working hours
Job source

Tech stack

Application Programming Interfaces (APIs) Amazon Web Services Amazon S3 Unit Testing Cloud Engineering Code Review Computer Programming Continuous Integration Information Engineering Extract Transform Load (ETL) Data Migration Data Systems
+27 more
Data Warehousing Relational Databases File Systems Amazon DynamoDB Elasticsearch Identity and Access Management Systems Analysis Python (Programming Language) Network Attached Storage (Server Appliance) Shell Script Software Engineering SQL Databases Data Ingestion Apache Spark State Machines AWS Lambda Cloudformation Amazon Relational Database Service Data Lakes Information Technology AWS Data Analytics Apache Kafka SAP Ariba Api Gateway Amazon Simple Queue Service (SQS) Data Pipelines Serverless Computing

Job description

Designing, building and maintaining efficient, reusable, and reliable architecture and code. Build reliable and robust Data ingestion pipelines (within AWS, on-prem to AWS ,etc.) Ensure the best possible performance and quality of high scale data engineering project Participate in the architecture and system design discussions Independently perform hands on development and unit testing of the applications. Collaborate with the development team and build individual components into complex enterprise web systems. Work in a team environment with product, production operation, QE/QA and cross functional teams to deliver a project throughout the whole software development cycle. Responsible to identify and resolve any performance issues Keep up to date with new technology development and implementation Participate in code review to make sure standards and best practices are met.

Requirements

Bachelor’s degree in computer science, Software Engineering, MIS or equivalent combination of education and experience Experience implementing, supporting data lakes, data warehouses and data applications on AWS for large enterprises Programming experience with Python, Shell scripting and SQL Solid experience of AWS services such as CloudFormation, S3, Athena, Glue, EMR/Spark, RDS, Redshift, DynamoDB, Lambda, Step Functions, IAM, KMS, SM etc. Solid experience implementing solutions on AWS based data lakes. Should have good experience with AWS Services - API Gateway, Lambda, Step Functions, SQS, DynamoDB, S3, Elasticsearch Serverless application development using AWS Lambda Experience in AWS data lake/data warehouse/business analytics Experience in system analysis, design, development, and implementation of data ingestion pipeline in AWS Knowledge of ETL/ELT End-to-end data solutions (ingest, storage, integration, processing, access) on AWS Architect and implement CI/CD strategy for EDP Implement high velocity streaming solutions using Amazon Kinesis, SQS, and Kafka (preferred) Migrate data from traditional relational database systems, file systems, NAS shares to AWS relational databases such as Amazon RDS, Aurora, and Redshift Migrate data from APIs to AWS data lake (S3) and relational databases such as Amazon RDS, Aurora, and Redshift Implement POCs on any new technology or tools to be implemented on EDP and onboard for real use-case AWS Solutions Architect or AWS Developer Certification preferred Good understanding of Lakehouse/data cloud architecture

Benefits & conditions

Pulled from the full job description 401(k) Health insurance Paid time off Vision insurance Dental insurance Employee assistance program Disability insurance, The base compensation range for this role in the posted location is: $85,000 - $95,000

Capgemini provides compensation range information in accordance with applicable national, state, provincial, and local pay transparency laws. The base compensation range listed for this position reflects the minimum and maximum target compensation Capgemini, in good faith, believes it may pay for the role at the time of this posting. This range may be subject to change as permitted by law.

The actual compensation offered to any candidate may fall outside of the posted range and will be determined based on multiple factors legally permitted in the applicable jurisdiction.

These may include, but are not limited to: Geographic location, Education and qualifications, Certifications and licenses, Relevant experience and skills, Seniority and performance, Market and business consideration, Internal pay equity.

It is not typical for candidates to be hired at or near the top of the posted compensation range.

In addition to base salary, this role may be eligible for additional compensation such as variable incentives, bonuses, or commissions, depending on the position and applicable laws.

Capgemini offers a comprehensive, non-negotiable benefits package to all regular, full-time employees. In the U.S. and Canada, available benefits are determined by local policy and eligibility and may include:

  • Paid time off based on employee grade (A-F), defined by policy: Vacation: 12-25 days, depending on grade, Company paid holidays, Personal Days, Sick Leave

  • Medical, dental, and vision coverage (or provincial healthcare coordination in Canada)
  • Retirement savings plans (e.g., 401(k) in the U.S., RRSP in Canada)
  • Life and disability insurance
  • Employee assistance programs
  • Other benefits as provided by local policy and eligibility

Important Notice: Compensation (including bonuses, commissions, or other forms of incentive pay) is not considered earned, vested, or payable until it becomes due under the terms of applicable plans or agreements and is subject to Capgemini’s discretion, consistent with applicable laws. The Company reserves the right to amend or withdraw compensation programs at any time, within the limits of applicable legislation.

About the company

Choosing Capgemini means choosing a company where you will be empowered to shape your career in the way you’d like, where you’ll be supported and inspired by a collaborative community of colleagues around the world, and where you’ll be able to reimagine what’s possible. Join us and help the world’s leading organizations unlock the value of technology and build a more sustainable, more inclusive world.

Apply for this position

This job is hosted externally. Click below to view the full posting and apply.

Apply on www.indeed.com

Good distractions

Talks and stories from around this role — technically off-topic, practically not.

3:03 min

Career evolution in data engineering and AI platforms

Maria Apazoglou · Coffee With Developers

3:24 min

The governance failures of centralized data lakes

Mario Meir-Huber · LIVE

3:43 min

The enduring legacy of the amazon S3 storage API

Chris Heilmann +3 · LIVE

6:24 min

Distributed data lakes and containerized computing clusters

Ulrich Wurstbauer +1 · LIVE

2:56 min

Core data lake environment and infrastructure requirements

Christoph Fassbach Christoph Fassbach +1 · WWC 2024

1:43 min

AWS infrastructure stack and data flow pipeline overview

Artem Volk Artem Volk +1 · WWC 2024

Videos

See all

Related articles

See all