Data Engineer- Lead

Bridgetown Consulting Group

Malvern, PA, United States

3 months ago

Apply on dice.com

Prepare application

Role details

Contract type

Permanent contract

Employment type

Full-time (> 32 hours)

Experience level

Expert

Experience required

5 years minimum

Working hours

Regular working hours

Job source

dice.com

Tech stack

Agile Methodology Amazon Web Services Amazon Elastic Compute Cloud Amazon S3 Data Analysis JIRA Build Automation Big Data Computer Programming Continuous Integration Extract Transform Load (ETL) Distributed Systems

+17 more

Apache Hive Python (Programming Language) Cloud Services Amazon Simple Notification Service (SNS) SQL Databases Apache Spark State Machines Electronic Medical Records Git Pyspark Data Analytics Bitbucket Data Management Functional Programming Software Version Control Data Pipelines Atlassian Bamboo

Job description

Building and maintaining ETL pipelines, Data Analysis, Reimaging Technical Architecture for Analytics

Requirements

12+ years of experience

· Python, SQL, AWS Services

· Strong, hands-on Data Engineers

· Comfortable working end-to-end (ingestion, transformation, delivery)

· Hands-on experience with AWS cloud services, including EMR, S3, EC2, Lambda, SNS, Step Functions, Glue, and ECS, for building, orchestrating, and operating scalable data pipelines and data platforms.

· Strong programming expertise in Python and SQL, with hands-on experience in Big Data technologies such as Apache Spark, Spark SQL, and PySpark to efficiently process and transform large-scale, complex datasets in distributed environments.

· Experience working in an Agile, CI/CD-driven environment, leveraging tools such as Git, Bitbucket, Bamboo, and JIRA for version control, automated builds, deployments, and sprint execution.

· Minimum of five years data analytics, programming, database administration, or data management experience.

· Undergraduate degree or equivalent combination of training and experience.