Data Engineer

The Dmv
Conshohocken, United States of America
14 days ago

Role details

Contract type
Temporary to permanent
Employment type
Full-time (> 32 hours)
Working hours
Regular working hours
Languages
English
Experience level
Senior
Compensation
$ 177K

Job location

Remote
Conshohocken, United States of America

Tech stack

Query Performance
Artificial Intelligence
Airflow
Amazon Web Services (AWS)
Data analysis
Batch Processing
Big Data
Information Engineering
Microsoft SQL Server
Data Ingestion
Snowflake
Database Replication
Data Pipelines

Job description

The client is seeking a hands-on Sr. Data Engineer to design, build, and maintain scalable data ingestion pipelines and optimize a Snowflake-based data warehouse environment supporting procurement and supply chain data operations., *Design, build, test, and implement scalable data pipelines using Python and SQL *Maintain and optimize the organization's Snowflake data warehouse, including data ingestion and query performance tuning *Extract data from APIs using Python and AWS Lambda, and automate workflows using AWS Airflow *Perform analysis and critical thinking to troubleshoot data-related issues and implement data quality checks and scripts *Collaborate with data engineers and architects to develop new pipelines and optimize existing ones *Maintain code via CI/CD processes within an Azure DevOps platform *Support CDC and batch ingestion processes, including cutover activities to ensure seamless data continuity for downstream business users

Requirements

*7 years of experience in Data Engineering with a focus on building and implementing scalable data ingestion pipelines *Minimum of 10 years advanced proficiency in both Python and SQL, with correct syntax at a senior level *Minimum of 6 years of hands-on Snowflake experience, including data ingestion, architecture, and query performance optimization *Experience with AWS services including Lambda, Airflow, Glue, S3, and SNS *Familiarity with both CDC and batch processing approaches for data replication *Experience owning data pipelines end-to-end using Fivetran or a comparable data ingestion tool

Preferred Qualifications: *Experience with DBT for pipeline and data model development *Background supporting large-scale data migrations, including SQL Server cutover projects *Experience working within large enterprise environments *Strong communication skills with the ability to work autonomously, adapt to shifting timelines, and remain receptive to feedback *, *7 years of experience in Data Engineering with a focus on building and implementing scalable data ingestion pipelines *Minimum of 10 years advanced proficiency in both Python and SQL, with correct syntax at a senior level *Minimum of 6 years of hands-on Snowflake experience, including data ingestion, architecture, and query performance optimization *Experience with AWS services including Lambda, Airflow, Glue, S3, and SNS *Familiarity with both CDC and batch processing approaches for data replication *Experience owning data pipelines end-to-end using Fivetran or a comparable data ingestion tool

Preferred Qualifications: *Experience with DBT for pipeline and data model development *Background supporting large-scale data migrations, including SQL Server cutover projects *Experience working within large enterprise environments *Strong communication skills with the ability to work autonomously, adapt to shifting timelines, and remain receptive to feedback

Apply for this position