Data Engineer
Role details
Job location
Tech stack
Job description
The client is seeking a hands-on Sr. Data Engineer to design, build, and maintain scalable data ingestion pipelines and optimize a Snowflake-based data warehouse environment supporting procurement and supply chain data operations., *Design, build, test, and implement scalable data pipelines using Python and SQL *Maintain and optimize the organization's Snowflake data warehouse, including data ingestion and query performance tuning *Extract data from APIs using Python and AWS Lambda, and automate workflows using AWS Airflow *Perform analysis and critical thinking to troubleshoot data-related issues and implement data quality checks and scripts *Collaborate with data engineers and architects to develop new pipelines and optimize existing ones *Maintain code via CI/CD processes within an Azure DevOps platform *Support CDC and batch ingestion processes, including cutover activities to ensure seamless data continuity for downstream business users
Requirements
*7 years of experience in Data Engineering with a focus on building and implementing scalable data ingestion pipelines *Minimum of 10 years advanced proficiency in both Python and SQL, with correct syntax at a senior level *Minimum of 6 years of hands-on Snowflake experience, including data ingestion, architecture, and query performance optimization *Experience with AWS services including Lambda, Airflow, Glue, S3, and SNS *Familiarity with both CDC and batch processing approaches for data replication *Experience owning data pipelines end-to-end using Fivetran or a comparable data ingestion tool
Preferred Qualifications: *Experience with DBT for pipeline and data model development *Background supporting large-scale data migrations, including SQL Server cutover projects *Experience working within large enterprise environments *Strong communication skills with the ability to work autonomously, adapt to shifting timelines, and remain receptive to feedback *, *7 years of experience in Data Engineering with a focus on building and implementing scalable data ingestion pipelines *Minimum of 10 years advanced proficiency in both Python and SQL, with correct syntax at a senior level *Minimum of 6 years of hands-on Snowflake experience, including data ingestion, architecture, and query performance optimization *Experience with AWS services including Lambda, Airflow, Glue, S3, and SNS *Familiarity with both CDC and batch processing approaches for data replication *Experience owning data pipelines end-to-end using Fivetran or a comparable data ingestion tool
Preferred Qualifications: *Experience with DBT for pipeline and data model development *Background supporting large-scale data migrations, including SQL Server cutover projects *Experience working within large enterprise environments *Strong communication skills with the ability to work autonomously, adapt to shifting timelines, and remain receptive to feedback