Data Engineer

We are hiring a Senior Data Engineer on behalf of our client, a large national retailer with a sophisticated technology organization supporting high-volume digital and enterprise operations. This is a hands-on, execution-focused opportunity supporting a Data Services Engineering team within a fraud and abuse technology environment. The Senior Data Engineer will build, modernize, and operate large-scale data pipelines and platforms across Hadoop and Google Cloud Platform (Google Cloud Platform). The ideal candidate is a strong individual contributor who can quickly become productive within an established architecture, independently own data engineering work from design through production support, and deliver reliable solutions for high-volume batch and streaming workloads. This is a 6-12 month contract position with potential for extension. The position follows a hybrid schedule with two days per week onsite, * Design, build, test, deploy, and support scalable batch and streaming data pipelines using Python, SQL, Apache Spark, Kafka, and related technologies.

• Support the modernization and migration of data pipelines and datasets from on-premises Hadoop environments to Google Cloud Platform (Google Cloud Platform), including BigQuery and other cloud-native services.
• Build and optimize ETL/ELT workflows for large-scale, high-volume datasets with an emphasis on performance, scalability, reliability, and cost efficiency.
• Develop trusted datasets supporting analytics, fraud investigation, reporting, and machine learning use cases.
• Implement appropriate data quality checks, testing, and validation processes.
• Troubleshoot production data issues, conduct root-cause analysis, and implement durable solutions to prevent recurring failures.
• Implement and maintain monitoring, alerting, automation, and CI/CD practices for production data pipelines.
• Work across both legacy Hadoop and modern Google Cloud Platform environments during ongoing platform modernization.
• Partner with engineers, analysts, data scientists, product teams, and other stakeholders to understand data requirements and deliver effective solutions.
• Identify technical risks, data quality concerns, performance issues, and opportunities to improve platform reliability.
• Contribute to technical design, code reviews, engineering standards, documentation, and knowledge transfer.

• 5+ years of hands-on data engineering or related software engineering experience building large-scale data systems.
• Strong proficiency in Python and SQL.
• Hands-on experience with Apache Spark and distributed data processing.
• Experience with the Hadoop ecosystem, including HDFS and Hive.
• Hands-on experience with Kafka or another event-streaming technology.
• Experience building and supporting ETL/ELT and data integration pipelines.
• Experience with Google Cloud Platform data services, particularly BigQuery.
• Experience optimizing data pipelines for performance, scalability, reliability, and cost.
• Experience supporting production data systems, including monitoring, logging, incident troubleshooting, and operational support.
• Working knowledge of Git, CI/CD, automated testing, and Agile engineering practices.
• Ability to independently own data engineering work from design through production deployment and ongoing support.
• Strong communication and collaboration skills with the ability to work effectively across technical and business teams., * Experience with additional Google Cloud Platform services such as Dataproc, Dataflow, Pub/Sub, or Cloud Storage.
• Experience migrating Hadoop workloads or enterprise data platforms to Google Cloud Platform or another public cloud environment.
• Experience with Spark Streaming, Flink, or similar stream-processing frameworks.
• Experience with data governance, lineage, security, privacy, and data quality frameworks.
• Experience with orchestration tools such as Airflow, Cloud Composer, Oozie, or similar technologies.
• Experience supporting machine learning data pipelines or Google Cloud Platform Vertex AI/MLOps workflows.
• Experience working with high-volume enterprise data in fraud, risk, security, retail, or e-commerce environments.

Dahl Consulting is proud to offer a comprehensive benefits package to eligible employees that will allow you to choose the best coverage to meet your family’s needs. For details, please review the DAHL Benefits Summary: .