Data Engineer

We’re looking for a skilled Data Engineer to help design and deliver scalable data solutions that power analytics and business decision-making. This hybrid opportunity offers the chance to work with modern cloud data platforms, large-scale processing tools, and a collaborative team focused on building reliable, high-performing pipelines. Responsibilities

• Design, build, test, and maintain scalable ETL and ELT pipelines for large and complex datasets.
• Develop distributed data processing solutions using Apache Spark and Google Cloud Platform services.
• Create and support analytical data assets in BigQuery, with a focus on performance, cost efficiency, and reliability.
• Build batch and streaming data workflows using technologies such as Spark, Hadoop, and Kafka.
• Develop ingestion and transformation processes across source systems, data lakes, and analytics environments.
• Troubleshoot production data issues, identify root causes, and implement durable fixes.
• Put data quality, monitoring, logging, and alerting controls in place to support stable operations.
• Partner with architects, engineers, and business stakeholders to turn requirements into production-ready solutions.
• Contribute to code reviews, reusable frameworks, and engineering best practices.
• Apply software engineering and DevOps practices, including version control, automated testing, CI/CD, and deployment automation.

• Strong experience building production data pipelines in a data engineering environment.
• Hands-on ETL/ELT development experience.
• Practical experience with Google Cloud Platform.
• Solid BigQuery skills, including SQL development and query optimization.
• Experience working with Apache Spark and distributed data processing.
• Experience with GCP Dataproc.
• Exposure to Hadoop and/or Kafka.
• Strong SQL skills and the ability to work with large datasets.
• Proficiency in Python and/or Java/Scala.
• Understanding of data modeling, partitioning, and common data/file formats.
• Experience supporting production systems and resolving issues quickly and effectively.
• Familiarity with DevOps practices, Git, automated deployments, and CI/CD workflows.

Preferred Skills

• Experience with Looker and/or LookML.
• Knowledge of GCS, BigLake, Hive, Apache Iceberg, and Parquet.
• Exposure to Terraform or other Infrastructure-as-Code tools.
• Experience building Kafka-based streaming solutions.
• Background in migrating workloads from on-premises Hadoop/Hive environments to GCP.
• Experience supporting data lake or platform modernization initiatives.
• Familiarity with data quality, governance, lineage, and metadata management.

Horizontal is committed to fostering an inclusive, respectful, and equitable environment where people from all backgrounds can thrive. We value diverse perspectives and encourage candidates to apply even if they do not meet every preferred qualification.

Please apply through this online posting or by visiting our Job Board at www.horizontaltalent.com/job-board. Applications will be accepted for 4 weeks. For those that join the team, we offer competitive compensation and benefits including medical, dental, vision, and retirement. Check out all we have to offer and how you can become part of the Horizontal Talent Team. The pay range for this role is $37 - $71 per hour based on qualifications and experience. Engineering about 14 hours ago Development Engineer Horizontal Talent Brooklyn Park, Minnesota $49.00 - $85.00 per hour