> Markdown version of [/jobs/ext/3091525-data-engineer](https://www.wearedevelopers.com/jobs/ext/3091525-data-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Data Engineer - **Company:** Horizontal Talent - **Location:** Brooklyn Park, MN, United States - **Salary:** $76,960.0 - $147,680.0 - **Contract:** Permanent contract - **Skills:** Java (Programming Language), Business Analytics Applications, Apache HTTP Server, Automation of Tests, Big Data, BigQuery, Code Review, Continuous Integration, Data Validation, Data Files, Extract Transform Load (ETL), Data Systems, Database Queries, DevOps, Distributed Computing Environment, Apache Hadoop, Apache Hive, Python (Programming Language), Meta-Data Management, Operational Databases, Cloudera, Software Construction, Data Streaming, Parquet, Data Logging, Data Processing, Google Cloud, Cloud Platform System, Apache Spark, Software Troubleshooting, Git, Data Lakes, Infrastructure Automation Frameworks, Deployment Automation, Apache Kafka, Terraform, Looker Analytics, Software Version Control, Data Pipelines - **Published:** September 26, 2026 - **Apply:** https://www.jofdav.com/jobs/59871047-data-engineer ## About the Role * Strong hands-on experience building production-grade data pipelines. * Solid ETL/ELT development experience. * Practical experience with Google Cloud Platform. * Strong SQL skills, including query development and optimization in BigQuery. * Hands-on experience with Apache Spark and distributed data processing. * Experience working with GCP Dataproc. * Background with Hadoop and/or Kafka. * Proficiency in Python and/or Java/Scala. * Understanding of data modeling, partitioning, distributed processing, and common data file formats. * Experience with DevOps practices, Git, automated deployments, and production support. * Strong troubleshooting, problem-solving, and collaboration skills. Preferred Skills * Experience with Looker and/or LookML. * Familiarity with GCS, BigLake, Hive, Apache Iceberg, and Parquet. * Experience using Terraform or other Infrastructure-as-Code tools. * Exposure to Kafka-based streaming pipelines. * Experience migrating workloads from on-premises Hadoop/Hive environments to Google Cloud. * Background in large-scale data lake or platform modernization efforts. * Familiarity with data quality, governance, lineage, and metadata management. Horizontal is committed to fostering an inclusive, respectful, and equitable environment where people of all backgrounds can contribute and grow. We value diverse perspectives and are dedicated to creating a workplace where everyone feels supported and empowered to succeed. ## Description We're looking for a skilled Data Engineer to help design, build, and support scalable data pipelines and analytics solutions for a hybrid opportunity in Brooklyn Park, MN. This role offers the chance to work with modern cloud data tools, large-scale processing technologies, and cross-functional teams to deliver reliable, production-ready data solutions. Responsibilities * Design, develop, test, and maintain scalable ETL/ELT pipelines for large and complex datasets. * Build and optimize distributed data processing solutions using Apache Spark and Google Cloud Dataproc. * Develop and support analytical datasets and data processing workloads in BigQuery. * Create batch and streaming data solutions using technologies such as Spark, Hadoop, and Kafka. * Develop ingestion and transformation workflows across source systems, data lakes, and analytics platforms. * Improve performance, scalability, reliability, and cost efficiency across Spark and BigQuery workloads. * Troubleshoot production data issues, perform root-cause analysis, and implement durable fixes. * Implement data quality checks, monitoring, logging, alerting, and operational controls. * Apply software engineering best practices, including source control, automated testing, CI/CD, and deployment automation. * Collaborate with architects, engineers, application teams, and business partners to deliver production data solutions. * Participate in code reviews and contribute to reusable frameworks and engineering best practices. ## Related Videos - [Parquet, Delta, Iceberg & Ducklake - An introduction for developers](https://www.wearedevelopers.com/videos/100075-parquet-delta-iceberg-ducklake-an-introduction-for-developers) - [Modern Data Architectures need Software Engineering](https://www.wearedevelopers.com/videos/1030-modern-data-architectures-need-software-engineering) - [From DevOps to Scaled DevOps: How We’re Rebuilding Continuous Delivery as a Platform](https://www.wearedevelopers.com/videos/100018-from-devops-to-scaled-devops-how-we-re-rebuilding-continuous-delivery-as-a-platform) - [How a Small Team Shrank a Microsoft Monorepo by 94%](https://www.wearedevelopers.com/videos/1236-how-a-small-team-shrank-a-microsoft-monorepo-by-94) - [From Messy Queries to Scalable Systems - How Data Engineering actually works](https://www.wearedevelopers.com/videos/100203-from-messy-queries-to-scalable-systems-how-data-engineering-actually-works) - [OLAP for AI Applications and why you should care](https://www.wearedevelopers.com/videos/100212-olap-for-ai-applications-and-why-you-should-care) ## Related Articles - [Making Data Warehouses Fast: A Developer’s Story](https://www.wearedevelopers.com/magazine/107-making-data-warehouses-fast-a-developer-s-story) - [Top Big Data Technologies That You Need to Know](https://www.wearedevelopers.com/magazine/108-top-big-data-technologies-that-you-need-to-know) - [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk) - [Got AI ideas but no money? Here are 10 free ways to level up your AI skills with Google Cloud](https://www.wearedevelopers.com/magazine/600-got-ai-ideas-but-no-money-here-are-10-free-ways-to-level-up-your-ai-skills-with-google-cloud) - [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers) - [How We Built a Worry-Free System That Runs for 10+ Years – And What We’d Do Again](https://www.wearedevelopers.com/magazine/751-how-we-built-a-worry-free-system-that-runs-for-10-years-and-what-we-d-do-again)