Data Engineer

Horizontal Talent
Pleasanton, CA, United States
12 days ago

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Experience level
Expert
Experience required
8 years minimum
Compensation
$74,880.0 - $145,600.0
Working hours
Regular working hours
Job source

Tech stack

Artificial Intelligence Airflow Big Data BigQuery Code Review Information Engineering Data Governance Extract Transform Load (ETL) Distributed Computing Environment Python (Programming Language) Meta-Data Management Scrum Methodology
+11 more
Cloud Services Cloudera SQL Databases Google Cloud Software Troubleshooting Git Build Management Pyspark Apache Kafka Video Streaming Data Pipelines

Job description

Join a dynamic Supply Chain Data & AI team as a Lead Data Engineer, where you’ll help design and deliver scalable cloud-based data solutions that power analytics and AI-driven decisions. This onsite role offers the opportunity to work across Sourcing, Transportation, and Warehouse Management domains while partnering with cross-functional teams to build high-quality, enterprise data products., * Design and build scalable data pipelines and data products on Google Cloud Platform.

  • Develop and optimize enterprise data solutions using Dataproc, BigQuery, SQL, and dbt.
  • Create reliable data models that support both analytical and operational reporting needs.
  • Build efficient ETL/ELT processes to ingest, transform, and publish data from multiple enterprise systems.
  • Partner with product managers, solution architects, data architects, analysts, and business stakeholders to turn requirements into technical solutions.
  • Lead technical design discussions and participate in code reviews to maintain engineering quality and consistency.
  • Improve performance, reliability, scalability, and cost efficiency across cloud data platforms.
  • Support monitoring, testing, production issue resolution, and continuous improvement efforts.
  • Contribute to reusable frameworks, documentation, and engineering standards.
  • Work within Agile delivery teams and help with sprint planning, estimation, and backlog refinement.
  • Mentor team members and help strengthen technical practices across the team.

Requirements

  • 8+ years of experience in data engineering with technical leadership on enterprise projects.
  • Strong hands-on experience with Google Cloud Platform.
  • Advanced proficiency in Dataproc, BigQuery, SQL, and dbt.
  • Solid understanding of modern ETL/ELT architecture and large-scale data processing.
  • Experience with data modeling, including dimensional, normalized, and warehouse design approaches.
  • Background building scalable, maintainable cloud-native data pipelines.
  • Experience with Git, CI/CD pipelines, and engineering best practices.
  • Strong troubleshooting, analytical, and problem-solving abilities.
  • Clear written and verbal communication skills with the ability to collaborate across teams., * Experience with Apache Airflow for workflow orchestration.
  • Exposure to Apache Kafka or other streaming technologies.
  • Working knowledge of PySpark for distributed data processing.
  • Proficiency in Python for automation, utilities, and data engineering tasks.
  • Familiarity with data quality, metadata management, and data governance practices.
  • Experience in retail, supply chain, transportation and logistics, warehouse management, or distribution center operations.
  • Self-driven mindset with a strong sense of ownership and commitment to high-quality delivery.
  • Ability to balance technical excellence with business priorities in a fast-paced environment.

Horizontal is committed to fostering an inclusive, equitable, and welcoming environment where diverse perspectives are valued and everyone has the opportunity to contribute and grow. We encourage candidates from all backgrounds to apply and bring their unique strengths to the team.

Benefits & conditions

Pulled from the full job description

  • Vision insurance
  • Dental insurance, Please apply through this online posting or by visiting our Job Board at www.horizontaltalent.com/job-board. Applications will be accepted for 4 weeks. For those that join the team, we offer competitive compensation and benefits including medical, dental, vision, and retirement. Check out all we have to offer and how you can become part of the Horizontal Talent Team. The pay range for this role is $36 - $70 per hour based on qualifications and experience.

Apply for this position

This job is hosted externally. Click below to view the full posting and apply.

Apply on www.indeed.com

Good distractions

Talks and stories from around this role — technically off-topic, practically not.

2:15 min

Empowering domain teams with an open data platform

Sandhya Menon Sandhya Menon · WWC Europe 2026

3:03 min

Career evolution in data engineering and AI platforms

Maria Apazoglou · Coffee With Developers

4:32 min

Harnessing Spark with Python using PySpark and Py4J

Ayon Roy · LIVE

6:21 min

Investigating push inefficiencies with upstream Git experts

Jonathan Creamer · Coffee With Developers

3:37 min

Scaling machine learning pipelines from prototypes to petabytes

Julian Joseph · LIVE

3:05 min

Audience questions on AI agents and pipeline vectorization

Joy Joy · WWC 2024

Videos

See all

Related articles

See all