Data Engineer

Skills, Inc.
Foster City, CA, United States
1 day ago
Apply on diversityjobs.com
Prepare application

Role details

Contract type
Temporary contract
Employment type
Full-time (> 32 hours)
Experience level
Expert
Experience required
5 years minimum
Compensation
$143,998 - $159,994
Working hours
Regular working hours

Tech stack

Artificial Intelligence Airflow Amazon Web Services Amazon S3 Computer Programming Databases Continuous Integration Information Engineering Relational Databases Dimensional Modeling Identity and Access Management Python (Programming Language)
+15 more
Operational Databases SAP (Applications) SAP HANA SQL Databases Data Processing Grafana Change Data Capture Git Data Layers Build Management Pyspark Cloudwatch Data Pipelines Amazon Redshift Databricks

Job description

We are seeking a highly skilled and hands-on data expert with deep manufacturing domain expertise to design, build, and maintain robust data pipelines for a cutting-edge manufacturing operations lakehouse. Working alongside a collaborative team of innovators, you will be responsible for building Change Data Capture (CDC) pipelines from scratch, transforming raw factory-floor, supply chain, and production data into actionable insights. This role offers the perfect blend of technical ownership, complex data modeling, and operational observability, allowing you to make a tangible footprint on the future of industrial automation.

What You Will Do

  • Design and build batch and streaming Change Data Capture (CDC) pipelines from scratch, ingesting manufacturing operations data from SAP (MES, HANA, SLT), on-premises databases, AWS DMS, and industrial sources like PLCs.
  • Build and maintain Bronze, Silver, and Gold (Medallion) layers and dimensional models tailored to manufacturing operations-including unit traceability, work orders, non-conformance reporting, assembly line station results, logistics, and first-pass yield.
  • Develop incremental SQL and dbt models (including MERGE/upsert, SCD Type 2, and late-arriving data handling) and process semi-structured test and diagnostic payloads using Python and PySpark.
  • Monitor pipeline health, data quality, and freshness SLAs using AWS CloudWatch alarms, log monitoring, Grafana dashboards, and automated data quality tests.
  • Orchestrate production workflows using Apache Airflow and manage AWS ingestion infrastructure (S3, IAM, DMS, Kinesis) using Git-based CI/CD.
  • Collaborate with cross-functional teams to build semantic layers that power business intelligence dashboards and downstream analytical applications.

Requirements

  • 5 to 10 years of hands-on data engineering experience.
  • Proven experience building data pipelines within manufacturing, industrial, automotive, or semiconductor operations.
  • Direct, hands-on experience working with SAP (ERP, MES, HANA, or SLT) and a deep understanding of factory-floor data concepts such as non-conformances, production testing/diagnostics, yield, work orders, or supply chain logistics.
  • Deep, hands-on experience building Change Data Capture (CDC) pipelines from scratch across relational databases and operational/industrial sources.
  • Strong proficiency in dimensional modeling, Medallion architecture (Bronze/Silver/Gold), and building semantic layers for analytics.
  • Advanced programming skills in Python, PySpark, and SQL, along with production experience in AWS (S3, DMS, Kinesis, CloudWatch, IAM) and workflow orchestration (Airflow or similar).

Bonus Points

  • Production experience with dbt, Amazon Redshift, Databricks, or Grafana.
  • Experience structuring lakehouse data to support AI and agentic applications.

About the company

Aquent has partnered with a pioneering leader in autonomous mobility and cutting-edge transportation technology to find an exceptional data professional ready to revolutionize factory-floor intelligence. In this high-impact role, you will have the unique opportunity to build critical data infrastructure from the ground up, directly influencing the production and scalability of next-generation vehicles. By bridging the gap between physical manufacturing operations and advanced cloud analytics, your work will empower real-time decision-making and drive the future of smart manufacturing.

Apply for this position

This job is hosted externally. Click below to view the full posting and apply.

Apply on diversityjobs.com
Prepare application

Good distractions

Loading talks and stories from around this role…