> Markdown version of [/jobs/ext/3078633-data-platform-engineer](https://www.wearedevelopers.com/jobs/ext/3078633-data-platform-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Data Platform Engineer - **Company:** Hadrian Automation - **Location:** Los Angeles, CA, United States - **Salary:** $170,000.0 - $300,000.0 - **Contract:** Permanent contract - **Skills:** Airflow, Data Analysis, Databases, Continuous Integration, Data Infrastructure, Data Systems, Distributed Data Store, Python (Programming Language), Machine Learning, Message Queuing Telemetry Transport (MQTT), Operational Data Store, Operational Databases, Performance Tuning, OPC Unified Architecture, Data Streaming, Transaction Data, Sql Optimization, Snowflake, Change Data Capture, Debezium, Kubernetes, Low Latency, Apache Flink, Apache Kafka, Operational Systems, Spark Streaming, Machine Learning Operations, Vertica, Stream Processing - **Published:** September 25, 2026 - **Apply:** https://www.juju.com/job/16_769bef171 ## About the Role * Experience building and operating production data infrastructure or distributed data systems, including on-call ownership and recovery efforts. * Strong production Python and advanced SQL and data-modeling skills, including incremental processing, temporal data, and schema evolution. * Experience with Kafka or another event-streaming platform, plus CDC or other stateful incremental pipelines. * Experience operating Snowflake, and with a lakehouse table format such as Iceberg, Delta, or Hudi, including expertise in partitioning and compaction. * Experience with tools such as Dagster, Airflow, Argo, or Prefect; dbt or similar transformation frameworks; and Kubernetes or infrastructure as code. * Strong judgment regarding contracts, failure modes, and the needs of downstream analytics, ML, and operational systems. What Will Set You Apart * Experience running Snowflake and Iceberg together or designing a hybrid warehouse and lakehouse architecture. * Production experience with PeerDB, Debezium, Flink, Spark Structured Streaming, Redpanda, Bufstream, or similar CDC and streaming systems. * Proficiency with ClickHouse or another low-latency analytical database, including performance tuning and lifecycle management. * Experience with industrial or edge data collection using OPC-UA, MTConnect, MQTT, historians, PLCs, or handling intermittently connected systems. * Background in performance-sensitive data systems built with Go, Rust, or Scala; regulated-environment experience; or contributions to dbt, Dagster, Iceberg, or related projects. ## Description Hadrian's factory data originates from production applications, operational databases, quality systems, ERP systems, and machines on the factory floor. This data fuels production schedules, machine-learning systems, engineering analysis, and company reporting. As a Data Platform Engineer, you will own key segments of the data flow from source to trusted dataset: ingestion, change data capture, streaming, lakehouse storage, orchestration, transformation, contracts, quality, and lineage. Some data sources may involve machines, PLCs, historians, and industrial protocols. Controls experience is advantageous but not required. This is a data-platform and distributed-systems role. Your work should enable new sources to be integrated easily, make failures straightforward to repair, and ensure datasets are reliable for use by Analytics, Data Science, Operations Research, and ML systems. What You'll Do * Build the data backbone for autonomous factories, transforming machine signals, quality events, work orders, and application changes into trusted data used by scheduling, ML, and operations. * Build ingestion, CDC, and streaming capabilities for transactional data, events, telemetry, and files; explicitly manage ordering, deletes, retries, replay, idempotence, and backpressure. * Define versioned data and event contracts with upstream teams, supported by testing and service targets for freshness, completeness, and correctness. * Model telemetry, quality events, work orders, and operational data into datasets with explicit grain, identity, time, provenance, and history. * Own Dagster orchestration, dbt transformation, data CI/CD, backfills, lineage, observability, and offline feature datasets for ML. * Collaborate with Manufacturing Operations and Infrastructure to acquire data from machines, PLCs, historians, OPC-UA, MTConnect, and MQTT sources as needed. ## Related Videos - [Fully Orchestrating Databricks from Airflow](https://www.wearedevelopers.com/videos/336-fully-orchestrating-databricks-from-airflow) - [Practical Change Data Streaming Use Cases With Debezium And Quarkus](https://www.wearedevelopers.com/videos/535-practical-change-data-streaming-use-cases-with-debezium-and-quarkus) - [Modern Data Architectures need Software Engineering](https://www.wearedevelopers.com/videos/1030-modern-data-architectures-need-software-engineering) - [From Messy Queries to Scalable Systems - How Data Engineering actually works](https://www.wearedevelopers.com/videos/100203-from-messy-queries-to-scalable-systems-how-data-engineering-actually-works) - [Python-Based Data Streaming Pipelines Within Minutes](https://www.wearedevelopers.com/videos/1233-python-based-data-streaming-pipelines-within-minutes) - [AI Model Management Life Circles: ML Ops For Generative AI Models From Research to Deployment](https://www.wearedevelopers.com/videos/1152-ai-model-management-life-circles-ml-ops-for-generative-ai-models-from-research-to-deployment) ## Related Articles - [Making Data Warehouses Fast: A Developer’s Story](https://www.wearedevelopers.com/magazine/107-making-data-warehouses-fast-a-developer-s-story) - [How We Built a Worry-Free System That Runs for 10+ Years – And What We’d Do Again](https://www.wearedevelopers.com/magazine/751-how-we-built-a-worry-free-system-that-runs-for-10-years-and-what-we-d-do-again) - [Top Big Data Technologies That You Need to Know](https://www.wearedevelopers.com/magazine/108-top-big-data-technologies-that-you-need-to-know) - [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers) - [MLops – Deploying, Maintaining And Evolving Machine Learning Models in Production](https://www.wearedevelopers.com/magazine/115-mlops-deploying-maintaining-and-evolving-machine-learning-models-in-production) - [Dev Digest 120 - Apple and peers](https://www.wearedevelopers.com/magazine/455-dev-digest-120-apple-and-peers)