> Markdown version of [/jobs/ext/1179713-senior-data-engineer-data-lead](https://www.wearedevelopers.com/jobs/ext/1179713-senior-data-engineer-data-lead). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Senior Data Engineer - Data Lead - **Company:** Foundry Robotics - **Location:** San Francisco, CA, United States - **Experience:** Expert - **Salary:** $150,000.0 - $250,000.0 - **Contract:** Permanent contract - **Skills:** Java (Programming Language), Application Programming Interfaces (APIs), Airflow, Amazon S3, Data Analysis, Business Logic, Microsoft Azure, Batch Processing, BigQuery, Cloud Computing, Cloud Database, Databases, Data Architecture, Data Cleansing, Information Engineering, Data Files, Extract Transform Load (ETL), Data Systems, Distributed Systems, Data Flow Control, Information Lifecycle Management, Python (Programming Language), Machine Learning, Meta-Data Management, NoSQL, Software Architecture, SQL Databases, Data Streaming, Technical Data Management Systems, TypeScript, Management of Software Versions, Data Logging, Pulumi, Apache Spark, Indexer, Backend, Data Layers, Event Driven Architecture, Kubernetes, Data Lineage, Apache Flink, Apache Kafka, Terraform, Data Pipelines - **Published:** July 4, 2026 - **Apply:** https://www.indeed.com/viewjob?jk=e376dd6a06085f83 ## About the Role * Between 5-7 years of experience in data engineering, with demonstrated work at terabyte-to-petabyte scale * Deep hands-on experience with data pipeline frameworks (Spark, Flink, Kafka, Airflow, dbt, or similar) * Strong proficiency in Python and SQL; backend experience in Go, Java, or TypeScript is a plus * Experience designing data architectures spanning streaming, batch, lakehouse, and warehouse patterns * Solid understanding of distributed systems, storage engines, and data modeling (relational and NoSQL) * Experience with cloud data services (AWS S3/Glue/Redshift, GCP BigQuery/Dataflow, or Azure equivalents) * Comfortable defining roadmaps, making architectural decisions, and driving alignment across teams * Comfortable operating independently and owning complex systems end-to-end Nice to Have (Not Required) * Experience with visual or image data pipelines at scale (video ingestion, frame extraction, annotation workflows) * Background in manufacturing, robotics, or industrial IoT data systems * Familiarity with ML data preparation workflows (feature stores, dataset versioning, labeling pipelines) * Experience with edge-to-cloud data architectures and intermittent connectivity patterns * Exposure to infrastructure-as-code (Terraform, Pulumi) and container orchestration (Kubernetes) ## Description We are hiring a Sr Data Engineer / Data Lead to own the data layer that powers Factory OS. You will design and operate petabyte-scale data pipelines that ingest visual, telemetry, and manufacturing data from the factory floor, move it reliably across edge, on-prem, and cloud environments, and make it available for ML training, analytics, and operational decision-making. You will define the data roadmap-architecture, governance, lifecycle, and tooling-and be the person the rest of the engineering org depends on for clean, reliable, well-modeled data. You will also contribute to backend services where data meets application logic. This is a hands-on engineering role. You will ship., Petabyte-Scale Data Pipelines * Design, build, and operate PB-scale data pipelines for ingesting, curating, indexing, and preparing manufacturing, visual, and telemetry data * Implement reliable data movement across embedded, edge, on-prem, and cloud compute environments * Build streaming and batch processing systems that handle high-throughput factory-floor data in real time * Implement data lifecycle management: retention, archival, compaction, and cost optimization at scale Data Architecture + Roadmap * Define and own the Factory OS data roadmap-architecture, governance, quality, and tooling strategy * Design data models, schemas, and contracts that serve application, ML, and analytics consumers * Establish data cataloging, lineage tracking, and discoverability across the platform * Drive data quality standards: validation, monitoring, alerting, and anomaly detection * Evaluate and adopt data technologies (warehouses, lakehouses, streaming platforms) as the platform scales Backend Services + Integration * Contribute to backend services where data pipelines meet application logic (APIs, event-driven systems, database layers) * Partner with ML engineers to ensure training and inference pipelines have access to clean, well-prepared datasets * Collaborate with full-stack developers, robotics teams, and operations to translate data needs into reliable infrastructure * Implement observability: metrics, logging, and tracing across data systems ## Related Videos - [From Messy Queries to Scalable Systems - How Data Engineering actually works](https://www.wearedevelopers.com/videos/100203-from-messy-queries-to-scalable-systems-how-data-engineering-actually-works) - [Unleashing Potential Across Teams: The Power of Infrastructure as Code](https://www.wearedevelopers.com/videos/930-unleashing-potential-across-teams-the-power-of-infrastructure-as-code) - [Leveraging Real time data in FSIs](https://www.wearedevelopers.com/videos/806-leveraging-real-time-data-in-fsis) - [Enjoying SQL data pipelines with dbt](https://www.wearedevelopers.com/videos/823-enjoying-sql-data-pipelines-with-dbt) - [AI Model Management Life Circles: ML Ops For Generative AI Models From Research to Deployment](https://www.wearedevelopers.com/videos/1152-ai-model-management-life-circles-ml-ops-for-generative-ai-models-from-research-to-deployment) - [Terraform for Developers](https://www.wearedevelopers.com/videos/3-terraform-for-developers) ## Related Articles - [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers) - [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk) - [How to Become an AI Engineer](https://www.wearedevelopers.com/magazine/331-how-to-become-an-ai-engineer) - [Dev Digest 120 - Apple and peers](https://www.wearedevelopers.com/magazine/455-dev-digest-120-apple-and-peers) - [Dev Digest 121 - AI goes offline](https://www.wearedevelopers.com/magazine/456-dev-digest-121-ai-goes-offline) - [Fully Remote Software Engineer Jobs](https://www.wearedevelopers.com/magazine/447-fully-remote-software-engineer-jobs)