Senior Data Platform Engineer

Robotics Technologies LLC
Sunnyvale, CA, United States
8 days ago
Apply on www.indeed.com
Prepare application

Role details

Contract type
Temporary contract
Employment type
Full-time (> 32 hours)
Experience level
Expert
Working hours
Regular working hours
Job source

Tech stack

Apache HTTP Server Microsoft Azure Continuous Integration Data Infrastructure Data Transformation Data Security Query Optimization Simple Data Format Parquet Apache Spark Data Lakes Kubernetes
+1 more
Infrastructure Automation Frameworks

Job description

Design, deploy, and operate data platform services on Kubernetes in Microsoft Azure. Build and manage lakehouse infrastructure on Apache Iceberg backed by ADLS, including catalog configuration, table maintenance, schema evolution, and partitioning strategy. Operate and tune Trino and Spark clusters for performance, reliability, and cost efficiency. Develop and maintain data transformation pipelines using DBT, with sound modeling, testing, and documentation practices. Define and enforce standards for data access, security, governance, and multi-tenant resource management across ADLS and query engines. Implement observability for the platform, including monitoring, alerting, capacity planning, and incident response. Partner with data engineers, analysts, and platform teams to onboard workloads and resolve performance bottlenecks. Automate infrastructure and deployments using infrastructure-as-code and CI/CD.

Requirements

Strong hands-on experience with Kubernetes, including deploying and operating stateful and distributed workloads. Strong hands-on experience with Microsoft Azure Cloud, including ADLS Hands-on experience with Apache Iceberg and Iceberg catalogs REST Expertise in distributed query and compute technologies, specifically Trino and Spark. Hands-on experience with DBT for data transformation and analytics engineering. Solid understanding of data lake and lakehouse architectures, file formats (Parquet, ORC), and query optimization. Equal Opportunity Employer

Apply for this position

This job is hosted externally. Click below to view the full posting and apply.

Apply on www.indeed.com
Prepare application

Good distractions

Loading talks and stories from around this role…