Software Engineer/Senior Software Engineer, Data & ML Platform

PlusAI, Inc.
Santa Clara, CA, United States
about 1 month ago
Apply on www.indeed.com
Prepare application

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Experience level
Expert
Compensation
$135,000.0 - $200,000.0
Working hours
Regular working hours
Job source

Tech stack

Apache HTTP Server Computing Platforms Big Data Distributed Computing Environment Distributed Data Store Node.Js Octopus Deploy Azure Machine Learning Autoscaling Apache Spark Data Lakes Kubernetes
+2 more
Information Technology Hardware Infrastructure

Job description

  • Operate and evolve our production Kubernetes clusters end to end: bare-metal provisioning automation, highly available control planes, node lifecycle, GPU container runtime, networking, and storage
  • Build safe, repeatable GitOps-based delivery for platform services and user applications using tools such as Argo CD, Helm, and Kustomize
  • Develop shared multi-tenant platform capabilities for scheduling, resource isolation, storage, networking, access control, secrets, and observability while improving CPU/GPU utilization and cost efficiency
  • Build and improve reusable distributed batch and workflow platforms for Spark data processing and GPU-based replay and simulation
  • Ensure that your work is performed in accordance with the company’s Quality Management System (QMS) requirements and contribute to continuous improvement efforts

Requirements

All key offline workloads - large-scale data processing, simulation, auto-labeling, scenario mining, and model training - run on the compute platform this role owns. In this role, you will improve the reliability and efficiency of our Kubernetes infrastructure, make workload onboarding simpler and more self-service, and build reusable batch and workflow capabilities for petabyte-scale processing. We are looking for strong Kubernetes and platform-engineering fundamentals, depth in at least one adjacent area-distributed data processing, ML/GPU infrastructure, or multi-tenant compute systems-and the curiosity and ownership to grow across the others.

We are open to candidates at either the Software Engineer or Senior Software Engineer level. Level will be determined by experience, technical depth, scope of ownership, and demonstrated impact. You do not need experience with every technology in our stack; we value strong fundamentals, ownership, and the ability to learn., * BS, MS, or PhD in Computer Science or a related technical field, or equivalent practical experience

  • Hands-on experience operating production Kubernetes clusters - node lifecycle, upgrades, troubleshooting - plus GitOps and infrastructure-as-code experience
  • Experience with GPU or ML workload scheduling, queueing and priorities, fractional GPU sharing, autoscaling, or multi-tenant resource management
  • Self-driven with a strong sense of ownership: a quick learner who is eager to take responsibility and drive projects forward end to end, * Experience with Ray or Kubeflow
  • Experience with lakehouse technologies such as Delta Lake or Apache Iceberg
  • Experience operating large-scale distributed data-processing and workflow systems, with hands-on depth in a system such as Apache Spark and working knowledge of Argo Workflows or an equivalent orchestrator

Apply for this position

This job is hosted externally. Click below to view the full posting and apply.

Apply on www.indeed.com
Prepare application

Good distractions

Talks and stories from around this role — technically off-topic, practically not.

45 sec

Working securely with Node.js path application programming interfaces

Sonya Moisset · World Congress 2023

3:28 min

Defining big data and machine learning fundamentals

Ayon Roy · LIVE

3:24 min

The governance failures of centralized data lakes

Mario Meir-Huber · LIVE

3:03 min

Career evolution in data engineering and AI platforms

Maria Apazoglou · Coffee With Developers

3:55 min

Identifying underlying Node.js runtime vulnerabilities using fuzzing tools

Sonya Moisset · World Congress 2023

2:14 min

Solving complex platform architecture challenges at an enterprise scale

Maria Apazoglou · Coffee With Developers

Videos

See all

Related articles

See all