> Markdown version of [/jobs/ext/1886882-ai-infrastructure-engineer](https://www.wearedevelopers.com/jobs/ext/1886882-ai-infrastructure-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# AI Infrastructure Engineer

- **Company:** Gatik AI, Inc.
- **Location:** Mountain View, CA, United States
- **Experience:** Expert
- **Salary:** $180,000.0 - $240,000.0
- **Contract:** Permanent contract
- **Skills:** A/B Testing, Application Programming Interfaces (APIs), Artificial Intelligence, Airflow, Amazon S3, Bash Shell, Batch Processing, Cloud Computing, Cloud Engineering, Code Review, Databases, Continuous Delivery, Continuous Integration, Information Engineering, Data Files, Data Integration, Extract Transform Load (ETL), DevOps, Distributed Computing Environment, Distributed Systems, Monitoring of Systems, InfiniBand, Python (Programming Language), Log Analysis, Routing, Operational Databases, Performance Tuning, Raw Data, Prometheus, Data Streaming, Systems Integration, AI Infrastructure, Graphics Processing Unit (GPU), Pytorch, Large Language Models, Grafana, Apache Spark, HybridCloud, Gaussian, Containerization, Data Lakes, AI Platforms, Git Flow, Kubernetes, Infrastructure Automation Frameworks, Information Technology, Low Latency, Deployment Automation, ONNX (Open Neural Network Exchange) Format, Apache Kafka, Data Management, Machine Learning Operations, TensorRT, Virtual Agents, Terraform, Data Pipelines, Docker, Elk Stack
- **Published:** July 31, 2026
- **Apply:** https://www.careerbuilder.com/job-details/senior-ai-infrastructure-engineer-mountain-view-ca--98165f70-227c-4616-ad4d-d60fa107ac91

## About the Role

* Experience: 5+ years in ML infrastructure, MLOps, or DevOps supporting high-scale compute environments.
 * ML Expertise: Deep understanding of multi-GPU training strategies (FSDP, DeepSpeed, Ray Train) and high-performance networking (NCCL, InfiniBand).
 * Infrastructure Automation: Mastery of Kubernetes, Terraform, and Helm, with a focus on GPU-native orchestration.
 * AI Agent Frameworks: Proven experience building or supporting Agentic Workflows for infrastructure or data automation (e.g., using LLMs to drive DevOps tasks).
 * Platform Mastery: Expertise in MLFlow, Argo Workflows, and Kubernetes.
 * Containerization: Strong experience with Docker, Kubernetes, and Helm.
 * Data & CI/CD: Proficiency in Apache Airflow, Kafka, Spark, and GitOps automation.
 * Core Skills: Proficiency in Python and Bash; experience with Go or Rust is a plus

Bonus Qualifications

 * Advanced AI Protocols: Familiarity with the Model Context Protocol (MCP) to standardize how AI agents interact with internal databases and orchestration APIs.
 * Hybrid & Physical AI: Experience in hybrid cloud and on-prem GPU cluster management for Physical AI workloads (e.g., 3DGS, World Models).
 * Agentic Observability: Experience utilizing LLMs for semantic monitoring and log analysis to detect complex distributed system failures that traditional threshold-based alerts miss., A/B Testing, Amazon Simple Storage Service (S3), Analysis Skills, Apache Kafka, Application Programming Interface (API), Artificial Intelligence (AI), Artificial Intelligence (AI) Agents, Audiovisual, Automation, Automotive Automation, Autonomous Driving Systems, Business-to-Business (B2B), Cargo/Freight, Cloud Computing, Code Reviews, Computer Systems, Continuous Deployment/Delivery, Continuous Integration, Data Management, Data Modeling, Data Sets, Database Extract Transform and Load (ETL), DevOps, Distributed Computing, Docker, Establish Priorities, Fortune 500 Customers, Futures, GPU (Graphics Processing Unit), Hardware-Software Integration, Hybrid Cloud, Intellectual Property (IP), Logistics, MCP - Microsoft Certified Professional, Metrics, Network System Hardware, Performance Engineering, Performance Metrics, Performance Tuning/Optimization, Philosophy, Profit & Loss, Public/Media/Press/Analyst Relations, Regulations, Return on Capital Employed (ROCE), Safety Systems, Startup,
Supply Chain, Sustainability, System Operations, Systems Administration/Management, Transportation Routing, Truck Driver, Trucking, Vehicle Fleets

## Description

We are seeking a Senior AI Infrastructure Engineer to design, build, and scale the high-performance AI platform powering our autonomous driving models. While researchers focus on developing perception, planning, and world models, you will be responsible for the underlying infrastructure that enables distributed training, experiment tracking, and seamless model deployment. You will bridge the gap between research and production, ensuring our AI stack is scalable, resilient, and highly efficient

This role is onsite 5 days a week at our Mountain View, CA office!

What you'll do

 * Distributed Training & ML Systems Support
 + Scale Research Workloads: Enable researchers to scale complex models (VLA, World Models) across multi-node setups using PyTorch Distributed, and Ray Train.
 + Performance Optimization: Architect and optimize multi-GPU setups, ensuring efficient model parallelism and data parallelism techniques across H100/A100 clusters.
 + Networking & Hardware Tuning: Optimize low-level communication (e.g., NCCL tuning, InfiniBand, or RoCE v2) to minimize latency for 3D Gaussian Splatting (3DGS) and large-scale training.
 + Intelligent Resource Scheduling: Optimize hardware utilization and cost-efficiency through Kubernetes-native GPU scheduling (NVIDIA GPU Operator, KubeFlow).
 + Inference Performance Engineering: Deploy and scale optimized model artifacts using TensorRT, ONNX Runtime, and Triton Inference Server, fine-tuning pipelines for both real-time and batch processing
 * Agentic Infrastructure & Automation
 + Self-Healing AI Infrastructure: Architect and deploy Autonomous AI Agents (LangGraph, CrewAI, or AutoGen) to monitor GPU cluster health, enabling automated real-time triage of hardware failures and NCCL timeouts.
 + Agentic DevOps & CI/CD: Develop agent-driven automation, such as Agentic PR Reviewers for infrastructure code and AI agents that proactively suggest model-specific Kubernetes resource optimizations.
 + Agentic Data Curation: Support researchers in building "Data Machines" where AI agents autonomously curate, label, and verify high-priority edge cases from raw data.
 * Model Management & Lifecycle (MLOps)
 + Automated Lifecycle Management: Design and maintain ML infrastructure leveraging MLFlow, Argo Workflows, and Kubernetes to automate the end-to-end model lifecycle.
 + Experiment & Model Tracking: Integrate feature stores and experiment tracking systems to provide a robust system of record for every model iteration.
 + Deployment Strategies: Implement robust serving mechanisms, including A/B testing, shadow deployments, and rollback mechanisms
 * Cloud-Native Foundations & Data Integration
 + Infrastructure as Code: Drive the "Everything as Code" philosophy using Terraform and Helm.
 + Data Pipelines: Collaborate with data teams to scale ETL pipelines using Apache Airflow, Kafka, and Spark for large-scale dataset management.
 + Integrated Data Factories: Collaborate with data engineering teams to scale high-bandwidth ETL pipelines using Apache Airflow, Kafka, and Spark, ensuring seamless data flow from raw sensor logs to optimized storage in S3, GCS, or Delta Lake
 * Monitoring & Observability
 + System Metrics: Define and track key ML system metrics, including training convergence, latency, throughput, and drift detection.
 + Infrastructure Health: Maintain deep visibility into platform health using Prometheus, Grafana, OpenTelemetry, and ELK Stack.
 + Deep Stack Observability: Develop comprehensive monitoring using Prometheus, Grafana, and OpenTelemetry to track low-level infrastructure health alongside high-level ML metrics like training convergence and throughput.
 + AI-Specific Metrics & Drift: Define and monitor critical ML system KPIs, including model latency, inference throughput, and feature drift detection


## Related Videos

- [From Messy Queries to Scalable Systems - How Data Engineering actually works](https://www.wearedevelopers.com/videos/100203-from-messy-queries-to-scalable-systems-how-data-engineering-actually-works)
- [From DevOps to Scaled DevOps: How We’re Rebuilding Continuous Delivery as a Platform](https://www.wearedevelopers.com/videos/100018-from-devops-to-scaled-devops-how-we-re-rebuilding-continuous-delivery-as-a-platform)
- [Docker Compose: Rediscovered](https://www.wearedevelopers.com/videos/1978-docker-compose-rediscovered)
- [Shipping Faster with Less: Render on Cloud Hosting, AI Workloads, and the Future of DevOps](https://www.wearedevelopers.com/videos/1894-shipping-faster-with-less-render-on-cloud-hosting-ai-workloads-and-the-future-of-devops)
- [AI Model Management Life Circles: ML Ops For Generative AI Models From Research to Deployment](https://www.wearedevelopers.com/videos/1152-ai-model-management-life-circles-ml-ops-for-generative-ai-models-from-research-to-deployment)
- [DevOps Maturity Check – a way to balance autonomy and alignment](https://www.wearedevelopers.com/videos/58-devops-maturity-check-a-way-to-balance-autonomy-and-alignment)

## Related Articles

- [MLOps And AI Driven Development](https://www.wearedevelopers.com/magazine/82-mlops-and-ai-driven-development)
- [Graph and AI Trends 2026: Why Is AI Running but Not Yet Delivering?](https://www.wearedevelopers.com/magazine/680-graph-and-ai-trends-2026-why-is-ai-running-but-not-yet-delivering)
- [How to Become an AI Engineer](https://www.wearedevelopers.com/magazine/331-how-to-become-an-ai-engineer)
- [MLOps – What’s the deal behind it?](https://www.wearedevelopers.com/magazine/125-mlops-what-s-the-deal-behind-it)
- [Got AI ideas but no money? Here are 10 free ways to level up your AI skills with Google Cloud](https://www.wearedevelopers.com/magazine/600-got-ai-ideas-but-no-money-here-are-10-free-ways-to-level-up-your-ai-skills-with-google-cloud)
- [Stephan Gillich - Bringing AI Everywhere](https://www.wearedevelopers.com/magazine/489-stephan-gillich-bringing-ai-everywhere)