Senior AI Platform Reliability Engineer

HTC Global Services, Inc.
Seattle, United States of America
5 days ago

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Working hours
Regular working hours
Languages
English
Experience level
Senior

Job location

Remote
Seattle, United States of America

Tech stack

API
Artificial Intelligence
Amazon Web Services (AWS)
Azure
Bash
Cloud Computing
Databases
Continuous Integration
DevOps
Disaster Recovery
Distributed Systems
Python
Key Management
PostgreSQL
Machine Learning
Enterprise Messaging Systems
MongoDB
Operational Data Store
Performance Tuning
Redis
Reliability Engineering
Prometheus
Webui
Software Deployment
Vault (Revision Control System)
Web Services
YAML
AI Infrastructure
Google Cloud Platform
Cloud Platform System
Large Language Models
Grafana
Multi-Cloud
Generative AI
AI Platforms
Kubernetes
Deployment Automation
Kafka
Terraform
Splunk
Appdynamics

Job description

Are you a Senior Site Reliability Engineer with direct experience operating AI or machine learning platforms in large-scale production environments? We are looking for you to join our growing team working a hybrid schedule at one of our three locations either in Seattle, Burbak or Orlando. If working on a highly collaborative cutting edge technology team and in a job that is not a traditional infrastructure-only SRE role then this is the position for you!

The selected engineer will help build, scale, and operate the cloud and Kubernetes infrastructure that enables enterprise AI capabilities across the company and the selected engineer will help design, scale, and operate the cloud and Kubernetes infrastructure supporting enterprise AI workloads.

You do not need to be an AI model developers, data scientists, or LLM experts. instead your day to day focus and experience with the infrastructure and operational challenges involved in:

  • Deploying AI models or AI services into production, + Lead the design, implementation, and operation of highly available infrastructure supporting enterprise AI platforms and services.

  • Build and operate Kubernetes-based environments used to deploy and scale AI workloads.

  • Support the production deployment of AI models, inference services, AI APIs, agents, and related platform capabilities.

  • Design infrastructure that enables AI workloads across distributed and multi-cloud environments.

  • Partner with AI engineers, platform engineers, architects, and application teams to move AI services from development into reliable production environments.

  • Establish deployment, scaling, capacity, and reliability patterns for AI-powered services.

  • Design and maintain Kubernetes infrastructure using Helm and Terraform.

  • Support AI platform dependencies such as model gateways, vector or operational data stores, messaging platforms, secrets management, and API services.

  • Develop scalable platform solutions capable of maintaining 99.99% availability.

  • Lead capacity planning for AI workloads, including compute, memory, storage, network, and service dependencies.

  • Build automated CI/CD pipelines using Harness or comparable enterprise deployment platforms.

  • Implement blue/green deployments, canary releases, automated rollback, and feature-flag strategies.

  • Build observability for AI platforms using metrics, logs, traces, service-level indicators, and workload-specific health signals.

  • Troubleshoot complex production issues involving AI services, Kubernetes, cloud infrastructure, databases, messaging, networking, and distributed systems.

  • Lead incident response, root-cause analysis, and permanent corrective-action planning.

  • Mentor SRE, DevOps, and platform engineers and establish engineering and operational standards.

  • Ensure AI infrastructure aligns with enterprise security, governance, compliance, and resiliency requirements.

Requirements

The ideal candidate combines strong AI platform infrastructure experience with deep expertise in SRE, Kubernetes, multi-cloud engineering, Infrastructure as Code, observability, and production reliability., + 7+ years of experience in Site Reliability Engineering, Platform Engineering, DevOps, cloud infrastructure, or a related field.

  • Direct experience supporting AI, machine learning, or model-serving platforms in production.

  • Experience deploying or operating AI models, inference services, AI APIs, agents, or AI-enabled applications in cloud environments.

  • Experience supporting AI workloads at enterprise or high-traffic scale.

  • Strong understanding of the infrastructure required to move AI services from development into production.

  • Expert-level Kubernetes administration and production operations experience.

  • Strong Helm and Terraform experience.

  • Experience designing scalable, highly available, distributed cloud infrastructure.

  • Hands-on experience with Google Cloud Platform, with additional AWS or Azure experience.

  • Experience building automated deployment pipelines using Harness or a comparable enterprise CI/CD platform.

  • Strong scripting and automation skills using Python, Bash, and YAML.

  • Experience supporting production databases, messaging, and platform services such as PostgreSQL, Redis, Kafka, MongoDB, and Vault.

  • Experience implementing observability using OpenTelemetry, Prometheus, Grafana, Splunk, AppDynamics, or similar tools.

  • Strong production troubleshooting and incident-management experience across cloud-native distributed systems.

  • Experience with capacity planning, performance tuning, disaster recovery, and high-availability design.

  • Strong communication and technical leadership skills.

Preferred Qualifications

  • Experience supporting generative AI, LLM, model inference, or AI-agent platforms.

  • Experience operating model gateways, AI APIs, GPU-enabled workloads, or distributed inference services.

  • Experience with AI platform technologies such as LiteLLM, Open WebUI, model-serving frameworks, vector databases, or similar platforms.

  • Experience defining SLOs, SLIs, error budgets, and reliability standards for AI services.

  • Experience with GCP-based AI infrastructure and services.

  • Experience supporting high-volume or globally distributed AI platforms.

  • Experience with progressive delivery, chaos engineering, and automated resilience testing.

  • Cloud or Kubernetes certifications.

About the company

What Makes HTC A Great Place To Build Your Future HTC Global Services wants you to join our team. Come build new things with us and advance your career. At HTC Global, you'll collaborate with experts, work alongside clients, and be part of high-performing teams driving success together. You'll have long-term opportunities to grow your career and develop skills in the latest emerging technologies. At HTC Global Services, our employees have access to a comprehensive benefits package. Benefits can include Group Health (Medical, Dental, and Vision), Paid Time Off, Paid Holidays, 401(k) matching, Group Life and Disability insurance, Professional Development opportunities, Wellness programs, and a variety of other perks. Our success as a company is built on inclusion and diversity. HTC Global Services is committed to providing a workplace free from discrimination and harassment, where every employee is treated with dignity and respect. We celebrate differences and believe that diverse cultures, perspectives, and skills drive innovation and success. HTC is an Equal Opportunity Employer and a proud National Minority Supplier. We seek to empower each individual, fostering an environment where everyone feels valued, included, and respected. #LI-NC1 #LI-DT1 #LI-Remote #Hiring #AIJobs #SREJobs #SeattleJobs #BurbankJobs #OrlandoJobs

Apply for this position