> Markdown version of [/jobs/ext/3822450-gpuinfrastructure-engineer](https://www.wearedevelopers.com/jobs/ext/3822450-gpuinfrastructure-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # GPUInfrastructure Engineer - **Company:** NVIDIA Ltd. - **Location:** United States (Remote available) - **Experience:** Expert - **Contract:** Contract - **Skills:** Application Programming Interfaces (APIs), Artificial Intelligence, Systems Engineering, Bash Shell, BIOS, Computer Clusters, Nvidia CUDA, Linux, Firmware, InfiniBand, Python (Programming Language), Key Management, Network Layer, Linux System Administration, Machine Learning, PCI Express, Performance Tuning, Remote Direct Memory Access, Ansible, Systems Integration, Data Logging, Graphics Processing Unit (GPU), Cloud Platform System, High Performance Computing, Performance Testing, System Availability, Large Language Models, Non-Uniform Memory Access, Usage Tracking, AI Platforms, Kubernetes, Infrastructure Automation Frameworks, Slurm, Triton Inference Server, Machine Learning Operations, Hardware Infrastructure, Nim (Programming Language), Api Design, Api Gateway, VLLM, Model Inference, Terraform, Docker - **Published:** October 18, 2026 - **Apply:** https://www.dice.com/job-detail/1597466e-40bd-4725-92b8-dc4c07596b6a ## About the Role 8+ years of Linux, infrastructure, HPC, cloud platform or systems engineering experience. - Strong hands-on experience with NVIDIA GPU infrastructure. - Experience deploying NVIDIA H100/H200 or comparable enterprise GPU platforms. - Strong knowledge of: - NVIDIA CUDA - NVIDIA GPU Drivers - NVIDIA Container Toolkit - NCCL - GPUDirect RDMA - NVIDIA AI Enterprise - Hands-on experience with SLURM. - Experience with NVIDIA MIG and GPU resource partitioning. - Strong Linux administration and troubleshooting skills. - Experience with InfiniBand/RDMA high-performance networking. - Understanding of GPU topology, PCIe, NUMA, NVLink and multi-GPU communication. - Experience deploying containerized AI/ML workloads. - Experience with Docker and Kubernetes/container orchestration technologies. - Strong understanding of HA, monitoring, logging and production infrastructure practices. AI/LLM Platform Experience Candidates should have hands-on experience with several of the following: - NVIDIA NIM - NVIDIA AI Enterprise - LLM inference infrastructure - vLLM, Triton Inference Server or similar serving technologies - API gateways - Multi-tenant AI platforms - Token metering and usage tracking - Model registries - AI governance and guardrails - PrometheGrafana or equivalent observability platforms - Python/Bash automation Preferred Experience - HPE ProLiant DL380/DL360 or similar enterprise GPU server platforms. - NVIDIA H100/H200 deployments. - Large-scale enterprise AI Factory implementations. - Production LLM serving and inference optimization. - Kubernetes GPU Operator. - Infrastructure automation using Ansible/Terraform. - Enterprise API gateway platforms. - FinOps/chargeback platforms. - Payment or billing-system integrations. - Telecom or other large regulated enterprise environments. - Security, data-sovereignty and compliance requirements. Ideal Candidate The ideal candidate is not solely an AI/ML developer or a traditional Linux administrator. We are looking for someone who understands the complete GPU-to-API stack: HPE GPU Hardware Linux/Firmware NVIDIA Drivers/CUDA InfiniBand/RDMA SLURM/MIG GPU Clusters NVIDIA NIM LLM Inference API Gateway Multi-Tenancy Token Metering Production Operations The candidate should be comfortable independently troubleshooting infrastructure from the GPU and network layer through the model-serving and API layer. ## Description We are looking for a Senior NVIDIA AI Factory / GPU Infrastructure Engineer to design, deploy, configure, and productionize a large-scale NVIDIA H200 GPU environment for an enterprise AI Factory initiative. The engineer will work on an environment consisting of 32 NVIDIA H200 GPUs across HPE Gen12 GPU servers, high-speed InfiniBand networking, NVIDIA AI Enterprise, GPU workload orchestration, multi-tenant inference, NVIDIA NIM, and token/API-based AI services. This is a hands-on role requiring strong experience taking GPU infrastructure from hardware readiness through production AI workloads. Physical rack/stack and power activities will be handled by the on-site infrastructure partner. Key Responsibilities GPU Infrastructure & Cluster Deployment - Validate readiness of HPE DL380a Gen12 GPU servers with NVIDIA H200 GPUs. - Perform GPU, NVLink, PCIe, firmware, BIOS, BMC and hardware health validation. - Configure and maintain NVIDIA GPU drivers, CUDA, container runtime and supporting libraries. - Validate GPU topology and multi-GPU communication. - Configure and troubleshoot high-speed InfiniBand/NDR networking. - Configure and validate GPUDirect RDMA and GPU-to-GPU communication. - Perform NCCL and GPU communication/performance testing. - Integrate high-performance storage for AI models, datasets and inference workloads. - Establish validated firmware, OS, driver and CUDA baselines. SLURM, MIG & GPU Resource Management - Deploy and configure SLURM for GPU workload scheduling and orchestration. - Configure NVIDIA Multi-Instance GPU (MIG) profiles where appropriate. - Implement GPU resource allocation, scheduling, quotas and workload isolation. - Support multi-user and multi-tenant GPU environments. - Monitor GPU utilization, memory, workload performance and cluster health. - Troubleshoot GPU scheduling, CUDA, NCCL, networking and performance issues. AI Factory Architecture - Build two isolated, highly available GPU clusters: one internal and one public-facing. - Design production-grade GPU cluster architecture with security and workload isolation. - Implement high availability, monitoring, logging and operational controls. - Support capacity planning and GPU sizing for AI/LLM workloads. - Develop deployment runbooks, architecture documentation and operational procedures. NVIDIA AI Enterprise & NIM - Deploy and configure NVIDIA AI Enterprise (NVAIE) components. - Deploy LLMs and AI models using NVIDIA NIM. - Configure scalable model-serving infrastructure. - Optimize inference workloads for NVIDIA H200 GPUs. - Perform model-serving benchmarking, capacity testing and performance tuning. - Integrate inference endpoints with enterprise API gateways. Multi-Tenant Inference Platform - Design and implement a secure Inference-as-a-Service platform. - Configure API-based access to hosted AI/LLM models. - Implement tenant, application and model-level isolation. - Support API key management and secure service access. - Implement quotas, rate limits and GPU resource controls. - Enable token-based usage metering and consumption tracking. - Support model registry, prompt logging and AI governance controls. Token Metering & API Gateway - Integrate inference services with an enterprise API gateway. - Capture input/output token consumption for LLM requests. - Implement token metering, quotas and rate limiting. - Enable per-tenant usage reporting and chargeback. - Integrate usage data with billing/payment platforms. - Support prepaid and postpaid AI consumption models. Production Readiness - Conduct end-to-end infrastructure and inference testing. - Validate GPU performance, networking, storage and workload scheduling. - Perform HA/failover and resiliency testing. - Support security hardening and production-readiness assessments. - Establish monitoring, alerting and operational dashboards. - Define production acceptance criteria and SLA measurements. - Provide technical documentation and knowledge transfer to client teams.