> Markdown version of [/jobs/ext/4009432-ai-infrastructure-engineer](https://www.wearedevelopers.com/jobs/ext/4009432-ai-infrastructure-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # AI Infrastructure Engineer - **Company:** Ova Technologies - **Location:** New York, NY, United States - **Experience:** Experienced - **Contract:** Permanent contract - **Skills:** Artificial Intelligence, Airflow, Amazon Web Services, Microsoft Azure, Bash Shell, Cloud Computing, Cloud Engineering, Nvidia CUDA, Continuous Integration, Linux, DevOps, Distributed File Systems, Distributed Computing Environment, Github, Identity and Access Management, Python (Programming Language), Key Management, PostgreSQL, Linux System Administration, Machine Learning, Redis, Prometheus, AI Infrastructure, Data Logging, Pulumi, Google Cloud, Load Balancing, System Availability, Large Language Models, Grafana, Apache Spark, Multi-Cloud, Generative AI, HybridCloud, Infrastructure as Code (IaC), Git, Gitlab-ci, Kubernetes, Infrastructure Automation Frameworks, Storage Technologies, Information Technology, Ray Serve, Slurm, Triton Inference Server, Machine Learning Operations, Hardware Infrastructure, Terraform, Docker, KServe, Elk Stack, Jenkins - **Published:** October 29, 2026 - **Apply:** https://www.thejobnetwork.com/job/77edc91b-be2a-4801-9185-1b1599f64a11/ai-infrastructure-engineer ## About the Role * Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, or a related field. * 4 years of experience in Infrastructure Engineering, Cloud Engineering, Platform Engineering, or DevOps. * Strong experience with Linux system administration. * Proficiency in Python, Bash, or Go for infrastructure automation. * Hands-on experience with Docker and Kubernetes. * Experience with one or more cloud platforms: AWS, Microsoft Azure, or Google Cloud Platform. * Experience with Infrastructure as Code tools such as Terraform or Pulumi. * Strong understanding of networking, storage, load balancing, and security. * Experience with CI/CD tools such as GitHub Actions, GitLab CI, or Jenkins. * Knowledge of monitoring and logging tools such as Prometheus, Grafana, ELK Stack, or OpenTelemetry. Preferred Qualifications * Experience managing NVIDIA GPU infrastructure and CUDA environments. * Experience with distributed computing frameworks such as Ray, Apache Spark, or Slurm. * Experience with AI model serving frameworks such as NVIDIA Triton Inference Server, KServe, or Ray Serve. * Familiarity with MLOps tools such as MLflow, Kubeflow, or Airflow. * Experience with vector databases and Generative AI infrastructure. * Knowledge of storage technologies for AI workloads, including object storage and distributed file systems. * Experience with high-performance computing (HPC) environments. * Familiarity with infrastructure security, compliance, and governance standards. Technical Skills * Linux * Python * Bash * Go (preferred) * Docker * Kubernetes * Terraform or Pulumi * AWS / Azure / Google Cloud Platform * NVIDIA GPU Technologies * CUDA * Prometheus * Grafana * OpenTelemetry * Git * GitHub Actions / GitLab CI / Jenkins * PostgreSQL * Redis * Ray * Apache Spark * Slurm * NVIDIA Triton Inference Server * KServe Soft Skills * Strong analytical and troubleshooting abilities * Excellent communication and collaboration skills * Ownership and accountability * Ability to work in cross-functional teams * Strong documentation and operational excellence mindset * Continuous learning and adaptability Nice to Have * Experience supporting Large Language Models (LLMs) and Generative AI platforms * Experience with Retrieval-Augmented Generation (RAG) infrastructure * Knowledge of AI infrastructure cost optimization strategies * Experience with multi-cloud or hybrid-cloud deployments * Cloud, Kubernetes, or Linux certifications ## Description We are seeking an AI Infrastructure Engineer to design, implement, and manage the infrastructure that powers AI and machine learning workloads. In this role, you will build scalable, secure, and high-performance environments for model training, inference, and deployment while optimizing compute, storage, networking, and GPU resources. You will work closely with AI/ML engineers, platform engineers, data engineers, and DevOps teams to ensure reliable and efficient AI operations., * Design, deploy, and maintain AI infrastructure across cloud and on-premises environments. * Build and manage GPU-enabled compute clusters for machine learning training and inference. * Implement scalable infrastructure for distributed AI workloads. * Deploy and manage Kubernetes clusters for containerized AI applications. * Automate infrastructure provisioning using Infrastructure as Code (IaC). * Develop and maintain CI/CD pipelines for AI infrastructure and services. * Optimize compute, storage, networking, and GPU utilization to improve performance and reduce costs. * Monitor infrastructure health, availability, capacity, and performance using observability tools. * Implement security best practices, identity management, secrets management, and compliance controls. * Support AI model deployment platforms and inference infrastructure. * Troubleshoot infrastructure, networking, and performance issues affecting AI workloads. * Collaborate with AI engineers, ML engineers, data engineers, and cloud teams to improve platform reliability and scalability. * Evaluate and implement emerging infrastructure technologies for AI workloads.