> Markdown version of [/jobs/ext/3963034-senior-lead-devops-engineer-on-prem-infrastructure-remote](https://www.wearedevelopers.com/jobs/ext/3963034-senior-lead-devops-engineer-on-prem-infrastructure-remote). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Senior/Lead DevOps Engineer (On-Prem Infrastructure) - Remote - **Company:** SoftServe, Inc. - **Location:** New York, NY, United States (Remote available) - **Experience:** Expert - **Salary:** $113,120 - $141,400 - **Contract:** Permanent contract - **Skills:** Artificial Intelligence, Cloud Computing, Computer Clusters, Cyber Security, DevOps, Virtual Private Networks (VPN), Peering, Inference Optimization, Software Deployment, Graphics Processing Unit (GPU), Load Balancing, Delivery Pipeline, Multi-Cloud, Firewalls (Computer Science), Gitlab-ci, Kubernetes, Enterprise Integration, Machine Learning Operations, Hardware Infrastructure, Software Version Control - **Published:** October 27, 2026 - **Apply:** https://www.careerbuilder.com/job-details/senior-lead-devops-engineer-on-prem-infrastructure-remote-new-york-ny--5358166e-3268-40f5-94dd-209c9b812c41 ## About the Role DevOps, Banking Services, Team Lead/Manager, GPU (Graphics Processing Unit), Artificial Intelligence (AI), Cloud Computing, Security Infrastructure, Network Integration, Operating Systems, Systems Administration/Management, * 5+ years of experience in DevOps/Infrastructure Engineering, with deep hands-on expertise in on-premises Kubernetes * Strong experience managing GPU compute infrastructure (NVIDIA BCM, Run: AI, or equivalent) * Proficiency with GitLab CI/CD for pipeline automation * Solid networking fundamentals (firewalls, load balancing, VPNs, peering) * Experience working directly with banking or other regulated-industry clients * Prior cloud exposure (GCP or similar) is a plus, though not required for this role * Demonstrated LLMOps experience, including model versioning, inference optimization, production monitoring and evaluation, and ongoing lifecycle management of deployed models (required) ## Description Our client builds and operates hybrid infrastructure, spanning on-premises GPU clusters and multi-cloud environments, for enterprise banking clients. Youll join a delivery team managing large-scale Kubernetes and cloud infrastructure for demanding, security-critical AI/ML and business workloads., * Manage and operate large-scale on-premises Kubernetes clusters, with hands-on ownership of CNI and CSI configuration, networking, and storage integration * Manage operating systems across GPU nodes using NVIDIA BCM * Optimize GPU resource allocation and model performance tuning using NVIDIA Run: AI and GPU/model parameter configuration * Set up and manage CI/CD pipelines using GitLab CI/CD for infrastructure and application delivery * Configure networking, firewalls, and load balancing across on-prem environments * Ensure infrastructure meets banking-grade compliance and security standards * Collaborate directly with client stakeholders and lead small delivery teams