HPC Infrastructure & Cluster Engineer

Marathon TS is seeking a TS/SCI-cleared Infrastructure Engineer to manage and optimize a high-performance compute environment supporting intensive AI/ML workloads. This role is heavily focused on Linux cluster administration, GPU infrastructure, workload scheduling, high-speed networking, storage, and containerized environments., * Cluster Administration: Manage the day-to-day operations of the customer compute cluster, including Linux operating system administration, hardware monitoring, patching, and system upgrades.

• Resource and Job Management: Configure, maintain, and optimize workload management and orchestration platforms, utilizing the Run:AI job scheduler to ensure efficient distribution of intensive AI/ML workloads across the cluster.
• Infrastructure Optimization: Tune cluster performance at the hardware, operating system, and network levels to maximize compute efficiency and data throughput for customer workloads.
• Storage and Network Management: Administer storage solutions and high-speed networking fabrics. Support the transition to and ongoing management of an InfiniBand GPU-to-GPU network infrastructure to minimize latency for distributed operations.
• Environment Configuration: Partner with technology integration teams to provision specific environments, dependencies, and container platforms, specifically leveraging Red Hat OpenShift, required for seamless customer model deployment.
• Security and Compliance: Ensure all infrastructure components remain compliant with federal security standards, implementing strict access controls and maintaining system accreditations.

Artificial Intelligence (AI), Sensitive Compartmented Information (SCI), Top Secret Clearance, Computer Systems, GPU (Graphics Processing Unit), Linux Operating System, Broadband, Linux Administration, Systems Administration/Management, Software Patches, * Clearance: Active TS/SCI with the ability to obtain CI Poly.

• Experience: 5+ years of experience in Linux systems administration and infrastructure management with a specific focus on high-performance computing environments.
• Technical Skills:
• Expertise in managing bare-metal servers, enterprise storage arrays, and advanced network configurations (Experience with InfiniBand).
• Strong proficiency with workload managers, job schedulers, and AI orchestration tools (e.g., Run:AI, SLURM).
• Hands-on experience with enterprise container orchestration platforms, specifically OpenShift or Kubernetes.
• Experience writing automation and configuration scripts (e.g., Bash, Python) to streamline cluster maintenance.
• Troubleshooting Focus: Proven ability to diagnose and resolve complex hardware, network, and OS-level issues.

Preferred Qualifications:

• Familiarity with parallel file systems and high-throughput storage architectures.
• Prior experience engineering or managing high-speed GPU-to-GPU communication topologies.