HPC Storage & Data Management Engineer #26-27412

• The HPC Storage & Data Management Engineer is a dedicated infrastructure specialist responsible for the design, administration, and ongoing optimization of storage environments within Client’s enterprise-scale, AWS-based HPC platform. This role exists within ADT but operates in close collaboration with the Digital Technology (DT) team in storage operations and data lifecycle management. The engineer serves approximately 1,500 global users supporting engineering simulation workloads across Gas Power and broader client business units., Storage Architecture & Optimization
• Manage and optimize the full HPC storage topology including home directories, high-performance local scratch/compute, project storage, and archive tiers; ensure appropriate data placement aligned with workload performance requirements.

Quota & Capacity Management

• Administer and enforce per-user and per-project storage quotas; monitor utilization trends across the user base and provide capacity forecasting inputs to the HPC Principal Engineer and DT team.

Dual-Protocol File Permission Management

• Manage file permissions and access control lists (ACLs) across multi-protocol (NFS/SMB) environments, ensuring consistent access governance for both Linux HPC and Windows VDI users.

Cost Visibility & Reporting

• Contribute to storage cost benchmarking and reporting, providing data-driven insight into storage spend within the AWS environment.

Data Lifecycle Management

• Define and enforce data lifecycle policies governing the progression of data through active, scratch, cold, and archive states; implement and maintain automated tiering and expiration workflows.

Archiving & Retention Strategy

• Lead the definition and implementation of an engineering simulation data archival standard; ensure alignment with the broader engineering simulation data retention strategy.

Collaborative Storage Governance

• Partner with DT to define and maintain collaborative team and project folder structures; establish and communicate standard work for shared HPC storage use.

AI Readiness Enablement

• Support the organization’s initiative to define engineering simulation data standards that ensure data is structured and accessible for AI/ML workloads.

• Demonstrated ability to develop and document standard operating procedures
• Experience in HPC storage administration in a prod
• Experience with quota management systems and storage capacity planning
• Expertise in high performance parallel/distributed file systems (e.g., Lustre, GPFS/IBM Spectrum Scale, BeeGFS, WekaIO, or similar)
• Proficiency in storage tiering, data lifecycle management, and archival tooling
• Solid Linux systems administration skills
• Strong experience with NFS and SMB/CIFS dual permission/ACL management

Nice To Have

• Experience in an AWS or cloud hybrid storage environment (S3, FSx for Lustre, EFS)
• Experience working within a DevOps or collaborative Infrastructure as Code model
• Familiarity with engineering simulation data (CFD, FEA, or similar) and associated file size/storage patterns
• Familiarity with Slurm job scheduler in the context of scratch and burst storage