NAS Storage Engineer - Senior

Health Information Technology & Services (HITS) organization has an exciting opportunity to employ a Senior Storage Engineer to support and maintain our enterprise NAS storage infrastructure - providing support, provisioning, using, patching, maintaining, updating, and upgrading NAS storage systems. When hired, you will report to the HITS Enterprise Infrastructure Storage Services Team which implements and manages the Michigan Medicine Enterprise Storage environment and provides infrastructure services to many of the enterprise-wide applications used by Michigan Medicine (MM).

The storage team manages NAS, SAN, and File Systems. In the NAS storage farm, we currently have six Dell PowerScale clusters totaling 320 nodes and 37 PB (replicated), and we have two NetApp clusters totaling 3 PB. In addition, we have an LTFS Tape-as-NAS solution currently scaled to 4 PB, and a growing presence of Cloud-Volumes-OnTap. These systems provide network shares for clinical, research, and education customers within Michigan Medicine.

You will focus on NAS architecture for on-prem and cloud migration strategies, and operational support of the NAS storage hardware and software systems - monitoring, break/fix, capacity management, vendor support ticket coordination, design, naming standards, best practices, data center engineering documentation and coordination. You will work independently and anticipate and resolve issues to prevent service impacts. You will collaborate with other IT and customer teams for coordination of activities.

Responsibilities*

• Design and maintain system architecture to support a sustainable enterprise file system infrastructure. Keep informed on technological advances in file system storage solutions. Make recommendations on future solutions and improvements.
• Design and implement emerging Cloud technologies to support geo-redundant disaster recovery and high availability copies of on-premises data and storage systems.
• Implement and maintain NAS technologies (Dell/EMC/Isilon, NetApp, StrongLink, and Cloud). This includes but is not limited to daily operational work to monitor and maintain all aspects of the hardware and software running on the NAS arrays.
• Work with Stronglink NAS Archive using IBM 3592 LTFS
• Develop and document standards/procedures/strategies/scripts that maximize the functionality and reliability of the storage solution(s).
• Provide Storage technical assistance to HITS teams for storage needs, address applications performance issues and general platform problem analysis.
• Monitor the storage hardware/software and maintain/create monitoring and alerting scripts as needed to keep the system healthy.
• Coordinate with vendor and datacenter teams to implement break/fix procedures and improve/document/verify those procedures over time.
• Coordinate with the datacenter engineering team to maintain hardware inventories within Service Now, and coordinate additional documentation of hardware rack locations, specific installation details, cable management, and other data center related documentation. Develop reporting and process to validate information on an ongoing basis.
• Provide storage engineer support of the filesystem team that manages NTFS permissions. Design, engineering, and support work relating to enterprise design of NTFS permissions is required.
• Perform storage capacity monitoring and reporting to ensure that adequate data is available to assess the impact of growth trends and application changes.
• Assess current hardware and software technology and its applicability to our environment. Make recommendations for acquisitions, engineering, and configuration changes.
• Triage of Incident/Problem tickets related to permission, capacity, and file organization issues.
• You will work independently, using multiple resources (man pages, google, etc.) for troubleshooting., Positions that are eligible for hybrid or mobile/remote work mode are at the discretion of the hiring department. Work agreements are reviewed annually at a minimum and are subject to change at any time, and for any reason, throughout the course of employment. Learn more about the work modes.

Work Schedule

This position is being offered as a Hybrid position. There will be times that it will be appropriate for you to work from home. This position will be 40-60% remote work depending on the operational efforts, meetings, and project needs. May require working during non-business hours and on weekends.

Participation in a 24/7 on-call rotation required, including triage of Incident / Problem / Vulnerability tickets., Michigan Medicine conducts background screening and pre-employment drug testing on job candidates upon acceptance of a contingent job offer and may use a third party administrator to conduct background screenings. Background screenings are performed in compliance with the Fair Credit Report Act. Pre-employment drug testing applies to all selected candidates, including new or additional faculty and staff appointments, as well as transfers from other U-M campuses.

• Bachelor’s degree or equivalent combination of education and experience
• 5 or more years of experience in engineering support of large complex highly available enterprise NAS storage, either Dell PowerScale or NetApp
• 1-3 years of experience in design and implementation of cloud storage solutions, * Experience with design and deployment of storage systems using Isilon/PowerScale NAS hardware at multi-petabyte scale in auto-tiering and DR clustering.
• Experience with design and deployment of storage systems using NetApp FAS hardware running CIFS and NFS over Ethernet.
• Experience with design and implementation of hybrid-cloud storage systems for HA/DR.
• Ability to design and create scalable cloud storage solutions on AWS and/or Azure.
• Scripting skills with Unix shells (ksh, bash, etc.) or Windows PowerShell for custom monitoring and reporting.
• Working with vendors on patches, workarounds, or other best practices to resolve security vulnerabilities identified by scanning process.
• Experience with NAS migration tools (such as Rsync) to copy data from non-enterprise hardware into the NAS farm.
• Experience with design, deployment, and maintenance of NAS permissions (ACLs, NTFS) and NAS protocols (CIFS, NFS, etc.)
• 3 or more years of experience in a related role requiring customer service, organization, prioritization, troubleshooting to successfully design services and workflows in an enterprise architecture.
• History of working with customers to use standardized permission structures.
• Keep skills current with vendor technology as it changes and matures.
• Solid and effective interpersonal, written, and verbal communication skills.
• Communicate and work with employees at all levels within the organization.
• Excellent analytical and problem-solving ability.
• Strong organizational skills with attention to detail.

We offer a benefits package that includes comprehensive training and career development opportunities, generous retirement savings plans, ample paid time off, and a wealth of family care support: https://careers.umich.edu/benefits

Michigan Medicine improves the health of patients, populations, and communities through excellence in education, patient care, community service, research, and technology development, and through leadership activities in Michigan, nationally and internationally. Our mission is guided by our Strategic Principles and has three critical components; patient care, education and research that together enhance our contribution to society.

Health Information Technology & Services (HITS) plays an essential role in the success of Michigan Medicine by providing clinicians, educators, researchers, students, and staff with exceptional technology-related information, products, services, and support.

The HITS Enterprise Infrastructure (EI) division secures, manages, and architects Michigan Medicine data centers, networks, and communication infrastructures. The team oversees all Michigan Medicine IT-related technical infrastructure, including data centers and communication facilities; wired, wireless, and in-house cellular networks; spectrum management; communication; complex device management; fiber/wiring plant; hosting; directories, databases, and storage; and continuity of operations institution-wide. The team works with partners across Michigan Medicine to assure the security, stability, reliability, scalability, and economic management of the institution’s technology infrastructure.