Lead Data Operations Engineer
Role details
Job location
Tech stack
Job description
Collaborate with teams (Cloud Operations, Global Information Security, Cloud DBAs, and Networking). Design, implement, and maintain AWS infrastructure and services within a managed service environment to implement infrastructure platforms that meet business requirements. Design, implement, and maintain AWS data lake environment. Lead migration efforts from Infrastructure for migration of legacy BigData Solution into EMR. Collaborate with Data Science teams to establish best practices for data ingestion and processing in the data lake, ensuring support for business users. Collaborate with platform, cloud operations teams to implement Terraform infrastructure, manage system installations, patches, and upgrades. Manage EMR, Talend, and IICS cluster workloads. Address performance issues in tools like Alation, MSTR, Tableau, and Alteryx. Work with AWS services such as EMR, IAM, EC2, VPC, Lambda, S3, EKS, ECS, KMS, Kinesis, Config, CloudTrail, EventBridge, RDS, SNS, SQS, ELB, and ASG. Troubleshoot issues related to applications, data, cloud infrastructure, Linux, and automation scripts. The position supervises direct reports. Job requires up to 5% of domestic travel.
Requirements
Bachelor's degree or equivalent, or higher, in Computer Science, Analytics, or a related field and five (5) years of progressive experience in Systems Engineering/Administration, or related occupation.
Requires progressive experience in data integration and proficiency in AWS cloud and Hadoop Big Data architecture; utilizing HDP ecosystem, CI/CD pipelines, and configuration management; using Linux, including Solaris, Ubuntu, CentOS, AIX, HP-UX, and Windows (NT/2000/XP/Vista/7/8); utilizing NetApp Filer, EMC, AWS S3, Apache Tomcat, WebLogic, and grid computing configuration and installations; applying source code management practices and systems, particularly Bamboo and GitHub; and deploying and managing AWS services including EC2, S3, EBS, IAM, ELB, CloudFormation, Terraform, and CloudWatch using both the AWS Console and AWS CLI.