Lead Azure Data Engineer & Architect

The Lead Big Data Engineer and Data Architect plays a critical role in big data development within the data analytics engineering organization of MetLife Data & Analytics. This position has the responsibility for architecture & design of data and analytics solutions, building ETL, data warehousing, and reusable components using cutting-edge big data and cloud technologies. The role is based in Cart, NC, Tampa, FL, Wilmington, DE, or NY, NY, and supports MetLife's commitment to data-driven decision-making and operational excellence., * Design and solution end-to-end data architecture for data hubs/data products and data all the way from source systems to consumption via web applications and reporting & analytics solutions.

• Ingest huge volumes of data from various platforms for Analytics needs and write high-performance, reliable, and maintainable ELT code
• Collect, store, process, and analyze large datasets to build and implement extract, transfer, load (ETL) processes
• Develop reusable frameworks to reduce the development effort involved, thereby ensuring cost savings for the projects.
• Develop quality code with thought-through performance optimizations in place right at the development stage.
• Appetite to learn new technologies and be ready to work on new cutting-edge cloud technologies.
• Work with teams spread across the globe in driving the delivery of projects and recommend development and performance improvements.
• Extensive experience with various database types and knowledge to leverage the right one for the need.
• Strong understanding of data tools and ability to leverage them to understand the data and generate insights
• Hands-on experience in building/designing at-scale Data Lake, Data warehouses, data stores for analytics consumption on-prem and Cloud (real-time as well as batch use cases).
• Utilize Cloud technologies (preferably Azure) to enable PaaS-centric enterprise solutions.
• Implement solutions that support dynamic scaling, including throttling and bursting for high-volume data workloads.
• Establish and evangelize modern software development practices, including CI/CD, automated testing, and code quality standards
• Develop and support an API catalog for data services, ensuring standardization and security.
• Optimize reusable frameworks, Spark jobs for performance and cost efficiency in large-scale environments.
• Ability to interact with business analysts and functional analysts in getting the requirements and implementing ETL solutions.

• 8+ years of overall experience and delivery experience with 6+ years of recent experience in data engineering.
• Bachelor's/ master's degree in information technology/computer science or a relevant domain
• Databricks certifications and/ or Microsoft Azure Certifications
• Strong analytic skills related to working with unstructured datasets.
• Data architecture (traditional - examples include Oracle & SQL Server + modern - examples include AWS & Azure) and knowledge of data architecture patterns.
• Strong experience in building/designing Data warehouses, data stores for analytics consumption on Cloud (real-time as well as batch use cases)
• Ability to interact with business analysts and functional analysts in getting the requirements and implementing the ELT solutions.
• Proficiency and extensive experience with Spark & Python/ Scala and performance tuning
• Hands-on experience building and implementing a data ingestion and curation process developed using Cloud data tools such as Cosmos DB, Data Factory, Spark (Scala/Python), Data bricks, Delta Lake, code versioning experience using Azure DevOps, etc.
• Very good problem solver and excellent communication skills - both written and verbal

Preferred:

• Experience using Event Hub for data integrations.
• Performance tuning on Azure SQL Synapse dedicated SQL Pool and server SQL Pools, Cosmos SQL APIs loading and consumption optimizations.
• Eagerness to learn new technologies on the fly and ship to production.
• Working knowledge of Azure DevOps pipelines.
• Good scripting experience primarily on shell/bash/ PowerShell would be desirable.
• Hive database management and Performance tuning - Partitioning / Bucketing.
• Prior experience leveraging AI and ML capabilities to automate and optimize complex workflows with intelligent use of low or no-code.

The MetLife Corporate Functions Data Office is part of the Data and Analytics Organization (D&A) within GTO. Our mission is to implement scalable data solutions for our stakeholders to generate actionable insights. We partner with Technology, and our Business and functional partners, to build and deploy the next generation data solutions for MetLife as we head into our New Frontier strategy!, At MetLife, we're leading the global transformation of an industry we've long defined. United in purpose, diverse in perspective, we're dedicated to making a difference in the lives of our customers.