Lead Data Engineer

The Lead Data Engineer will design, build, and maintain the organization’s enterprise data platform, leading the technical implementation of data pipelines, warehouses, and analytics infrastructure that powers business intelligence, reporting, and advanced analytics across the PBM and Pharmacy organization. This hands-on technical leadership role sets data engineering standards, mentors team members, and partners with business and technology stakeholders to deliver trusted, well-governed, and timely data products.

• Essential Duties and Responsibilities
• Data Platform Engineering:
• Design and build scalable, reliable data pipelines that ingest, transform, and load data from operational systems, clinical platforms, claims, and third-party sources.
• Develop and maintain the enterprise data warehouse, data lake, and analytical data models that serve reporting and analytics use cases.
• Design data services and event-driven integration patterns that enable scalable downstream consumption by analytics platforms, operational systems, APIs, and AI-enabled applications.

Technical Leadership:

• Serve as the senior technical authority for data engineering, setting standards for code quality, pipeline design, data modeling, and testing across the team.
• Lead technical planning for data engineering initiatives, breaking work into well-scoped tasks and coordinating delivery across team members.
• Mentorship and Collaboration:
• Provide technical direction and code review for data engineers, ensuring consistency, quality, and adherence to standards.
• Participate in hiring and onboarding of data engineering team members, including technical interviews and skills assessments.
• Mentor data engineers across all levels, fostering a culture of technical excellence, knowledge sharing, and continuous improvement.

Data Architecture and Modeling:

• Partner with the Architecture team to define and implement data architecture, including data warehouse models, data lake structures, and integration patterns.
• Apply dimensional modeling, normalization, and modern data modeling techniques (e.g., Kimball, Data Vault) to support analytics and reporting requirements.

Performance and Reliability:

• Optimize query performance, storage costs, and pipeline runtime across the data platform.
• Implement observability, monitoring, and alerting for production data pipelines, and partner with operations to ensure timely incident response.
• Identify reliability, data quality, and performance risks and develop mitigation strategies to ensure platform stability and data trustworthiness.

Data Governance and Compliance:

• Implement controls to ensure compliance with HIPAA, PHI/PII handling, and other regulatory requirements applicable to the healthcare and pharmacy sectors.
• Partner with security and compliance teams on access control, encryption, audit logging, and data lineage for sensitive data assets.
• Design and enable scalable, governed data access patterns that support AI/ML systems, intelligent automation, and emerging agentic workflows, including structured, semantic, and real-time data consumption patterns.

Business Partnership:

• Partner with analytics, business intelligence, and product teams to understand data needs and deliver fit-for-purpose datasets, models, and pipelines.
• Translate business and reporting requirements into well-designed technical data engineering solutions.

Tooling and Innovation:

• Evaluate and recommend new data engineering tools, frameworks, and cloud services that improve productivity, scalability, or cost-efficiency.
• Stay current on advances in cloud data platforms, lakehouse architectures, streaming technologies, and AI/ML data infrastructure.

Support and Troubleshooting:

• Provide production support for critical data pipelines, participating in on-call rotations as needed.
• Diagnose and resolve complex data quality, performance, and integration issues spanning multiple systems and platforms.

Operational Oversight:

• Implement data quality validation, backup and recovery, and pipeline monitoring to ensure continuous data delivery.
• Recommend tooling and infrastructure needed to support the enterprise data platform.
• Prepare and review data platform health metrics, pipeline performance reports, and project status updates.

Documentation:

• Implement and maintain metadata, lineage, cataloging, and semantic data definitions that improve discoverability, trust, and machine usability of enterprise data assets.

Do you have experience in Tooling?, Do you have a Master’s degree?, * Bachelor’s degree in Computer Science, Information Systems, Data Engineering, or related field or relevant experience. Master’s degree in a relevant discipline preferred.

• 8+ years of professional data engineering experience, with at least 2 years in a senior or technical lead capacity preferred.
• Hands-on experience designing and operating enterprise data platforms in a healthcare, pharmaceutical, or Pharmacy Benefit Management environment preferred.
• Prior experience implementing real-time data streaming pipelines that power reporting, dashboards, and operational visibility preferred.

Knowledge, Skills, and Abilities

• Expert-level proficiency in SQL, Python, and modern data engineering frameworks (e.g., Spark, dbt, Airflow).
• Deep experience with cloud data platforms (Snowflake, Databricks, Redshift, or BigQuery) and storage layers (S3, ADLS).
• Strong understanding of data modeling, warehousing patterns (Kimball, Data Vault), and lakehouse architectures.
• Strong understanding of regulatory standards affecting the healthcare and pharmacy sectors, including HIPAA.
• Proficient in modern cloud platforms (AWS, Azure), CI/CD for data, and infrastructure-as-code tools (Terraform, CloudFormation).
• Experience building data foundations for AI agents and RAG-based applications, including semantic modeling, metadata enrichment, vector-search integration, governed APIs/tools, and secure access patterns for machine-consumable enterprise data.
• Familiarity with Microsoft Power BI, including semantic models, datasets, and enablement of self-service reporting and dashboards.
• Familiarity with real-time and streaming data technologies (e.g., Kafka, Kinesis, Spark Streaming, Flink) supporting reporting and operational visibility.
• Excellent communication skills, capable of explaining technical concepts to both engineering and business stakeholders.
• Ability to lead technical initiatives end-to-end while mentoring engineers and driving quality and reliability.