Lead Data Engineer
Role details
Job location
Tech stack
Job description
At IT Labs, we are looking for an experienced Lead Data Engineer to support one of the most innovative companies in the digital health space.
Our client is transforming healthcare through a technology-driven platform that connects employers and patients with leading medical providers, helping reduce costs and improve outcomes. As they continue to scale, they are investing heavily in modernizing their data infrastructure and analytics capabilities.
As a Lead Data Engineer, you will drive the design and evolution of the company's data platform, building scalable data pipelines and robust Snowflake-based solutions. Working closely with cross-functional teams, you will help deliver reliable, high-quality data that powers business decisions and supports the company's growth.
What You'll Do
- Lead the design, development, and optimization of scalable data platforms and pipelines.
- Design, build, and maintain production-grade ETL/ELT workflows for batch and near real-time data processing.
- Drive the migration and modernization of data assets from BigQuery and other analytical platforms into Snowflake.
- Develop robust integrations with enterprise applications, APIs, and external data providers.
- Build and optimize Snowflake data models, schemas, Snowpipe ingestion processes, and query performance.
- Collaborate with business stakeholders, analysts, and engineering teams to translate business requirements into scalable data solutions.
- Implement data quality, validation, monitoring, and observability frameworks.
- Establish and promote best practices for data governance, lineage tracking, metadata management, and security.
- Support both batch and streaming data architectures using technologies such as Kafka, Event Hub, or equivalent.
- Mentor other engineers and contribute to architectural decisions, technical standards, and engineering best practices.
Ensure data platforms meet performance, reliability, scalability, and regulatory requirements.
Requirements
Do you have experience in Technology security practices?, * 7+ years of experience in data engineering within cloud or hybrid environments.
- Strong experience designing, building, and maintaining production-scale ETL/ELT pipelines.
- Advanced SQL skills and proficiency with data transformation techniques.
- Strong Python programming skills for data processing and automation.
- Hands-on experience with Snowflake, including:
- Data modeling
- Snowpipe
- Query optimization
- Schema design
- Experience migrating from BigQuery or similar columnar data warehouse technologies.
- Experience with batch and streaming data processing platforms such as Kafka, Event Hub, or equivalent.
- Experience building enterprise-grade API and source-system integrations.
- Experience implementing data quality and validation frameworks.
- Good understanding of:
- Data governance
- Data lineage
- Access control and security best practices
- Strong problem-solving and communication skills.
- Ability to work effectively in a collaborative, cross-functional environment.
Nice to Have
- Experience with Databricks, including:
- Delta Lake
- Unity Catalog
- Spark-based pipelines
- Experience with Spark, Scala, or PySpark.
- Familiarity with Tableau.
- Understanding of HIPAA and HITRUST data handling practices.
- Experience with dbt (data build tool) for transformation layer management.
- Experience working within healthcare, insurance, or regulated industries.
Benefits & conditions
- Full-time, 40h/week
- US-based - US citizenship is required
- Remote
- Contract or B2B arrangement
Our values
We are a company that seeks the best for both our employees and clients, reaching beyond expectations in turning dreams into reality. Our way of working is rooted in our core values (Integrity, Excellence, Proactivity, Innovation, and People), with an expectation that our future colleagues will make these their second nature in their everyday work and life. We don't ask for perfection, but we do appreciate people motivated to better themselves in every conceivable aspect.