Data Lakehouse Engineer

The Position is for a skilled and detail-oriented Data Lakehouse Engineer with a strong background in Enterprise Data Office (EDO) principles to design, build, and maintain next-generation data lakehouse platforms. This role combines technical excellence with data governance and compliance, supporting enterprise-wide data strategies and analytics initiatives.

The Position will play a key role in the following assignments:

Building snowflake external tables against data saved in AWS S3 in Apache Iceberg.

Using existing pipeline to migrate data from one AWS account in S3 to another AWS account where the data is stored in Apache iceberg tables on S3.

Other Responsibilities:

Design and implement scalable data lakehouse architectures using modern open formats such as Apache Iceberg.

Build and manage Snowflake external tables referencing Apache Iceberg tables stored on AWS S3.

Develop and maintain data ingestion, transformation, and curation pipelines, supporting structured and semi-structured data.

Collaborate with EDO, data governance, and business teams to align on standards, lineage, classification, and data quality enforcement.

Implement data catalogs, metadata management, lineage tracking, and enforce role-based access controls.

Ensure data is compliant with privacy and security policies across ingestion, storage, and consumption layers.

Optimize cloud storage, compute usage, and pipeline performance for cost-effective operations

Proven experience building Snowflake external tables over Apache Iceberg tables on AWS S3.

Strong understanding of Iceberg table concepts like schema evolution, partitioning, and time travel.

Proficient in Python, SQL and Spark or other distributed data processing frameworks.

Solid understanding of AWS data services including S3, Glue, Lake Formation, and IAM.

Hands-on with data governance tools (e.g., Collibra/ Alation/ Informatica) and EDO-aligned processes.

Experience implementing metadata, lineage, and data classification standards.

Good to Have Skills:

Experience in Insurance Domain

Experience migrating JSON files to Apache Iceberg tables on AWS S3 using configuration-driven rules.

Familiarity with ETL orchestration tools such as Apache Airflow, DBT, or AWS Step Functions.

Prior exposure to MDM, data quality frameworks, or stewardship platforms