Senior Data Engineer

Experteer Overview In this role you will design, build, and operate a Databricks-based data lakehouse on AWS to ingest, cleanse, and publish data for internal teams, client products, and AI systems. You will own end-to-end pipelines, provide technical leadership across onshore and offshore teams, and ensure governed, reliable data delivery in a regulated financial services environment. The opportunity centers on enabling data-driven insights and AI-enabled data access at scale. You will shape the platform, drive performance, and mentor engineers. Compensation / Benefits * Design, build, and operate end-to-end data pipelines on Databricks (Lakeflow, Delta Lake, Asset Bundles) deployed via CI/CD * Implement medallion architecture with raw -> cleansed -> conformed -> published layers, including quarantine handling * Ingest data from JDBC, file drops, SaaS APIs, and CDC feeds using Auto Loader and incremental patterns * Migrate workloads from Airflow (MWAA) to Databricks-native orchestration and optimize Spark/SQL performance * Model data with dbt, PySpark, and SQL for BI, APIs, and AI data access; ensure testable transformations * Implement data quality and reconciliation within pipelines; maintain production monitoring and SLA adherence * Governance in Unity Catalog, including tagging, lineage, and entitlements for multi-tenant access * Deliver data outbound via Snowflake, secure views, exports, and APIs; onboard new clients * Publish curated datasets for AI engineering and maintain Genie spaces and MCP services for AI agents * Provide technical leadership, design reviews, standards, and platform roadmap; communicate with non-technical stakeholders Tasks * 6+ years of data engineering experience in production cloud data platforms * Demonstrated technical leadership: mentoring, design leadership, delivery ownership * Bachelor's or Master's degree in Computer Science, Engineering, or related field or equivalent experience Key requirements * Alter Domus Academy with six learning zones * Support for professional accreditations * Flexible arrangements and generous holidays (plus birthday leave) * Continuous mentoring and career progression * Employee Assistance Program (EAP) * Employee Share Plan

New and optimize Spark/SQL performance * Model data with dbt, PySpark, and SQL for BI, APIs, and AI data access; ensure testable transformations * Implement data quality and reconciliation within pipelines; maintain production monitoring and SLA adherence * Governance in Unity Catalog, including tagging, lineage, and entitlements for multi-tenant access * Deliver data outbound via Snowflake, secure views, exports, and APIs; onboard new clients * Publish curated datasets for AI engineering and maintain Genie spaces and MCP services for AI agents * Provide technical leadership, design reviews, standards, and platform roadmap; communicate with non-technical stakeholders Tasks * 6+ years of data engineering experience in production cloud data platforms * Demonstrated technical leadership: mentoring, design leadership, delivery ownership * Bachelor's or Master's degree in Computer Science, Engineering, or related field or equivalent experience Key requirements * Alter Domus Academy with six aaa aaaL_ zones * Support for professional accreditations * Flexible arrangements and generous holidays (plus birthday leave) * Continuous mentoring and career progression * Employee Assistance Program (EAP) * Employee Share Plan