Data Engineer (Azure Data Factory)

We are seeking a skilled Senior Data Engineer with deep expertise in Azure Data Factory to design, build and optimize robust data pipelines across analytical workloads. In this role, you will author and deploy ADF pipelines, implement incremental load patterns and tune database performance to support enterprise-grade BI solutions. Responsibilities Author ADF pipelines using Copy Activity, Script Activity, ForEach, Execute Pipeline as well as parameterized datasets and linked services Debug failed runs in the ADF monitoring view using Azure IR Connect ADF instances to a Git repository such as Azure Repos or GitHub, working with the collaboration branch and feature branch model Publish ADF artifacts from the Git branch and deploy them to staging and production instances through a CI/CD pipeline Implement SQL Server Change Tracking with CHANGETABLE queries, version-based watermarking and the standard incremental load pattern documented by Microsoft Design star schemas with fact/dimension modeling for analytical workloads, including choosing grain, identifying dimensions and handling slowly changing dimensions Build merge functions and bulk loading processes into delta tables using COPY protocol and PL/pgSQL Optimize PostgreSQL performance through checkpoint config, WAL tuning and query optimization Configure PgBouncer session and transaction mode while understanding the I/O constraint chain across VM ceiling, disk throughput and connection limits Define indexing strategies for analytical queries and debug performance with EXPLAIN ANALYZE Requirements 3+ years of experience in data engineering with proven hands-on ADF pipeline authoring and deployment Expertise in ADF's Git-connected authoring model, export parameterization and deployment of artifacts through a CI/CD pipeline to multiple environments Proficiency in SQL Server Change Tracking, CHANGETABLE queries and version-based watermarking Skills in PostgreSQL fundamentals including COPY protocol, PL/pgSQL and indexing strategies for analytical queries Competency in PostgreSQL performance tuning covering checkpoint config, WAL tuning and PgBouncer session vs. transaction mode Background in star schema design with fact/dimension modeling and slowly changing dimensions Understanding of how BI tools generate SQL against star schemas Familiarity with EXPLAIN ANALYZE for query performance debugging English proficiency at B2 level or higher Nice to have Showcase of ThoughtSpot-specific experience Familiarity with live-query BI tools such as Looker, Tableau live connections or Power BI DirectQuery