Mid-level Data Engineer

JOB DESCRIPTION Insight Global is looking for a mid-level Data Engineer to join the Data Integrations team of a Marketing Software client. This person will migrate existing integrations to the Unified Data Platform, deliver new integrations using established patterns, and safely retire legacy pipelines. This hands-on mid-level engineer should be able to understand existing implementations, deliver scoped integration work, validate the results, and communicate clearly with the internal team.

KEY RESPONSIBILITIES ? Understand existing dbt and legacy Python integrations, including their source data, transformation logic, schedules, and downstream dependencies. ? Migrate integrations to the Unified Data Platform using patterns and guidance provided by the internal team. ? Configure Fivetran or Airbyte integrations where suitable; implement straightforward Python integrations when standard connectors are insufficient. ? Integrate with internal and external REST APIs, handling authentication, pagination, rate limits, retries, and incremental extraction. ? Configure transformations, schedules, dependencies, tests, and monitoring; support incremental loads, deduplication, and backfills. ? Validate migrated data against the source or legacy implementation and resolve routine discrepancies and failures. ? Submit changes through merge requests, address code review feedback, and use existing CI/CD and automated testing processes. ? Document implementation and operational details, support handover, and safely remove legacy pipelines after successful migration.

? Strong SQL skills and practical dbt experience. ? Practical Python skills: read, troubleshoot, modify, and port existing code; build straightforward custom integrations. ? Experience building and migrating data integrations and ETL/ELT pipelines. ? Understanding of REST API authentication, pagination, rate limits, retries, and incremental extraction. ? Understanding of incremental loading, CDC, idempotency, deduplication, schema evolution, and backfills. ? Understanding of SCD types and handling current versus historical data. ? Practical git-based development experience, including merge requests, code reviews, CI/CD, and automated testing. ? Basic Airflow experience covering DAGs, schedules, dependencies, retries, and monitoring. ? Attention to detail, particularly in data validation and safe migration. ? Ability to investigate routine issues independently and recognise when to escalate. ? Clear communication about progress, assumptions, blockers, and risks. ? Ability to follow platform standards, work with review feedback, and maintain useful documentation. ? BigQuery experience is preferred. ? Hands-on experience with Fivetran or Airbyte. ? Experience with migration reconciliation, cutovers, and legacy pipeline decommissioning. Scope and ownership