Senior Data Engineer

We are hiring for a Senior Data Engineer for an end-client in the Ad-Media Measurement space. This is for the Ad Optimization team - the revenue-facing edge of the data platform that turns attributed conversions and impressions into optimization feeds that external partners use to make their advertising models smarter. This is a Contractor (1099) role and includes availability for occasional off-hours release / incident support. Formal on-call is not required unless separately established. Responsibilities

• Build, optimize, and operate production Apache Spark pipelines that generate attributed conversion and optimization feeds from large-scale impression and conversion data
• Design and evolve data processing and models across modern data lake and warehouse technologies, orchestrated with a production workflow scheduler on AWS
• Build and operate partner optimization feeds and integrations with external partners, including schemas, identity fields and unique IDs, file delivery, reconciliation, and SLAs
• Build, change, and operate production REST APIs, including PHP APIs, delivering secure, backward-compatible changes end-to-end: implementation, validation, authorization, data access, documentation, automated testing, and production diagnostics
• Trace and modify behavior across controllers, database queries, templates, JavaScript, and automated tests
• Diagnose production failures, reconcile partner-facing data, execute backfills, and manage staged releases and rollbacks
• Write and maintain unit and integration tests across pipelines, APIs, and feeds

• 7+ years of professional experience in software or data engineering
• 4+ years building, optimizing, and maintaining production Apache Spark pipelines, with strong Java (JVM Spark) AND Python (PySpark) skills
• Advanced SQL, relational data modeling, and large-scale data processing; production experience with Snowflake, MySQL, and Iceberg
• 3+ years operating AWS data workloads using S3 and Airflow (or equivalent production orchestration); container orchestration and data-catalog experience a plus
• 3+ years building, changing, and operating production REST APIs
• Production full-stack experience with PHP, Symfony (or comparable PHP MVC), Doctrine, Twig, and JavaScript
• Demonstrated AI-assisted development - Must be comfortable using AI tools for code generation, code review, documentation, investigation, working with agentic code harnesses, and spec-driven development, with judgment on when NOT to trust AI output. This is an AI-forward engineering org.
• Ability to independently deliver secure, backward-compatible API changes
• Experience with external provider / client integrations involving schemas, APIs, file delivery, identity fields, reconciliation, and SLAs

Preferred / Nice to Have

• Direct experience building partner optimization or advertising feeds with external platforms (DSPs, publishers, measurement partners)
• Experience in AdTech / Advertising measurement - attribution, conversions, audience/impression data, identity matching
• Experience with deterministic and probabilistic identity matching, unique IDs, device IDs, identity graphs
• Familiarity with edge/log delivery infrastructure and pixel/impression tracking
• Experience with privacy and compliance-driven workflows (deletion, opt-out, suppression, retention)
• Experience with data lake table formats and query engines at large scale, * 4+ yrs Java (JVM Spark) + Python (PySpark) prod?
• 3+ yrs S3 + Airflow/orchestration?
• 3+ yrs building/operating prod REST APIs?
• AI-assisted dev tools in professional capacity?
• Prod experience with Snowflake, MySQL, Iceberg?
• PHP/Symfony, Doctrine, Twig, JS prod experience?
• US Citizen / Green Card?