TELECOMMUTE Data Engineer (Billing & Compensation Data Pipelines)

DKMRBH Inc.

United States

1 day ago

Apply on www.dice.com

Prepare application

Role details

Contract type

Permanent contract

Employment type

Full-time (> 32 hours)

Experience level

Expert

Experience required

5 years minimum

Working hours

Regular working hours

Job source

www.dice.com

Tech stack

Artificial Intelligence Airflow Microsoft Azure Cloud Database Directed Acyclic Graph (Directed Graphs) Data Validation Data Deduplication Extract Transform Load (ETL) Data Transformation Data Structures Software Debugging Document-Oriented Databases

+9 more

Job Scheduling Python (Programming Language) Scrum Methodology SQL Databases System Testing Transact-SQL Snowflake Build Management Data Pipelines

Job description

Design and build ETL/ELT pipelines pulling revenue and expense data from on-premise source systems into Snowflake.
Develop Airflow DAGs for scheduling, dependency management, retries, and monitoring of data pipeline jobs.
Implement calculation logic across multiple data points, such as gross vs. net revenue and expense allocations, in SQL (Snowflake SQL/T-SQL) or Python.
Perform data validation, reconciliation, and deduplication to ensure billing/revenue figures are accurate before downstream consumption.
Debug and maintain existing SQL views, including identifying column mismatches, join logic errors, and typos.
Optimize queries/views to avoid full-table scans and unfiltered CTEs that degrade production performance.
Build monitoring/alerting for pipeline failures, data quality issues, and SLA breaches within Airflow.
Document data lineage, transformation logic, and calculation rules to support audit and traceability requirements.
Collaborate with technical leads and business stakeholders to translate revenue/expense business rules into pipeline logic.

Requirements

Minimum 5 years of relevant professional experience.
Strong knowledge of Apache Airflow for job scheduling, orchestration, and DAG development.
Strong knowledge of Snowflake SQL and/or T-SQL, including CTEs, window functions, date-based transformations, and multi-source joins.
Experience with ETL/ELT development pulling data from on-premise systems into cloud data warehouses.
Experience with Python for pipeline scripting, custom operators, and data transformation logic.
Strong understanding of revenue and expense data structures.
Experience in writing and executing unit/system test cases for data pipelines.
Financial industry experience, preferably in investment management or billing/compensation systems.
Ability to manage time independently and work independently.
Experience with CI/CD pipelines, preferably Azure DevOps.
Experience in an Agile/Scrum environment is a plus.
Experience debugging SQL views for column mismatches, join errors, and performance issues is a plus.
Experience using AI capabilities is a plus.