Azure Data Engineer

We are seeking a Senior Data Engineer to design, build, and operate highly scalable batch and streaming data pipelines supporting T Mobile$B!G(Bs Finance and Intelligence platforms. This role requires deep expertise in modern cloud data stacks (Snowflake, Databricks, dbt), strong SQL/Python skills, and solid understanding of finance data domains including billing, revenue, GL, and OPEX. The ideal candidate owns complex pipelines end to end, mentors junior engineers, and helps drive platform standards and best practices., Data Pipeline Development Design and build scalable, reliable ELT/ETL pipelines for finance data (billing, revenue, GL, OPEX). Implement batch and incremental ingestion patterns (full load, CDC, watermark-based). Build idempotent, rerunnable pipelines with robust error handling, retry logic, and dead-letter queue patterns. Platform & Tooling Develop and optimize pipelines using Snowflake (Snowpipe, Streams, Tasks, Dynamic Tables, performance tuning). Build data processing workflows in Databricks (PySpark, Delta Live Tables, Unity Catalog, job clusters). Create and maintain dbt models, tests, snapshots, macros, and packages with CI integration. Orchestrate data workflows using Airflow or Azure Data Factory (DAG design, dependencies, scheduling, alerts). Cloud Infrastructure Work within Azure (ADLS Gen2, Event Hub, ADF, Azure Functions, Key Vault) and/or AWS (S3, Glue, Lambda, Secrets Manager). Apply Infrastructure as Code fundamentals (Terraform, Bicep) for pipeline and resource provisioning. Apply cloud cost awareness including compute sizing, partitioning strategies, and storage optimization. Languages & Frameworks Write advanced SQL (CTEs, window functions, query tuning, execution plan analysis). Develop in Python (pandas, PySpark, requests, pytest, logging). Read and modify existing Scala/Spark jobs as needed. Use shell scripting for automation and operational tasks. Streaming & Real Time Processing Build near real time pipelines using Apache Kafka / Azure Event Hub. Implement Spark Structured Streaming with stateful aggregations, watermarking, and checkpointing. Support finance use cases such as revenue reconciliation and fraud signal feeds. Data Quality & Testing Implement unit and integration testing for pipelines (pytest, dbt tests). Create data quality checks (row counts, nulls, duplicates, referential integrity). Use Great Expectations or custom frameworks for validation. Monitor SLAs for pipeline latency and data freshness with alerting. Data Modeling Support Implement architected schemas (star, snowflake, data vault). Manage Slowly Changing Dimensions (SCD Type 1 & 2) for finance entities. Define partitioning and clustering strategies for large-scale finance tables. Support semantic layer definitions (metrics and dimensions). DevOps & Engineering Practices Participate in CI/CD for data pipelines using GitHub Actions or Azure DevOps. Follow Git branching strategies (trunk-based, feature branches). Perform code reviews and enforce engineering standards. Support environment promotion patterns (dev $B"(B QA $B"(B prod). Security & Governance Implement RBAC and row/column-level security in Snowflake and Databricks. Ensure PII and CPNI handling per T Mobile TISS 310 policy. Manage secrets securely (Key Vault, environment variables, no hardcoded credentials). Implement data lineage and audit instrumentation for compliance. Collaboration & Communication Partner with Data Architects to translate design specs into production-ready pipelines. Work closely with Data Analysts to optimize downstream consumption performance. Communicate pipeline incidents and data issues clearly to business stakeholders. Participate in on-call rotation to support production pipelines.

Senior-Level Expectations Own delivery of complex, multi-source pipelines with minimal direction. Mentor junior and mid-level data engineers through pairing and code reviews. Identify and drive technical debt reduction alongside feature delivery. Contribute to and shape team standards, templates, and reusable components. Influence tooling, framework, and platform decisions across the team.

8+ years of experience in data engineering or platform engineering roles. Strong experience with Snowflake, Databricks, and dbt in production environments. Advanced SQL and Python skills. Experience building finance or regulated data pipelines at scale. Preferred Qualifications Telecom industry experience (ARPU, churn, prepaid/postpaid metrics). Experience with both Azure and AWS cloud platforms. Prior experience supporting financial reporting and period-end close cycles.