Data Engineer

ITAC
Birmingham, AL, United States
2 days ago
Apply on itacsolutions.com
Prepare application

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Compensation
$100,000
Working hours
Regular working hours

Tech stack

Artificial Intelligence Unit Testing Code Review Python (Programming Language) Operational Databases Standard Sql Apache Spark Git Pytest Data Lakes Pagination Api Management
+1 more
Databricks

Job description

  • Onboard new data sources end to end.
  • Build, run, and monitor production data pipelines.
  • Develop curated datasets used for reporting, applications, and AI.
  • Build API integrations with solid authentication and failure handling.
  • Write testable Python and deploy all work through Git and CI.
  • Partner with senior engineers on design and code review.
  • Document pipelines to the standards of a regulated environment.

Requirements

  • Be able to do the job as described.
  • Experience with Spark and Delta Lake, including incremental loads and reruns that don’t duplicate data.
  • Hands-on Databricks experience with jobs and workflows, plus familiarity with Unity Catalog.
  • Deploys through Git and CI, not by working in a workspace UI.
  • API integration experience, including pagination, authentication, and retry handling.
  • Testable Python, with unit testing such as pytest in your workflow.
  • Strong SQL, including window functions and tracking records that change over time.

About the company

Our client is a financial services firm building a modern data platform that brings data from many systems together into a single governed source of truth. That data feeds reporting, internal applications, and AI work across the business. You’ll help onboard new data sources, build the pipelines that move them, and develop the curated datasets the firm relies on every day.

Apply for this position

This job is hosted externally. Click below to view the full posting and apply.

Apply on itacsolutions.com
Prepare application

Good distractions

Loading talks and stories from around this role…