Data Engineer
ITAC
Birmingham, AL, United States
2 days ago
- Discuss this with your agent
- Open in Claude
- Open in ChatGPT
Apply on itacsolutions.com
Prepare application
- Draft this with your agent
- Open in Claude
- Open in ChatGPT
Role details
Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Compensation
$100,000
Working hours
Regular working hours
Job source
Tech stack
Artificial Intelligence
Unit Testing
Code Review
Python (Programming Language)
Operational Databases
Standard Sql
Apache Spark
Git
Pytest
Data Lakes
Pagination
Api Management
+1 more
Databricks
Job description
- Onboard new data sources end to end.
- Build, run, and monitor production data pipelines.
- Develop curated datasets used for reporting, applications, and AI.
- Build API integrations with solid authentication and failure handling.
- Write testable Python and deploy all work through Git and CI.
- Partner with senior engineers on design and code review.
- Document pipelines to the standards of a regulated environment.
Requirements
- Be able to do the job as described.
- Experience with Spark and Delta Lake, including incremental loads and reruns that don’t duplicate data.
- Hands-on Databricks experience with jobs and workflows, plus familiarity with Unity Catalog.
- Deploys through Git and CI, not by working in a workspace UI.
- API integration experience, including pagination, authentication, and retry handling.
- Testable Python, with unit testing such as pytest in your workflow.
- Strong SQL, including window functions and tracking records that change over time.
About the company
Our client is a financial services firm building a modern data platform that brings data from many systems together into a single governed source of truth. That data feeds reporting, internal applications, and AI work across the business. You’ll help onboard new data sources, build the pipelines that move them, and develop the curated datasets the firm relies on every day.
Apply for this position
This job is hosted externally. Click below to view the full posting and apply.
Apply on itacsolutions.com
Prepare application
- Draft this with your agent
- Open in Claude
- Open in ChatGPT
Good distractions
Loading talks and stories from around this role…