Data Engineer - Airflow, dbt, Kubernetes/OpenShift

Conch Technologies

Jersey City, NJ, United States

5 days ago

Apply on www.careerjet.com

Prepare application

Role details

Contract type

Temporary to permanent

Employment type

Full-time (> 32 hours)

Experience level

Expert

Experience required

8 years minimum

Working hours

Regular working hours

Job source

www.careerjet.com

Tech stack

Artificial Intelligence Airflow Data Analysis Continuous Integration Information Engineering Fault Tolerance Python (Programming Language) OpenShift Oracle Warehouse Builder Performance Tuning Cloud Services SQL Databases

+13 more

Enterprise Data Management Data Processing Enterprise Software Applications Macros Sql Optimization Autoscaling System Availability Git Containerization Kubernetes Performance Monitor Data Management Data Pipelines

Job description

We are seeking a highly skilled Data Engineer with strong enterprise data engineering experience and deep expertise in Apache Airflow, dbt Core, Python, Kubernetes/OpenShift, and SQL. The role will focus on building, operating, and optimizing scalable data pipelines supporting financial, accounting, migration, and high-volume data processing workloads. Key Responsibilities

Design, develop, and maintain complex Apache Airflow DAGs for batch and event-driven pipelines.
Optimize Airflow scheduling, executors, workers, dependencies, retries, SLAs, and alerting.
Lead dbt Core implementation, including data modeling, testing, documentation, macros, incremental models, and CI/CD.
Develop staging, intermediate, and mart data models following analytics engineering best practices.
Deploy and manage data workloads on Kubernetes/OpenShift.
Implement workload distribution, horizontal scaling, resource optimization, and autoscaling.
Monitor and tune pipeline performance across Airflow, dbt, data platforms, and infrastructure.
Implement observability through logs, metrics, alerts, and proactive monitoring.
Ensure high availability, fault tolerance, security, and resiliency.
Collaborate with data architects, platform engineers, and business stakeholders.
Support financial reporting, accounting, regulatory data, and enterprise migration initiatives., Senior Staff Engineer, AI Compute (Remote Eligible) At Capital One, we are creating responsible and reliable AI systems, changing banking for good. For years, Capital One has been …
2 days ago

Requirements

10+ years of Data Engineering, Analytics Engineering, or Platform Engineering experience.
Expert-level Apache Airflow DAG development, orchestration, scheduling, and performance tuning.
Expert-level dbt Core modeling, testing, macros, implementation, and optimization.
Strong Python and advanced SQL skills.
Deep production experience with Kubernetes and/or OpenShift.
Strong understanding of distributed workload management and performance optimization.
Experience with cloud data platforms, containerized deployments, CI/CD, Git, and enterprise production environments.

Preferred Qualifications

Financial services or accounting platform experience.
Enterprise system migration experience.
Experience with Oracle data warehouses.
Experience supporting high-volume data processing and regulatory reporting.

Must-Haves * Expert Apache Airflow / DAG development and performance tuning.

Expert dbt Core / data modeling / testing / implementation.
Strong Python + SQL.
Production Kubernetes/OpenShift experience.
Enterprise-scale data engineering and distributed workload optimization experience., We are seeking a highly skilled Senior Data Engineer with 8+ years of hands-on experience in enterprise data engineering, including deep expertise in Apache Airflow DAG development…
15 hours ago, We are seeking a highly skilled Senior Data Engineer with 8+ years of hands-on experience in enterprise data engineering, including deep expertise in Apache Airflow DAG development…
16 hours ago