Data Engineer - Python, dbt & Oracle

Interon IT Solutions LLC

Jersey City, NJ, United States

5 days ago

Apply on www.careerjet.com

Prepare application

Role details

Contract type

Permanent contract

Employment type

Full-time (> 32 hours)

Experience level

Expert

Experience required

2 years minimum

Working hours

Regular working hours

Job source

www.careerjet.com

Tech stack

Airflow CA Workload Automation Ae Batch Processing Unix Databases Continuous Integration Directed Acyclic Graph (Directed Graphs) Data Governance Data Infrastructure Extract Transform Load (ETL) Database Queries Linux

+16 more

File Systems Perl (Programming Language) Job Scheduling Python (Programming Language) Oracle (Applications) Oracle SQL Developer Performance Tuning PL-SQL Scripting File Transfer Protocol (FTP) Git Data Lineage Bitbucket Restful APIs Data Pipelines Jenkins

Job description

Our Client is seeking an experienced Data Engineer to support a major data infrastructure modernization initiative. This project involves assessing and modernizing approximately 700 800 legacy Perl scripts and RunMyJobs/PL/SQL scheduling workflows. The selected candidate will help migrate these legacy processes to a modular, observable, and testable data platform using Python, dbt, Apache Airflow, and Oracle 19c., * Assess, redesign, and migrate legacy Perl scripts, RunMyJobs workflows, and PL/SQL scheduling logic to Python scripts and Apache Airflow DAGs.

Design, build, and deploy production-grade Airflow DAGs for complex, interdependent enterprise batch workflows.
Develop custom Airflow operators and sensors for Oracle 19c, REST APIs, SFTP/FTP transfers, and external triggers.
Build and maintain data transformation workflows using dbt.
Write and optimize complex Oracle SQL and PL/SQL queries.
Improve the performance and reliability of data pipelines operating across on-premises systems.
Implement error handling, retry processes, SLA alerts, execution logs, audit trails, and data-lineage monitoring.
Troubleshoot failed or partially completed jobs across application logs, databases, and Linux/Unix filesystems.
Work closely with DBAs and application teams to document job dependencies, identify critical processing paths, and improve data quality and recoverability.
Support parameter-driven automation and reusable pipeline-development standards.

Requirements

Strong hands-on experience with dbt and Oracle 19c. Both are mandatory.
At least five years of Python development experience, particularly in automation, scripting, and enterprise ETL pipelines.
At least two years of experience developing and deploying production-grade Apache Airflow DAGs, custom operators, and sensors.
Strong experience with Oracle SQL, PL/SQL development, complex queries, and performance tuning.
Advanced knowledge of Linux/Unix environments and shell-based processing.
Experience modernizing or replacing enterprise job schedulers such as RunMyJobs or AutoSys.
Experience developing REST API integrations and SFTP/FTP file-transfer processes.
Strong understanding of batch processing, job dependencies, failure recovery, and operational monitoring.
Proven ability to troubleshoot issues across application logs, databases, filesystems, and data pipelines., * Experience modernizing or replacing legacy Perl applications or scripts.
Previous experience in banking, investment management, or financial services.
Familiarity with financial data such as holdings, transactions, net asset values, and tax lots.
Experience with Git or Bitbucket, Jenkins, and CI/CD-based deployment processes.
Knowledge of operational risk, data governance, audit controls, and data lineage within financial systems.