Data Engineer - Python, dbt & Oracle

Our Client is seeking an experienced Data Engineer to support a major data infrastructure modernization initiative. This project involves assessing and modernizing approximately 700 800 legacy Perl scripts and RunMyJobs/PL/SQL scheduling workflows. The selected candidate will help migrate these legacy processes to a modular, observable, and testable data platform using Python, dbt, Apache Airflow, and Oracle 19c., * Assess, redesign, and migrate legacy Perl scripts, RunMyJobs workflows, and PL/SQL scheduling logic to Python scripts and Apache Airflow DAGs.

• Design, build, and deploy production-grade Airflow DAGs for complex, interdependent enterprise batch workflows.
• Develop custom Airflow operators and sensors for Oracle 19c, REST APIs, SFTP/FTP transfers, and external triggers.
• Build and maintain data transformation workflows using dbt.
• Write and optimize complex Oracle SQL and PL/SQL queries.
• Improve the performance and reliability of data pipelines operating across on-premises systems.
• Implement error handling, retry processes, SLA alerts, execution logs, audit trails, and data-lineage monitoring.
• Troubleshoot failed or partially completed jobs across application logs, databases, and Linux/Unix filesystems.
• Work closely with DBAs and application teams to document job dependencies, identify critical processing paths, and improve data quality and recoverability.
• Support parameter-driven automation and reusable pipeline-development standards.

• Strong hands-on experience with dbt and Oracle 19c. Both are mandatory.
• At least five years of Python development experience, particularly in automation, scripting, and enterprise ETL pipelines.
• At least two years of experience developing and deploying production-grade Apache Airflow DAGs, custom operators, and sensors.
• Strong experience with Oracle SQL, PL/SQL development, complex queries, and performance tuning.
• Advanced knowledge of Linux/Unix environments and shell-based processing.
• Experience modernizing or replacing enterprise job schedulers such as RunMyJobs or AutoSys.
• Experience developing REST API integrations and SFTP/FTP file-transfer processes.
• Strong understanding of batch processing, job dependencies, failure recovery, and operational monitoring.
• Proven ability to troubleshoot issues across application logs, databases, filesystems, and data pipelines., * Experience modernizing or replacing legacy Perl applications or scripts.
• Previous experience in banking, investment management, or financial services.
• Familiarity with financial data such as holdings, transactions, net asset values, and tax lots.
• Experience with Git or Bitbucket, Jenkins, and CI/CD-based deployment processes.
• Knowledge of operational risk, data governance, audit controls, and data lineage within financial systems.