Senior Data Engineer

Mizuho Americas Services LLC

Jersey City, NJ, United States

2 days ago

Role details

Contract type

Permanent contract

Employment type

Full-time (> 32 hours)

Experience level

Expert

Experience required

5 years minimum

Compensation

$105,000.0 - $190,000.0

Working hours

Regular working hours

Job source

www.jofdav.com

Tech stack

Application Programming Interfaces (APIs) Airflow Confluence JIRA Microsoft Azure Big Data Databases Continuous Integration Information Engineering Data Integration DevOps Python (Programming Language)

+19 more

Online Analytical Processing Online Transaction Processing Scrum Methodology Reference Data SQL Databases SQL Server Integration Services Data Streaming UML Data Processing Cloud Platform System Test-Driven Development (TDD) Pandas Pyspark Core Data Apache Kafka Software Version Control Data Pipelines Servicenow Databricks

Job description

Building and maintaining data pipelines to ingest data from source systems into Databricks
Designing, Building and maintaining generic frameworks to ingest data through the medallion layers
Working with the core data platform team to integrate the framework with reference architecture
Assisting the data mesh teams with best practices and onboarding to the new framework

Requirements

10+ years of data engineering experience in building OLTP and OLAP databases, both on-prem and in the cloud
5+ years of experience with large scale data processing using SQL, Python, Pandas and Pyspark
5+ years of experience building robust and scalable data pipelines in Databricks Lakehouse platform on any key cloud platform provider, preferably Azure cloud
Experience with implementing Medallion architecture for data processing
Experience with implementing Streaming data ingestion using Kafka
Test driven development experience
Proven experience in solution design and applying best design practices
Data modelling and ER/UML diagram skills
Experience with orchestration/enterprise scheduler platforms like Airflow, Tidal etc..
Experience with implementing API based data integration pipelines
Experience with source control and DevOps for CI/CD
Experience working in Agile/scrum methodologies
Experience with Jira, confluence for project work and documentation
Experience with Service now for incident management
Finance domain knowledge in Investment Banking (Securities, Capital markets) and Security reference data knowledge
Nice to have ADF and SSIS experience
College degree required.

Benefits & conditions

The expected base salary ranges from $105,000 - $190,000. Salary offers are based on a wide range of factors including relevant skills, training, experience, education, and, where applicable, certifications and licenses obtained. Market and organizational factors are also considered. In addition to salary and a generous employee benefits package, including Medical, Dental and 401K plans, successful candidates are also eligible to receive a discretionary bonus.