Developer Hadoop

S&K GLOBAL SOLUTIONS LLC
Pennington, United States of America
2 days ago

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Working hours
Regular working hours
Languages
English
Experience level
Senior

Job location

Pennington, United States of America

Tech stack

Java
Airflow
Cloudera Impala
Information Engineering
Data Governance
ETL
Hadoop
Hadoop Distributed File System
Hive
Python
Apache Oozie
Performance Tuning
Standard Sql
Scala
Workflow Management Systems
Apache Yarn
Data Lake
PySpark
Kafka
Spark Streaming
Data Management
Stream Processing
Control M

Requirements

Must Have Technical/Functional Skills Primary Skill: Data Engineering, Platform Engineering or architecture roles. Deep Expertise in Pyspark. Experience: 10+ yrs Roles & Responsibilities

  • Deep Expertise in PySpark, including performance tuning and optimization
  • Strong python development experience in large-scale distributed environment
  • Solid knowledge of Hadoop ecosystem (HDFS,Hive/Impala, YARN)
  • Proven experience designing and governing enterprise, regulatory facing data platforms.
  • Expertise in designing data lakes, ELT/ETL pipelines, batch and real time data processing solution
  • Proficiency in programming languages such as Java, Scala and SQL
  • Strong understanding of non-functional requirements and production support models
  • ·Clear written and verbal communication skills with ability to influence across organizations

Preferred Skills / Experience

  • Financial services experience, particularly in Market Surveillance, AML, Fraud, Or Risk Technology
  • Experience supporting regulatory or audit facing platforms
  • Kafka and Spark Structured streaming exposure
  • Familiarity with Orchestration tools(Airflow,Control-M,Oozie)
  • Knowledge of data governance, lineage, and data quality controls

Apply for this position