Hadoop Data Engineer

Qode LLC
Dallas, United States of America
yesterday

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Working hours
Regular working hours
Languages
English
Experience level
Senior

Job location

Dallas, United States of America

Tech stack

Java
Airflow
Amazon Web Services (AWS)
Azure
Big Data
Unix
Computer Programming
Information Engineering
ETL
Data Security
Data Systems
Data Warehousing
Hadoop
Hadoop Distributed File System
MapReduce
HBase
Hive
Python
NoSQL
Apache Oozie
Performance Tuning
Scrum
Cloud Services
Azure
SQL Databases
Sqoop
Data Streaming
Talend
Unstructured Data
Workflow Management Systems
Data Ingestion
Spark
Information Technology
Integration Frameworks
Kafka
Data Management
Data Pipelines
Amazon Web Services (AWS)

Job description

Hadoop Data Engineer responsible for designing, developing, and maintaining large-scale data processing systems within a distributed Hadoop ecosystem. The role focuses on enabling data-driven decision-making across banking operations, risk management, compliance, and customer analytics., * Design, develop, and maintain scalable data pipelines using Hadoop ecosystem tools (HDFS, Hive, Spark, Sqoop, Kafka).

  • Build and optimize ETL/ELT processes to support data ingestion from multiple banking systems.
  • Develop and manage big data solutions for structured and unstructured data.
  • Collaborate with data analysts, data scientists, and business stakeholders to deliver data solutions.
  • Ensure data quality, integrity, and governance aligned with banking and regulatory standards.
  • Perform performance tuning and optimization of Hadoop/Spark jobs.
  • Implement data security controls to comply with financial regulations (e.g., PCI, SOX).
  • Support real-time and batch data processing frameworks.
  • Troubleshoot production issues and provide continuous support for data platforms.
  • Work with cloud platforms (e.g., AWS, Azure) for modern data solutions.

Requirements

Do you have experience in UNIX?, Do you have a Master's degree?, * Strong experience with:

  • Hadoop ecosystem (HDFS, MapReduce, Hive, HBase)
  • Apache Spark (Scala/Python)
  • SQL & NoSQL databases
  • ETL tools (Informatica, Talend, or similar)
  • Kafka or other streaming tools
  • Proficiency in programming:
  • Python / Java / Scala
  • Experience with:
  • Data warehousing concepts
  • Workflow orchestration tools (Airflow, Oozie)
  • Unix/Linux environments
  • Knowledge of cloud data platforms (AWS EMR, Azure Data Lake) is a plus

Domain Knowledge

  • Understanding of banking and financial services data
  • Exposure to risk, compliance, or fraud analytics is preferred

Soft Skills

  • Strong problem-solving and analytical abilities
  • Excellent communication and collaboration skills
  • Ability to work in Agile/Scrum environments, * Bachelor's or Master's degree in:
  • Computer Science, Information Technology, or related field
  • Typically 5-10 years of experience in data engineering or big data development

Apply for this position