Hadoop Developer

IBA InfoTech Inc.

Charlotte, NC, United States

about 2 months ago

Role details

Contract type

Permanent contract

Employment type

Full-time (> 32 hours)

Working hours

Regular working hours

Job source

ibainfotech.com

Tech stack

Java (Programming Language) Agile Methodology Data Analysis Apache HTTP Server Big Data Databases Data as a Services Extract Transform Load (ETL) Data Systems Data Warehousing DevOps Apache Hadoop

+15 more

Python (Programming Language) Standard Sql Scala (Programming Language) Simple Data Format Sqoop Enterprise Data Management Parquet Data Ingestion Apache Spark Virtual Environment Data Lakes Apache Flume Information Technology Data Management Data Pipelines

Job description

As a Hadoop Developer, the ideal candidate for this developer role should be able to:

Build high performing data models on big-data architecture as data services.
Build a high performing and scalable data pipeline platform using Hadoop, Apache Spark.
Partner with Enterprise data teams such as Data Management & Insights and Enterprise Data Environment (Data Lake) and identify the best place to source the data
Work with business analysts, development teams and project managers for requirements and business rules.
Collaborate with the source system and approved provisioning point (APP) teams, Architects, Data Analysts and Modelers to build scalable and performant data solutions.
Effectively work in a hybrid environment where legacy ETL and Data Warehouse applications and new big-data applications co-exist
Work with Infrastructure Engineers and System Administrators as appropriate in designing the big-data infrastructure.
Work with DBAs in Enterprise Database Management group to troubleshoot problems and optimize performance
Support ongoing data management efforts for Development, QA and Production environments
Utilizes a thorough understanding of available technology, tools, and existing designs.
Leverage knowledge of industry trends to build best in class technology to provide a competitive advantage.
Provide QA support & do Testing as needed.

Requirements

ETL (Extract, Transform, Load) Programming experience
Experience in Hadoop ecosystem tools for real-time batch data ingestion, processing, and provisioning such as Apache Flume, Apache Sqoop, Apache Spark
Java/Scala/Python experience
Agile experience
Good SQL knowledge
Design and development experience with columnar databases using Parquet or ORC file formats on Hadoop
Apache Spark design and development experience using Scala, Java, Python or Data Frames with Resilient Distributed Datasets (RDDs)

Desired Qualifications

Excellent verbal, written, and interpersonal communication skills
Ability to work effectively in a virtual environment where key team members and partners are in various time zones and locations
Knowledge and understanding of project management methodologies: used in waterfall or Agile development projects
Knowledge and understanding of DevOps principles
Reporting experience, analytics experience or a combination of both
A BS/BA degree or higher in information technology