AWS Data Engineer

IBA InfoTech Inc.

Raleigh, NC, United States

about 2 months ago

Role details

Contract type

Permanent contract

Employment type

Full-time (> 32 hours)

Experience level

Experienced

Experience required

3 years minimum

Working hours

Regular working hours

Job source

ibainfotech.com

Tech stack

Agile Methodology Amazon Web Services Amazon Elastic Compute Cloud Amazon S3 Continuous Integration Extract Transform Load (ETL) Data Structures DevOps Apache Hadoop MapReduce Apache HBase Apache Hive

+18 more

Python (Programming Language) NoSQL Software Engineering Unstructured Data Parquet Data Processing Apache Yarn Apache Spark AWS Lambda Git Amazon Relational Database Service Information Technology Atlassian Tools AWS Glue AWS Data Analytics Functional Programming Data Pipelines Jenkins

Job description

Create data pipelines for a state of the art analytics application in the finacial industry
Code data processing jobs in AWS Glue, Python, and Spark.
Work with stakeholders to identify and document requirements.
Configure and schedule data pipelines
Translate business requirements to technical specifications and coded data pipelines.
Troubleshoot data pipelines

Requirements

Passionate coders with 3-5 years of application development experience.
Proficiency with Spark with Python is a must. Must have worked on projects that have resulted in code being deployed to production
Experience with AWS Glue strongly desired. However, will consider strong Python and AWS Lambda candidates.
Expert knowledge of developing in AWS environments.
Knowledge of data formats and ETL and ELT processes in a Hadoop environment including Hive, Parquet, MapReduce, YARN, HBase and other NoSQL databases.
Experience in dealing with structured, semi-structured and unstructured data in batch and real-time environments.
Experience with working in AWS environments including EC2, S3, Lambda, RDS, etc. Familiarity with DevOps and CI/CD as well as Agile tools and processes including Git, Jenkins, Jira and Confluence.
Client facing or consulting experience highly preferred.
Skilled problem solvers with the desire and proven ability to create innovative solutions.
Flexible and adaptable attitude, disciplined to manage multiple responsibilities and adjust to varied environments.
Future technology leaders- dynamic individuals energized by fast paced personal and professional growth.
Phenomenal communicators who can explain and present concepts to technical and non-technical audiences alike, including high level decision makers.
Bachelor’s Degree in MIS, Computer Science, Math, Engineering or comparable major.
Solid foundation in Computer Science, with strong competencies in data structures, algorithms and software design.
Knowledge and experience in developing software using agile methodologies.
Proficient in authoring, editing and presenting technical documents.