Senior Data Engineer

Qode LLC

Pittsburgh, PA, United States

2 months ago

Role details

Contract type

Permanent contract

Employment type

Full-time (> 32 hours)

Experience level

Expert

Experience required

8 years minimum

Working hours

Regular working hours

Job source

indeed.com

Tech stack

Agile Methodology Artificial Intelligence Data Analysis JIRA Big Data Computer Programming Databases Data Architecture Information Engineering Extract Transform Load (ETL) Data Mining Database Theory

+16 more

Apache Hadoop Hadoop Distributed File System MapReduce Apache Hive Python (Programming Language) Machine Learning Systems Development Life Cycle Sqoop Data Streaming Informatica Powercenter Apache Spark Pyspark Information Technology AWS Data Analytics Apache Kafka Data Pipelines

Requirements

Do you have experience in Spark implementation?, Do you have a Master’s degree?, * Experience: 8-10 years in Data Engineering and Data Analysis.

Informatica Expertise: Strong hands-on experience in Informatica PowerCenter/IDQ for ETL design, development, and optimization.
PySpark Development: Advanced skills in PySpark for large-scale data processing, transformation, and analytics.
Hadoop Ecosystem: Solid working knowledge of Hadoop technologies (HDFS, Hive, Sqoop, MapReduce).
Programming Skills: Proficiency in Python and Kafka for streaming and batch data pipelines.
Database & Modeling: Strong understanding of database concepts, data design, data modeling, and ETL workflows.
ETL Lifecycle: Experience in analyzing, designing, and coding ETL programs including data extraction, ingestion, quality checks, normalization, and loading.
Agile Delivery: Hands-on experience with Agile methodology and Jira for project delivery.
Client Interaction: Proven ability in client-facing roles with strong communication and leadership skills to coordinate across SDLC., * Exposure to AWS data components and analytics.
Familiarity with machine learning models and AI concepts.
Experience with data modeling tools such as Erwin., * Master’s or Bachelor’s degree in Computer Science or related field.
Strong problem-solving skills and ability to work in cross-functional teams.