PySpark/Spark Data Pipeline

Computer Enterprises, Inc.

Pittsburgh, PA, United States

3 months ago

Apply on indeed.com

Prepare application

Role details

Contract type

Permanent contract

Employment type

Full-time (> 32 hours)

Working hours

Regular working hours

Job source

indeed.com

Tech stack

Artificial Intelligence Business Logic Data Validation Dataspaces Software Debugging Linux Apache Hadoop Python (Programming Language) Object-Oriented Software Development Performance Tuning SQL Databases Apache Spark

+5 more

Git Pandas Pyspark Data Pipelines Jenkins

Job description

Participate on daily scrum calls for upcoming change release implementations - assigned work, status, questions, collaboration. Coding, testing, advancements, and modernization as a software developer. Key Responsibilities

Participate on daily scrum calls for upcoming change release implementations - assigned work, status, questions, collaboration.
Coding, testing, advancements, and modernization as a software developer

Requirements

Do you have experience in System performance optimization?, * Strong Spark + SQL transformation skills

Rigorous data validation & reconciliation mindset
Ability to translate STM/business logic into working pipelines
Experience operating in structured, governed data ecosystems
Practical debugging and performance optimization skills
Python, SQL, Git and Linux skills are required.
experience in working with wrangling data and building data pipelines

Preferred Skills

Preferrable experience in Jenkins, Apache Spark & Hadoop (Pandas is also good too), and object oriented programming.
Flex Skills Knowledge of AI space, acceleration enhancements and improvements.
Experience with Co-Pilot or AI code enablement functionality.