PySpark/Spark Data Pipeline

Computer Enterprises, Inc.
Pittsburgh, United States of America
29 days ago

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Working hours
Regular working hours
Languages
English

Job location

Pittsburgh, United States of America

Tech stack

Artificial Intelligence
Business Logic
Data Validation
Dataspaces
Software Debugging
Linux
Hadoop
Python
Object-Oriented Software Development
Performance Tuning
SQL Databases
Spark
GIT
Pandas
PySpark
Data Pipelines
Jenkins

Job description

Participate on daily scrum calls for upcoming change release implementations - assigned work, status, questions, collaboration. Coding, testing, advancements, and modernization as a software developer. Key Responsibilities

  • Participate on daily scrum calls for upcoming change release implementations - assigned work, status, questions, collaboration.
  • Coding, testing, advancements, and modernization as a software developer

Requirements

Do you have experience in System performance optimization?, * Strong Spark + SQL transformation skills

  • Rigorous data validation & reconciliation mindset
  • Ability to translate STM/business logic into working pipelines
  • Experience operating in structured, governed data ecosystems
  • Practical debugging and performance optimization skills
  • Python, SQL, Git and Linux skills are required.
  • experience in working with wrangling data and building data pipelines

Preferred Skills

  • Preferrable experience in Jenkins, Apache Spark & Hadoop (Pandas is also good too), and object oriented programming.
  • Flex Skills Knowledge of AI space, acceleration enhancements and improvements.
  • Experience with Co-Pilot or AI code enablement functionality.

#INDGEN #ZR

Apply for this position