Senior ETL Developer - Assistant Vice President

Senior ETL Developer will be responsible for designing, implementing, and optimizing distributed data processing jobs to handle large-scale data in Hadoop Distributed File System(HDFS) using Apache Spark and Python. This role required deep understanding of data engineering principles, proficiency in Python and hands-on experience with Spark and Hadoop ecosystems. Developer will collaborate with data engineers, analysts, and business stakeholders to process, transform and drive insights and data driven decisions., Data Processing and Transformation:

• Design and Implement of Spark applications to process and transform large datasets in HDFS.
• Develop ETL Pipelines in Spark using Python for data Ingestion, cleaning, aggregation, and transformations.

Performance Optimization:

• Optimize Spark jobs for efficiency, reducing run time and resource usage.
• Finetune memory management, caching, and partitioning strategies for Optimal performance *

Data Engineering with Hadoop and Spark:

• Load data from different sources into HDFS, ensuring data accuracy and integrity.
• Integrate Spark Applications with Hadoop frameworks like Hive, Sqoop etc.
• Testing and debugging:
• Troubleshoot and debug Spark Job failures, monitor job logs, and Spark UI to Identify Issues.

• 5-8 years of relevant experience in Software Development
• Experience in systems analysis and programming of software applications
• Experience in managing and implementing successful projects
• Working knowledge of consulting/project management techniques/methods
• Ability to work under pressure and manage deadlines or unexpected changes in expectations or requirements

Education:

• Bachelor’s degree/University degree or equivalent experience

Primary Location Full Time Salary Range:$96,960.00 - $145,440.00

Working at Citi is far more than just a job. A career with us means joining a team of approximately 219,000 dedicated people from around the globe. At Citi, you’ll have the opportunity to grow your career, give back to your community and make a real impact.