ETL Lead - DataStage CP4D | AWS Glue

We are seeking a highly experienced ETL Lead to design, lead, and optimize enterprise data integration workflows using IBM DataStage on Cloud Pak for Data (CP4D), AWS Glue & Lambda, and Snowflake. The ideal candidate will drive modern data transformation strategies for large-scale data ingestion pipelines supporting analytics, governance, and AI/ML workloads., * Lead the end-to-end design and implementation of ETL workflows for structured and semi-structured data from various sources (SFTP, DB2, Oracle, APIs).

• Architect and maintain data ingestion and transformation pipelines using IBM DataStage (CP4D) and AWS Glue/Lambda functions.
• Optimize data load performance and manage large data volumes with effective partitioning, incremental loads, and error handling.
• Collaborate with cloud engineers and data modelers to ensure data is curated for consumption in Snowflake (Silver/Gold/Platinum layers).
• Ensure data lineage, metadata management, and data quality in compliance with enterprise data governance standards.
• Partner with governance teams to ensure integration with Collibra, or other metadata and privacy tools.
• Support CI/CD integration, parameterization, and version control of ETL code via Git and DevOps pipelines.
• Lead and mentor a team of onshore/offshore ETL developers; establish best practices and code review processes.
• Troubleshoot production ETL issues and participate in on-call rotations as needed.

• 8+ years of enterprise data integration experience with at least 3+ years as a technical lead.
• Strong expertise in IBM DataStage, including modern deployments on Cloud Pak for Data (CP4D).
• Solid experience with AWS Glue (PySpark) and AWS Lambda for scalable, serverless data transformation.
• Proven knowledge of Snowflake data warehousing, including external stages, streams, tasks, and performance tuning.
• Experience with complex ETL orchestration involving parallel processing, dynamic parameterization, and error handling.
• Strong SQL, Python (preferred for Glue), and performance tuning skills.
• Understanding of data lakehouse architectures, S3, and ingestion patterns (batch/real-time).
• Familiarity with data governance and lineage tools (e.g., Collibra, BigID, Manta).
• Knowledge of CI/CD, Git, Jenkins, and code promotion strategies.
• Strong verbal and written communication skills; experience working in agile/scrum delivery model.

Preferred Qualifications

• IBM DataStage CP4D certification
• AWS Certified Data Analytics or Solutions Architect - Associate
• Snowflake SnowPro certification
• Experience working in regulated industries (e.g., financial services, utilities, healthcare)