ETL Developer

We’re looking for an experienced ETL Developer to build and optimize high-throughput, resilient data pipelines that move and transform data from on-prem sources to cloud destinations. You’ll design, develop, and support Ab Initio graphs and plans, integrate with AWS and/or Azure services, and partner with platform, security, and analytics teams to deliver governed, production-grade data at scale.

What You’ll Do

• Design & Develop: Build ETL processes using Ab Initio graphs/plans (GDE, Co>Operating System, EME, and Conduct>It) to ingest, transform, and publish data.
• Migrate Pipelines: Move data workflows from on-prem to cloud targets (batch and near-real-time), ensuring restartability, parameterization (PDL), metadata management, and resiliency.
• Cloud Integration: Integrate with AWS (e.g., S3, Redshift, Glue/Glue Catalog, Lambda, EMR, MSK/Kinesis) and/or Azure (e.g., ADLS Gen2, Synapse, Data Factory, Event Hubs).
• Source Systems & CDC: Build connectors/jobs for Oracle, SQL Server, DB2, files, MQ/Kafka; implement incremental loads/CDC patterns where applicable.
• Performance & Reliability: Tune memory/parallelism/partitioning, optimize file and database I/O, and implement robust error handling, alerting, and SLA monitoring.
• Security & Governance: Apply data quality checks (e.g., Ab Initio DQE), lineage/metadata practices, and enterprise security controls (IAM, KMS/Key Vault, tokenization).
• DevOps & CI/CD: Utilize version control (Git/EME), automated build/deploy, environment promotion, and infrastructure coordination with platform teams.
• Operations: Troubleshoot production issues, perform capacity planning, and deliver clear runbooks.
• Collaboration: Partner with data architects, platform/cloud engineers, and analysts; contribute to standards and best practices.

• 5+ years of hands-on ETL development using Ab Initio (GDE, EME, Co>Operating System; Conduct>It scheduling).
• Proven Delivery: Experience with high-volume batch and near-real-time pipelines, including restart/recovery, parameterization, and metadata-driven design.
• Database Expertise: Strong SQL and performance tuning across major RDBMS (Oracle/SQL Server/DB2; plus Redshift/Synapse/Snowflake a plus).
• Cloud Experience: Production experience integrating Ab Initio with AWS and/or Azure for data landing, processing, and analytics.
• Technical Foundations: Solid Linux/Unix fundamentals and scripting (bash, Python preferred).
• Messaging & Networking: Experience with Kafka/MQ (publish/subscribe), file transfer patterns, and secure networking (VPC/VNet, PrivateLink/Private Endpoints).
• Data Integrity: Familiarity with data quality, lineage, and compliance for sensitive data (e.g., PHI/PII).
• Problem Solving: Excellent troubleshooting skills and the ability to own solutions end-to-end.

Preferred / “Nice to Have”

• Cloud Certifications: AWS Data Analytics / Solutions Architect, or Azure Data Engineer Associate.
• Modern Data Stack: Experience with Glue/Spark or Synapse Spark for complementary processing; EMR/Databricks exposure a plus.
• Orchestration: Experience with Airflow/ADF alongside Conduct>It; event-driven designs with Lambda/Functions.
• Infrastructure as Code (IaC): Awareness of Terraform, CloudFormation, or Bicep to collaborate with platform teams.
• Storage Patterns: Experience with Snowflake or Delta Lake patterns on S3/ADLS.
• Observability: Proficiency with CloudWatch, Azure Monitor, Prometheus/Grafana, or Splunk.
• Methodology: Agile/SAFe delivery in regulated environments (healthcare/financial services)., * Ab Initio: 5 years (Required)

$55 - $60 an hour - Contract