Sr. ETL Developer

Insight Global is seeking a Sr. ETL/Pipeline Engineer to support a large-scale initiative focused on modernizing and scaling personalized retail experiences across our client’s digital and in-store platforms This individual will be responsible for designing, building, migrating, validating, and supporting enterprise data pipelines that enable scalable data movement, transformation, enrichment, and downstream consumption across cloud and distributed processing environments. Day-to-day responsibilities include translating source-to-target mappings, business rules, schemas, and data quality requirements into reliable pipeline solutions; developing and optimizing ETL workflows using Python, PySpark, Spark SQL, and SQL; supporting ingestion from files, cloud storage, APIs, databases, data lake sources, and warehouse platforms; executing functional, integration, reconciliation, regression, performance, and data quality testing; and troubleshooting job failures, defects, orchestration dependencies, logging, alerting, retries, and production support readiness. This person will partner closely with data engineers, platform teams, architects, QA, DevOps, product owners, analysts, and downstream consumers to manage delivery milestones, resolve blockers, document pipeline behavior, and ensure workflows are stable, scalable, and ready for production handoff. The ideal candidate will have a strong data engineering background, experience supporting complex enterprise pipeline environments, and the ability to remain hands-on while driving ETL development, testing, validation, optimization, release readiness, and operational support.

We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment regardless of their race, color, ethnicity, religion, sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military or uniformed service member status, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or recruiting process, please send a request to HR@insightglobal.com.To learn more about how we collect, keep, and process your private information, please review Insight Global’s Workforce Privacy Policy: https://insightglobal.com/workforce-privacy-policy/.

• Strong ETL and data pipeline development experience using Python, PySpark, Spark SQL, SQL, shell scripting, and distributed data processing frameworks.
• Practical exposure to batch processing, real-time data flows, file-based ingestion, cloud integration as a source or destination, data warehouse integration, source-to-target mapping, transformation logic, schema validation, and data reconciliation.
• Hands-on capability with pipeline orchestration, job scheduling, dependency management, retries, error handling, logging, monitoring, alerting, and operational troubleshooting.
• Working knowledge of data quality validation, contract testing, functional testing, integration testing, regression testing, performance testing, and defect remediation for data pipelines.
• Proficiency with agile delivery tools, code repositories, pull requests, peer reviews, CI/CD concepts, deployment readiness, documentation, and stakeholder communication in an enterprise data engineering environment. - Preferred exposure to GCP integration, especially BigQuery, along with cloud data platforms such as AWS or Azure, including cloud storage, data warehouse services, managed compute, IAM, environment configuration, deployment dependencies, and production support considerations.
• Practical exposure to cloud storage, data warehouse platforms, distributed processing services, orchestration tools, Kafka, NoSQL platforms, data lake patterns, or equivalent enterprise data ecosystem components.
• Preferred experience implementing ETL batch jobs or real-time data flows with BigQuery as a source, transformation layer, analytical store, or destination within broader cloud data integration patterns.
• Experience migrating legacy ETL jobs, refactoring existing pipelines, modernizing batch workflows, tuning Spark jobs, reducing runtime failures, and improving operational reliability.
• Familiarity with data governance, lineage, metadata capture, schema evolution, data retention, privacy controls, access management, and audit-ready pipeline documentation.
• Exposure to automated testing frameworks, test data management, contract validation, data observability, quality dashboards, centralized logging, and incident response processes.
• Ability to operate in milestone-based, outcome-based, or acceptance-driven delivery models with clear ownership of deliverables, dependencies, risks, and closure criteria.
• Relevant certifications or training in data engineering, cloud platforms, Spark, DevOps, agile delivery, quality engineering, or enterprise application development.