Lead Data Engineer (contract)
- Discuss this with your agent
- Open in Claude
- Open in ChatGPT
Role details
Tech stack
+21 more
Job description
- Production operations experience: monitoring, SLAs, incident response, root cause analysis, and performance optimization.
Requirements
In this contingent resource assignment, you may: Consult on or participate in moderately complex initiatives and deliverables within Software Engineering and contribute to large-scale planning related to Software Engineering deliverables. Review and analyze moderately complex Software Engineering challenges that require an in-depth evaluation of variable factors. Contribute to the resolution of moderately complex issues and consult with others to meet Software Engineering deliverables while leveraging solid understanding of the function, policies, procedures, and compliance requirements. Collaborate with client personnel in Software Engineering. Required Qualifications: Software Engineering experience, or equivalent demonstrated through one or a combination of the following: work or consulting experience, training, military experience, education., * Experience working in hybrid environments (on-prem + cloud) and supporting data migration/modernization initiatives.
- Experience with scheduling/orchestration in Autosys and Airflow-based orchestration (Cloud Composer direction).
- Experience with Git-based workflows, code reviews, and automated testing practices for data pipelines.
- Experience with Harness, Jenkins and uDeploy based CICD environments.
- Practical experience using AI-assisted coding tools in daily development to improve productivity without compromising quality or security.
- Ab Initio development/maintenance experience and/or hands-on migration of Ab Initio graphs to modern Spark/SQL patterns.
- Experience with Dataplex and broader data governance concepts (metadata, classification, stewardship, lineage practices).
- Experience with Informatica Data Quality implementation patterns (profiling, rules, scorecards/metrics, exception workflows).
- Experience designing near real-time patterns (micro-batch/event-driven concepts) and handling late-arriving/out-of-order data.
- Familiarity with GCP operational practices for data workloads (service accounts/IAM basics, job monitoring, quota/cost controls).
Key Requirements:
- Applicants must be authorized to work for ANY employer in the U.S. This position is not eligible for visa sponsorship.
- Data Engineering experience, or equivalent demonstrated through one or a combination of the following: work experience, training, military experience, education
- PL/SQL and SQL skills with proven experience in Oracle, Teradata, Python and/or BigQuery: complex query development, tuning, and debugging.
- Ab Initio skills with proven experience to build complex graphs, Psets, performance tuning.
- UNIX Shell Script
- programming skills in Python; hands-on PySpark for distributed data processing
- ETL/ETL design, data warehousing concepts, and data modeling best practices
Benefits & conditions
Pulled from the full job description
- 401(k)
- Health insurance
- Life insurance
Apply for this position
This job is hosted externally. Click below to view the full posting and apply.
Apply on www.indeed.comGood distractions
Talks and stories from around this role — technically off-topic, practically not.
Moments
Explore playlistsVideos
See allRelated articles
See all
Making Data Warehouses Fast: A Developer’s Story
Data Engineer Salary UK
How to Become an AI Engineer
Fully Remote Software Engineer Jobs