> Markdown version of [/jobs/ext/1994025-sr-data-engineer](https://www.wearedevelopers.com/jobs/ext/1994025-sr-data-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Sr. Data Engineer - **Company:** Kelly Services Inc. - **Location:** Charlotte, NC, United States - **Experience:** Expert - **Salary:** $111,405.0 - $125,528.0 - **Contract:** Temporary contract - **Skills:** Artificial Intelligence, Airflow, Automation of Tests, CA Workload Automation Ae, BigQuery, Profiling, Code Review, Computer Programming, Continuous Integration, Data Dictionary, Information Engineering, Data Governance, Extract Transform Load (ETL), Data Migration, Data Security, Data Warehousing, Software Debugging, Distributed Computing Environment, Middleware, Identity and Access Management, Python (Programming Language), Metadata, Meta-Data Management, Oracle (Applications), Performance Tuning, Query Optimization, Standard Sql, Secure Coding, PL-SQL, SQL Databases, Teradata SQL, Google Cloud, GitHub Copilot, Apache Spark, Ab Initio, Pyspark, Git Flow, Data Pipelines, Jenkins - **Published:** August 8, 2026 - **Apply:** https://dejobs.org/x/x/BEE0CC2EA1A049C1A40B02B1ECF83741/job/ ## About the Role * 4+ years of Data Engineering experience, or equivalent demonstrated through one or a combination of the following: work experience, training, military experience, education * 4+ years PL/SQL and SQL skills with proven experience in Oracle, Teradata, Python and/or BigQuery: complex query development, tuning, and debugging. * 4+ years Ab Initio skills with proven experience to build complex graphs, Psets, performance tuning. * 3+ years programming skills in Python; hands-on PySpark for distributed data processing * 3+ years of ETL/ETL design, data warehousing concepts, and data modeling best practices Desired Skills & Experience * Production operations experience: monitoring, SLAs, incident response, root cause analysis, and performance optimization. * Experience working in hybrid environments (on-prem + cloud) and supporting data migration/modernization initiatives. * Experience with scheduling/orchestration in Autosys and Airflow-based orchestration (Cloud Composer direction). * Experience with Git-based workflows, code reviews, and automated testing practices for data pipelines. * Experience with Harness, Jenkins and uDeploy based CICD environments. * Practical experience using AI-assisted coding tools in daily development to improve productivity without compromising quality or security. * Ab Initio development/maintenance experience and/or hands-on migration of Ab Initio graphs to modern Spark/SQL patterns. * Experience with Dataplex and broader data governance concepts (metadata, classification, stewardship, lineage practices). * Experience with Informatica Data Quality implementation patterns (profiling, rules, scorecards/metrics, exception workflows). * Experience designing near real-time patterns (micro-batch/event-driven concepts) and handling late-arriving/out-of-order data. * Familiarity with GCP operational practices for data workloads (service accounts/IAM basics, job monitoring, quota/cost controls). ## Description * Build and maintain scalable batch and near real-time data pipelines using AB Initio, Python, ?PySpark, PL / SQL and SQL to ingest, transform, and publish curated datasets across on-prem and Google Cloud platforms. * Develop and optimize BigQuery transformations and data models, including partitioning, clustering, query optimization, and cost/performance tuning. * Support modernization/migration from Teradata and Ab Initio workflows to GCP/BigQuery, including logic re-platforming, reconciliation, parallel runs, and controlled cutovers. * Implement orchestration and scheduling for pipelines using legacy Autosys while driving migration toward Google Cloud Composer (Airflow), including dependency management, retries, SLAs, and backfills. * Apply data governance and discovery practices using Dataplex: metadata management, dataset organization, classification support, and ensuring data is consumption-ready. * Build and operationalize data quality controls using Informatica Data Quality: profiling, rule implementation, thresholds, exception handling, and embedding quality gates into pipelines. * Ensure operational excellence: monitoring, alerting, runbooks, incident triage/root cause analysis, and continuous improvements to reliability and performance. * Implement secure data engineering practices: least-privilege access, PII handling/masking where required, retention controls, and audit-friendly documentation. * Partner with product, analytics, and engineering stakeholders to translate requirements into clear data contracts, curated datasets, and maintainable documentation (data dictionaries, reconciliation notes, operational runbooks). * Must-have: Use AI-assisted coding tools (e.g., GitHub Copilot, Devin, or similar) to accelerate development while maintaining strong code review discipline, testing, and secure coding standards. * Closely partner with Product Owners, Architects and Engineers on definition, design, development, integration, testing and support of reliable and reusable Data pipelines. * Analyze highly complex business requirements; generate technical specifications to design ETL processes. * Act as an expert technical resource for analysis and provides critical direction to less experienced staff. Work with team members to provide insight into solving complex problems with middleware while leveraging enterprise and industry best practices (including scalability, availability, maintainability, and flexibility). ## Related Videos - [From Messy Queries to Scalable Systems - How Data Engineering actually works](https://www.wearedevelopers.com/videos/100203-from-messy-queries-to-scalable-systems-how-data-engineering-actually-works) - [The Road to MLOps: How Verivox Transitioned to AWS](https://www.wearedevelopers.com/videos/1050-the-road-to-mlops-how-verivox-transitioned-to-aws) - [Enjoying SQL data pipelines with dbt](https://www.wearedevelopers.com/videos/823-enjoying-sql-data-pipelines-with-dbt) - [AI Model Management Life Circles: ML Ops For Generative AI Models From Research to Deployment](https://www.wearedevelopers.com/videos/1152-ai-model-management-life-circles-ml-ops-for-generative-ai-models-from-research-to-deployment) - [Our GitOps approach for deploying an Identity Provider and an API Gateway in a SaaS company](https://www.wearedevelopers.com/videos/776-our-gitops-approach-for-deploying-an-identity-provider-and-an-api-gateway-in-a-saas-company) - [Alibaba Big Data and Machine Learning Technology](https://www.wearedevelopers.com/videos/37-alibaba-big-data-and-machine-learning-technology) ## Related Articles - [Making Data Warehouses Fast: A Developer’s Story](https://www.wearedevelopers.com/magazine/107-making-data-warehouses-fast-a-developer-s-story) - [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers) - [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk) - [Coffee with Developers - Maria Apazoglou - Making AI understandable for all in production](https://www.wearedevelopers.com/magazine/475-coffee-with-developers-maria-apazoglou-making-ai-understandable-for-all-in-production) - [How to Become an AI Engineer](https://www.wearedevelopers.com/magazine/331-how-to-become-an-ai-engineer) - [Top Big Data Technologies That You Need to Know](https://www.wearedevelopers.com/magazine/108-top-big-data-technologies-that-you-need-to-know)