> Markdown version of [/jobs/ext/2919347-lead-data-engineer](https://www.wearedevelopers.com/jobs/ext/2919347-lead-data-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Lead Data Engineer - **Company:** PERI - OPERATIVE STAFFING SOLUTIONS, LLC - **Location:** Charlotte, NC, United States - **Experience:** Expert - **Salary:** $187,200.0 - $208,000.0 - **Contract:** Temporary contract - **Skills:** Artificial Intelligence, Airflow, Amazon Web Services, Amazon S3, Microsoft Azure, Big Data, Cloud Computing Security, Cloud Engineering, Code Review, Continuous Delivery, Continuous Integration, Data Architecture, Information Engineering, Data Governance, Extract Transform Load (ETL), Data Systems, Data Vault Modeling, Data Warehousing, Dimensional Modeling, Distributed Computing Environment, Github, Identity and Access Management, Python (Programming Language), Meta-Data Management, Performance Tuning, Query Optimization, Amazon Simple Notification Service (SNS), Workflow Management Systems, Enterprise Data Management, Feature Engineering, Data Ingestion, Large Language Models, State Machines, Generative AI, Containerization, Data Lakes, Pyspark, Apache Kafka, Data Management, Machine Learning Operations, Cloud Migration, Cloudwatch, Amazon Simple Queue Service (SQS), Terraform, Data Pipelines, Amazon Elastic Mapreduce (EMR), Confluent, Amazon Redshift, Databricks - **Published:** September 15, 2026 - **Apply:** https://www.dice.com/job-detail/eb8a9501-fd02-4307-8481-9ddaaa348df6 ## About the Role * 8+ years of Data Engineering experience, including at least 5+ years of extensive experience working within AWS environments. * Expert-level experience with AWS services including S3, EMR, Glue Jobs, Lambda, Athena, CloudTrail, SNS, SQS, CloudWatch, and Step Functions. * Strong hands-on experience designing and implementing enterprise-scale data lakes and data warehouses using AWS Lake Formation, Amazon Redshift, and Amazon Athena. * Advanced Python development experience with extensive hands-on use of PySpark. * Advanced SQL expertise, including query optimization, large-scale data processing, and enterprise data warehousing. * Strong data modeling experience, including dimensional modeling, Data Vault, and other enterprise data modeling techniques. * Deep experience designing, developing, and optimizing scalable and resilient data pipelines within AWS environments. * Strong experience with distributed data processing frameworks, particularly PySpark and AWS EMR. * Extensive knowledge of database management, performance tuning, and data architecture best practices. * Experience with Kafka-based streaming architectures, preferably Confluent Kafka. * Expertise with Infrastructure as Code using Terraform. * Experience designing and implementing CI/CD frameworks using GitHub and GitHub Actions. * Deep knowledge of AWS IAM roles, policies, governance, security controls, and cloud security best practices. * Strong experience with workflow orchestration platforms such as AWS Step Functions, Apache Airflow, or equivalent technologies. * Experience leading cloud migration, modernization, or enterprise data platform initiatives. * Strong understanding of data governance, metadata management, data quality frameworks, observability, resiliency, and operational supportability. * Experience creating AI applications using AWS Bedrock. * Experience building AI-ready data pipelines and ML workflows, including feature engineering and MLOps. * Knowledge of Generative AI technologies, LLMs, Retrieval-Augmented Generation (RAG), and vector databases. * Experience working with cloud-based AI and data platforms such as AWS, Azure, or Databricks. * Ability to lead hands-on development efforts while providing technical direction, engineering oversight, and code reviews across multiple initiatives. * Experience establishing cloud development environments, infrastructure standards, security controls, and migration strategies across multiple accounts and environments. * Proven ability to identify data gaps, develop strategic remediation plans, and implement scalable automation solutions. * Experience designing highly reliable data pipelines with a strong emphasis on data quality, observability, resiliency, and operational support CONTRACTOR, FULL_TIME ## Description * Lead the design, architecture, and implementation of enterprise-scale data engineering solutions across AWS cloud environments. * Design, develop, and optimize scalable, resilient data pipelines, ETL processes, data ingestion frameworks, and orchestration workflows. * Architect and oversee enterprise data lake and data warehouse solutions using AWS Lake Formation, Amazon Redshift, Amazon Athena, S3, and related AWS technologies. * Collaborate with Lead Developers, Data Scientists, Architects, Product Owners, and business stakeholders to define technical strategy and scalable data solutions. * Provide hands-on technical leadership, engineering oversight, code reviews, and mentorship to Data Engineers and development teams. * Drive architectural decisions in partnership with Data and Solution Architects to ensure scalability, security, reliability, performance, and maintainability. * Design and support Kafka-based streaming and event-driven data architectures, preferably using Confluent Kafka. * Develop distributed data processing solutions using Python, PySpark, AWS EMR, and other cloud-native technologies. * Establish engineering standards and best practices for data modeling, ETL frameworks, pipeline reliability, monitoring, observability, and operational excellence. * Lead end-to-end solution delivery while ensuring alignment with business requirements, enterprise architecture standards, security controls, and regulatory requirements. * Design and implement Infrastructure as Code solutions using Terraform across multiple AWS accounts and environments. * Develop and maintain CI/CD frameworks using GitHub and GitHub Actions. * Oversee production support and operational management of AWS-based data platforms, including root-cause analysis, troubleshooting, and performance optimization. * Champion data governance, metadata management, data quality, observability, and data stewardship practices across platforms and teams. * Identify opportunities to modernize data architecture and improve operational efficiency through automation and cloud-native technologies. * Support the development of AI-ready data pipelines and machine learning workflows, including feature engineering and MLOps. * Design data solutions capable of supporting Generative AI applications, intelligent search, RAG architectures, LLMs, and vector databases. * Drive technical decision-making and clearly communicate complex data architecture concepts to both technical and non-technical stakeholders. ## Related Videos - [From Messy Queries to Scalable Systems - How Data Engineering actually works](https://www.wearedevelopers.com/videos/100203-from-messy-queries-to-scalable-systems-how-data-engineering-actually-works) - [Alibaba Big Data and Machine Learning Technology](https://www.wearedevelopers.com/videos/37-alibaba-big-data-and-machine-learning-technology) - [PySpark - Combining Machine Learning & Big Data](https://www.wearedevelopers.com/videos/44-pyspark-combining-machine-learning-big-data) - [Innovating Developer Tools with AI: Insights from GitHub Next](https://www.wearedevelopers.com/videos/1268-innovating-developer-tools-with-ai-insights-from-github-next) - [Data Science in Retail](https://www.wearedevelopers.com/videos/586-data-science-in-retail) - [Empowering Retail Through Applied Machine Learning](https://www.wearedevelopers.com/videos/976-empowering-retail-through-applied-machine-learning) ## Related Articles - [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk) - [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers) - [Making Data Warehouses Fast: A Developer’s Story](https://www.wearedevelopers.com/magazine/107-making-data-warehouses-fast-a-developer-s-story) - [How to Become an AI Engineer](https://www.wearedevelopers.com/magazine/331-how-to-become-an-ai-engineer) - [Coffee with Developers - Maria Apazoglou - Making AI understandable for all in production](https://www.wearedevelopers.com/magazine/475-coffee-with-developers-maria-apazoglou-making-ai-understandable-for-all-in-production) - [Got AI ideas but no money? Here are 10 free ways to level up your AI skills with Google Cloud](https://www.wearedevelopers.com/magazine/600-got-ai-ideas-but-no-money-here-are-10-free-ways-to-level-up-your-ai-skills-with-google-cloud)