> Markdown version of [/jobs/ext/495612-lead-data-engineer](https://www.wearedevelopers.com/jobs/ext/495612-lead-data-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Lead Data Engineer - **Company:** TalentOla View all jobs - **Location:** Charlotte, NC, United States - **Experience:** Expert - **Salary:** $63,000.0 - $134,500.0 - **Contract:** Permanent contract - **Skills:** Amazon Web Services, Amazon S3, Data Analysis, Apache HTTP Server, Unit Testing, Big Data, Software as a Service, Data Validation, Data Cleansing, Information Engineering, Data Infrastructure, Data Integrity, Extract Transform Load (ETL), IBM DB2, Relational Databases, Identity and Access Management, Data Intelligence, JSON, Python (Programming Language), PostgreSQL, MySQL, DataOps, TypeScript, Parquet, Snowflake, Generative AI, Data Lakes, Pyspark, Information Technology, AWS Glue, Data Analytics, AWS Data Analytics, Data Management, Data Pipelines - **Published:** June 8, 2026 - **Apply:** https://www.careerjet.com/jobad/us3d1f6b04dad25a1af752f5e882b8d183 ## About the Role * Somebody who has at least 12+ years of data engineering experience has played Lead Data Engineer role. * Bachelor's degree (or equivalent) in computer science, information technology, engineering, or related discipline * Education qualification: Any degree from a reputed college ## Description We're seeking an experienced pipeline-centric data engineer to put it to good use in building out ETL and Data Operations framework (Data Preparation / Normalization and Ontological processes). Technical Skills: * Lead the design, development, and maintenance of scalable data pipelines and ETL processes, ensuring data integrity and accessibility for business intelligence and advanced analytics. * Architect and manage robust data platforms on the AWS ecosystem, leveraging services like Glue, PySpark, Apache Iceberg, IAM, S3, and Secrets Manager to build a secure and efficient data infrastructure. * Provide technical guidance and mentorship to a team of data engineers, fostering a culture of high performance and continuous learning. * Utilize deep expertise in various RDBMS (e.g., MySQL, Db2, PostgreSQL, Snowflake) and different data formats (e.g., JSON, Parquet) to drive strategic data initiatives. * Champion the adoption of modern data technologies such as Apache Iceberg with AWS Glue to optimize data lake performance and analytics capabilities. * Apply advanced proficiency in Generative AI to automate and streamline data analysis, development, and documentation processes. * Business Acumen and Stakeholder Communication * Translate complex business challenges into clear, data-driven solutions, effectively communicating technical concepts and project progress to both technical and non-technical stakeholders, including senior leadership. * Act as a key liaison between the data engineering team and business units, providing data-backed insights and recommendations that directly influence business strategy. * Manage concurrent projects in a dynamic, research-oriented environment, ensuring timely delivery and high-quality outcomes. * Experience in insurance domain preferrable * AWS Data Engineering certification good to have, * Collaborate with business analysts and stakeholders to translate business needs into comprehensive source-to-target (S2T) data mappings. * Lead the design and development of robust data pipelines, using your deep understanding of existing ETL frameworks to build scalable and efficient solutions. * Analyze and understand diverse source systems and data formats to create and validate synthetic datasets for testing and development. * Develop automated data validation tools using Python to ensure data integrity across various ETL layers. * Ensure Industry best practice is followed in all aspects of project. * Develop robust development testing approach to minimize the quality, integration and user testing issues. * Leverage generative AI to enhance data analysis, accelerate development, Unit Testing and streamline documentation. * Apply statistical analysis to large datasets, uncovering key patterns, identifying potential challenges, and generating actionable insights. * Partner with business leaders to understand their challenges and provide data-driven recommendations that improve processes and inform strategic decisions., A Senior Full-Stack Engineer will lead the architecture and development of a multi-tenant SaaS platform designed for regulated financial and government procurement workflows. This … + 4 days ago + Apply easily, Role Overview: As a Senior Full-Stack Engineer specializing in TypeScript, you will play a pivotal role in designing and developing robust, scalable web applications. This positio… + 7 days ago + Apply easily ## Related Videos - [PySpark - Combining Machine Learning & Big Data](https://www.wearedevelopers.com/videos/44-pyspark-combining-machine-learning-big-data) - [From Messy Queries to Scalable Systems - How Data Engineering actually works](https://www.wearedevelopers.com/videos/100203-from-messy-queries-to-scalable-systems-how-data-engineering-actually-works) - [MySQL Protocol Features You Should Be Aware Of](https://www.wearedevelopers.com/videos/100267-mysql-protocol-features-you-should-be-aware-of) - [Tips and Tricks for Working with JSON](https://www.wearedevelopers.com/videos/1229-tips-and-tricks-for-working-with-json) - [Empowering Retail Through Applied Machine Learning](https://www.wearedevelopers.com/videos/976-empowering-retail-through-applied-machine-learning) - [Data Science in Retail](https://www.wearedevelopers.com/videos/586-data-science-in-retail) ## Related Articles - [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk) - [Top Big Data Technologies That You Need to Know](https://www.wearedevelopers.com/magazine/108-top-big-data-technologies-that-you-need-to-know) - [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers) - [How to Become an AI Engineer](https://www.wearedevelopers.com/magazine/331-how-to-become-an-ai-engineer) - [Data Analyst Salary in the UK](https://www.wearedevelopers.com/magazine/278-data-analyst-salary-in-the-uk) - [The Most Popular IT Jobs on the Market](https://www.wearedevelopers.com/magazine/376-the-most-popular-it-jobs-on-the-market)