> Markdown version of [/jobs/ext/2495682-lead-data-engineer](https://www.wearedevelopers.com/jobs/ext/2495682-lead-data-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Lead Data Engineer - **Company:** Diverse Lynx LLC - **Location:** Charlotte, NC, United States - **Experience:** Expert - **Contract:** Permanent contract - **Skills:** Amazon Web Services, Amazon S3, Data Analysis, Apache HTTP Server, Unit Testing, Big Data, Data Validation, Data Cleansing, Information Engineering, Data Infrastructure, Data Integrity, Extract Transform Load (ETL), IBM DB2, Relational Databases, Identity and Access Management, Data Intelligence, JSON, Python (Programming Language), PostgreSQL, MySQL, DataOps, Parquet, Snowflake, Generative AI, Data Lakes, Pyspark, Information Technology, AWS Glue, Data Analytics, AWS Data Analytics, Data Management, Data Pipelines - **Published:** August 19, 2026 - **Apply:** https://www.dice.com/job-detail/68306b29-f285-4ccb-be43-a7281bb06e9c ## About the Role * Somebody who has at least 12+ years of data engineering experience has played Lead Data Engineer role. * Bachelor's degree (or equivalent) in computer science, information technology, engineering, or related discipline * Education qualification: Any degree from a reputed college ## Description We're seeking an experienced pipeline-centric data engineer to put it to good use in building out ETL and Data Operations framework (Data Preparation / Normalization and Ontological processes). Technical Skills: * Lead the design, development, and maintenance of scalable data pipelines and ETL processes, ensuring data integrity and accessibility for business intelligence and advanced analytics. * Architect and manage robust data platforms on the AWS ecosystem, leveraging services like Glue, PySpark, Apache Iceberg, IAM, S3, and Secrets Manager to build a secure and efficient data infrastructure. * Provide technical guidance and mentorship to a team of data engineers, fostering a culture of high performance and continuous learning. * Utilize deep expertise in various RDBMS (e.g., MySQL, Db2, PostgreSQL, Snowflake) and different data formats (e.g., JSON, Parquet) to drive strategic data initiatives. * Champion the adoption of modern data technologies such as Apache Iceberg with AWS Glue to optimize data lake performance and analytics capabilities. * Apply advanced proficiency in Generative AI to automate and streamline data analysis, development, and documentation processes. * Business Acumen and Stakeholder Communication * Translate complex business challenges into clear, data-driven solutions, effectively communicating technical concepts and project progress to both technical and non-technical stakeholders, including senior leadership. * Act as a key liaison between the data engineering team and business units, providing data-backed insights and recommendations that directly influence business strategy. * Manage concurrent projects in a dynamic, research-oriented environment, ensuring timely delivery and high-quality outcomes. * Experience in insurance domain preferrable * AWS Data Engineering certification good to have Key Responsibilities * Collaborate with business analysts and stakeholders to translate business needs into comprehensive source-to-target (S2T) data mappings. * Lead the design and development of robust data pipelines, using your deep understanding of existing ETL frameworks to build scalable and efficient solutions. * Analyze and understand diverse source systems and data formats to create and validate synthetic datasets for testing and development. * Develop automated data validation tools using Python to ensure data integrity across various ETL layers. * Ensure Industry best practice is followed in all aspects of project. * Develop robust development testing approach to minimize the quality, integration and user testing issues. * Leverage generative AI to enhance data analysis, accelerate development, Unit Testing and streamline documentation. * Apply statistical analysis to large datasets, uncovering key patterns, identifying potential challenges, and generating actionable insights. * Partner with business leaders to understand their challenges and provide data-driven recommendations that improve processes and inform strategic decisions. ## Related Videos - [Tips and Tricks for Working with JSON](https://www.wearedevelopers.com/videos/1229-tips-and-tricks-for-working-with-json) - [MySQL Protocol Features You Should Be Aware Of](https://www.wearedevelopers.com/videos/100267-mysql-protocol-features-you-should-be-aware-of) - [From Messy Queries to Scalable Systems - How Data Engineering actually works](https://www.wearedevelopers.com/videos/100203-from-messy-queries-to-scalable-systems-how-data-engineering-actually-works) - [Introducing JSON Structure](https://www.wearedevelopers.com/videos/100219-introducing-json-structure) - [Alibaba Big Data and Machine Learning Technology](https://www.wearedevelopers.com/videos/37-alibaba-big-data-and-machine-learning-technology) - [Coding for Good: Achieving social change with an app](https://www.wearedevelopers.com/videos/1645-coding-for-good-achieving-social-change-with-an-app) ## Related Articles - [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk) - [Top Big Data Technologies That You Need to Know](https://www.wearedevelopers.com/magazine/108-top-big-data-technologies-that-you-need-to-know) - [Making Data Warehouses Fast: A Developer’s Story](https://www.wearedevelopers.com/magazine/107-making-data-warehouses-fast-a-developer-s-story) - [How to Become an AI Engineer](https://www.wearedevelopers.com/magazine/331-how-to-become-an-ai-engineer) - [Data Analyst Salary in the UK](https://www.wearedevelopers.com/magazine/278-data-analyst-salary-in-the-uk) - [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers)