> Markdown version of [/jobs/ext/1487363-lead-data-engineer](https://www.wearedevelopers.com/jobs/ext/1487363-lead-data-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Lead Data Engineer

- **Company:** Insight Global
- **Location:** Upper Dublin Township, PA, United States
- **Experience:** Expert
- **Salary:** $135,000.0 - $145,000.0
- **Contract:** Permanent contract
- **Skills:** Microsoft Azure, Big Data, Continuous Integration, Data Architecture, Information Engineering, Python (Programming Language), Enterprise Data Management, Microsoft Fabric, Pyspark, Information Technology, Epic Caboodle, Data Pipelines
- **Published:** July 29, 2026
- **Apply:** https://jobs.insightglobal.com/jobs/find_a_job/pennsylvania/upper-dublin/lead-data-engineer/job-557493/

## About the Role

Must-Haves:

 * 15+ years of experience in Data Engineering and a combination of technical leadership and hands-on engineering experience
 * Bachelor's degree in Computer Science, Information Technology, or related field
 * Some hands-on experience with Microsoft Fabric
 * Strong Python experience, specifically PySpark for Fabric Lakehouse development
 * Experience supporting and maintaining CI/CD pipelines
 * Hands-on experience with Azure technologies, Azure DevOps, and enterprise data platforms
 * Experience designing, building, and supporting large-scale data infrastructure and data pipelines

Nice to Have Skills & Experience

Plusses:

 * Healthcare or payer or academia industry experience
 * Epic experience or certifications (Clarity, Caboodle, Cogito)

## Description

Day-to-Day:

 * Serve as a senior technical lead within the Data Engineering organization
 * Oversee data intake processes and determine how new data sources are integrated into the enterprise environment
 * Design, build, and maintain Microsoft Fabric Lakehouse solutions
 * Develop and support PySpark-based data engineering solutions
 * Lead operational support and ongoing optimization of enterprise data platforms
 * Build and maintain CI/CD pipelines and deployment processes
 * Partner with business and technical teams to develop scalable data architectures
 * Support the full data lifecycle from ingestion through consumption by BI teams and external data consumers
 * Manage multiple concurrent initiatives while ensuring platform performance and reliability
 * Provide technical guidance and leadership across the engineering organization
 * Act as a hands-on contributor while helping drive team success and operational excellence, Insight Global is seeking a Lead Data Engineer for a leading healthcare client in the Philadelphia area. This individual will serve as a lead member of the Data Engineering organization responsible for designing, building, and supporting enterprise data platforms within Microsoft Fabric and Azure environments. They will be one of three leads on the team and will balance leadership and hands-on development. The ideal candidate brings experience with PySpark, Fabric Lakehouse development, CI/CD pipeline support, and modern cloud-based data architectures. This role goes beyond traditional development work, it requires someone who can take ownership of data initiatives from intake through delivery, building the underlying infrastructure that enables data to move through its lifecycle and ultimately be consumed by BI teams, external partners, and enterprise stakeholders. The team is seeking a highly proactive, hands-on leader who can independently drive projects forward, support operations,
mentor team members, and serve as a trusted technical resource. Candidates must possess strong communication skills, leadership potential, and the ability to step in immediately to support team operations.


## Related Videos

- [PySpark - Combining Machine Learning & Big Data](https://www.wearedevelopers.com/videos/44-pyspark-combining-machine-learning-big-data)
- [Alibaba Big Data and Machine Learning Technology](https://www.wearedevelopers.com/videos/37-alibaba-big-data-and-machine-learning-technology)
- [From Messy Queries to Scalable Systems - How Data Engineering actually works](https://www.wearedevelopers.com/videos/100203-from-messy-queries-to-scalable-systems-how-data-engineering-actually-works)
- [Empowering Retail Through Applied Machine Learning](https://www.wearedevelopers.com/videos/976-empowering-retail-through-applied-machine-learning)
- [Data Science in Retail](https://www.wearedevelopers.com/videos/586-data-science-in-retail)
- [Modern Data Architectures need Software Engineering](https://www.wearedevelopers.com/videos/1030-modern-data-architectures-need-software-engineering)

## Related Articles

- [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk)
- [What Are The Top Skills Required For Azure Developers?](https://www.wearedevelopers.com/magazine/77-what-are-the-top-skills-required-for-azure-developers)
- [Top Big Data Technologies That You Need to Know](https://www.wearedevelopers.com/magazine/108-top-big-data-technologies-that-you-need-to-know)
- [The Most Popular IT Jobs on the Market](https://www.wearedevelopers.com/magazine/376-the-most-popular-it-jobs-on-the-market)
- [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers)
- [Making Data Warehouses Fast: A Developer’s Story](https://www.wearedevelopers.com/magazine/107-making-data-warehouses-fast-a-developer-s-story)