> Markdown version of [/jobs/ext/1994024-lead-data-engineering-dae-consultant](https://www.wearedevelopers.com/jobs/ext/1994024-lead-data-engineering-dae-consultant). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Lead Data Engineering (DAE) Consultant

- **Company:** Allstate Corporation
- **Location:** San Francisco, CA, United States (Remote available)
- **Experience:** Expert
- **Salary:** $110,000.0 - $181,025.0
- **Contract:** Permanent contract
- **Skills:** Artificial Intelligence, Data Analysis, Big Data, Computer Programming, Information Engineering, Data Infrastructure, Extract Transform Load (ETL), Data Systems, Python (Programming Language), DataOps, SQL Databases, Data Strategy, Microsoft Fabric, Data Analytics, Machine Learning Operations, Data Pipelines
- **Published:** August 8, 2026
- **Apply:** https://www.dice.com/job-detail/ad2e05be-5654-41b3-be9b-d74d496904b3

## About the Role

The ideal candidate has strong expertise in Python, SQL, and Microsoft Fabric, along with experience leveraging AI/ML tools and modern data platforms. This role requires someone who thrives in a fast-paced environment, can navigate ambiguity, and is highly detail-oriented., 5+ years of experience in Data Engineering or related field

 - Strong expertise in Python, SQL, and Microsoft Fabric
 - Experience building ETL/ELT pipelines at scale
 - Experience supporting Data Science and analytics use cases
 - Experience with AI/ML tools or platforms
 - Ability to work in a fast-paced environment
 - Ability to manage ambiguity and prioritize effectively
 - Strong attention to detail and data quality
 - Strong communication and stakeholder engagement skills, Actionable Insights, Analytics, Communication, Computer Programming, Data Analytics, Data Engineering, Data Science, Product Roadmapping, Python (Programming Language), Structured Query Language (SQL), When working from home, you must have a dedicated, private workspace free from distractions, along with appropriate desk and seating. Reliable internet is required, with minimum speeds of 50 MB download and 5 MB upload.

## Description

We are seeking a Lead Data Engineering (DAE) Consultant to design and deliver scalable data infrastructure and advanced analytics solutions that support Data Science and business teams. This role acts as a technical leader and trusted partner, working closely with data scientists, analysts, and stakeholders to transform complex, ambiguous problems into production-ready data solutions., Lead the design and implementation of scalable data pipelines and ETL workflows

 - Architect and optimize enterprise data infrastructure using Microsoft Fabric
 - Partner with stakeholders to translate ambiguous business needs into effective technical solutions
 - Translate complex technical solutions into clear, actionable insights for business partners
 - Develop prototypes and lead transition into production-grade systems
 - Establish and maintain data quality, monitoring, and validation frameworks
 - Leverage AI/ML tools to enhance automation and insights
 - Drive change management and adoption of new data solutions
 - Optimize performance for large-scale data environments
 - Serve as SME guiding data engineering best practices
 - Influence data strategy and support evolving business needs, The candidate(s) offered this position will be required to submit to a background investigation.

Joining our team isn't just a job - it's an opportunity. One that takes your skills and pushes them to the next level. One that encourages you to challenge the status quo. One where you can shape the future of protection while supporting causes that mean the most to you. Joining our team means being part of something bigger - a winning team making a meaningful impact.

Allstate generally does not sponsor individuals for employment-based visas for this position.

Effective July 1, 2014, under Indiana House Enrolled Act (HEA) 1242, it is against public policy of the State of Indiana and a discriminatory practice for an employer to discriminate against a prospective employee on the basis of status as a veteran by refusing to employ an applicant on the basis that they are a veteran of the armed forces of the United States, a member of the Indiana National Guard or a member of a reserve component.


## Related Videos

- [From Messy Queries to Scalable Systems - How Data Engineering actually works](https://www.wearedevelopers.com/videos/100203-from-messy-queries-to-scalable-systems-how-data-engineering-actually-works)
- [Alibaba Big Data and Machine Learning Technology](https://www.wearedevelopers.com/videos/37-alibaba-big-data-and-machine-learning-technology)
- [Modern Data Architectures need Software Engineering](https://www.wearedevelopers.com/videos/1030-modern-data-architectures-need-software-engineering)
- [PySpark - Combining Machine Learning & Big Data](https://www.wearedevelopers.com/videos/44-pyspark-combining-machine-learning-big-data)
- [The Data Mesh as the end of the Datalake as we know it](https://www.wearedevelopers.com/videos/156-the-data-mesh-as-the-end-of-the-datalake-as-we-know-it)
- [Why and when should we consider Stream Processing frameworks in our solutions](https://www.wearedevelopers.com/videos/1085-why-and-when-should-we-consider-stream-processing-frameworks-in-our-solutions)

## Related Articles

- [How to Become an AI Engineer](https://www.wearedevelopers.com/magazine/331-how-to-become-an-ai-engineer)
- [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers)
- [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk)
- [What Industries Outside of AI Are Hiring The Most AI Experts?](https://www.wearedevelopers.com/magazine/98-what-industries-outside-of-ai-are-hiring-the-most-ai-experts)
- [Data Analyst Salary in the UK](https://www.wearedevelopers.com/magazine/278-data-analyst-salary-in-the-uk)
- [Navigating the AI Shift](https://www.wearedevelopers.com/magazine/629-navigating-the-ai-shift)