Lead Data Engineer

In this role, you will directly contribute to Lam's Enterprise AI strategy by building the scalable, AI-ready data foundation that powers generative AI, machine learning, and advanced analytics across the company. You will define and deliver modern data architecture, trusted data products, and production-grade data pipelines that enable Enterprise AI use cases at scale. What you'll do

• Architect and build scalable, production-grade data pipelines and data products on Azure, Microsoft Fabric, and related cloud data platforms to support Enterprise AI use cases.
• Design and implement data architecture patterns, including Medallion architecture and curated data layers, to improve data quality, usability, and governance for AI and analytics workloads.
• Develop and optimize large-scale data processing solutions using Python, SQL, Spark, and modern Azure data services to support high-volume, high-performance workloads.
• Build AI-ready data assets for machine learning and generative AI applications, including vector search, Retrieval-Augmented Generation (RAG), graph-based use cases, and semantic data models.
• Partner closely with AI/ML engineers, data scientists, business teams, and platform teams to translate AI use-case requirements into scalable, governed, and production-ready data solutions.
• Drive architecture decisions for data integration, storage, transformation, and serving layers that support the long-term Enterprise AI roadmap.
• Contribute to engineering quality through CI/CD, automation, observability, and operational readiness practices for data pipelines and platforms.
• Document architecture decisions, technical designs, and operating procedures to support maintainability, operational readiness, and long-term platform sustainability.

• Bachelor's degree in Computer Science, Computer Engineering, Information Systems, or a related field.
• 8+ years of experience in data engineering, data architecture, platform engineering, or related technical roles.
• Strong hands-on experience designing and building cloud-based data solutions on Microsoft Azure.
• Strong proficiency in Python, SQL, and Spark for data transformation, pipeline development, and large-scale data processing.
• Experience with modern data architecture patterns, including Lakehouse and Medallion architecture, and building production-grade data pipelines that integrate data from multiple enterprise systems.
• Experience partnering with cross-functional teams to deliver scalable technical solutions that support business and platform goals.

Preferred qualifications

• Experience with Microsoft Fabric, Azure Databricks, Azure Data Factory, Synapse, or related Azure data services.
• Experience supporting AI/ML or generative AI use cases, including data preparation for model training, inference, vector search, or Retrieval-Augmented Generation (RAG) workflows.
• Experience with graph data models, knowledge graph technologies, or graph databases such as Neo4j.
• Experience with Azure AI Search, event-driven ingestion patterns, CI/CD for data engineering, and monitoring or observability practices for data platforms.

CA San Francisco Bay Area Salary Range for this position: $141,000.00 -$307,000.00.

The above salary range for this position is relevant to applicants that reside or work onsite in the California, San Francisco Bay Area only. Salary offers will depend on factors that include the location you work from, your level, education, training, specific skills, years of experience and comparison to other employees already in this role. Actual salary may vary from salary offered due to numerous factors including but not limited to unpaid time off, unpaid leave, company mandated shutdown, and other relevant factors.

We believe it is important for every person to feel valued, included, and empowered to achieve their full potential. By bringing unique individuals and viewpoints together, we achieve extraordinary results.