Principal Data Engineer
- Discuss this with your agent
- Open in Claude
- Open in ChatGPT
Prepare application
- Draft this with your agent
- Open in Claude
- Open in ChatGPT
Role details
Tech stack
+10 more
Job description
We are seeking an experienced Data Engineering Manager to lead and support the development of modern, scalable data platforms. This individual will play a key role in designing, building, and optimizing enterprise-level data solutions using Microsoft Azure and Azure Databricks. The ideal candidate will bring strong hands-on data engineering experience, particularly with Azure Databricks, PySpark, Python, SQL, and Lakehouse Architecture. This role is suited for a technical leader who enjoys remaining involved in development while guiding and mentoring data engineering teams. We are also open to experienced Lead Data Engineers, Principal Data Engineers, and Databricks Technical Leads who have strong technical expertise and leadership capabilities., * Lead the design, development, and implementation of scalable enterprise data platforms using Azure Databricks.
- Build, maintain, and optimize batch and streaming data pipelines using Azure Data Factory and other Azure data services.
- Design and implement Lakehouse Architecture, Delta Lake, and Medallion Architecture.
- Develop and optimize data processing solutions using PySpark, Python, SQL, and Apache Spark.
- Improve data pipeline performance, scalability, reliability, and efficiency.
- Implement data governance, security, and quality standards across enterprise data platforms.
- Provide technical leadership, mentorship, and guidance to data engineering team members.
- Collaborate with cross-functional teams to deliver reliable data solutions that support business and analytics needs.
Requirements
- 5-7+ years of experience in data engineering, data architecture, or related technical roles.
- 2-4 years of hands-on experience working with Azure Databricks.
- Strong experience with PySpark, Python, SQL, and Apache Spark.
- Hands-on experience with Delta Lake, Lakehouse Architecture, and Medallion Architecture.
- Experience building and managing batch and streaming data pipelines using Azure Data Factory and Azure data services.
- Strong understanding of data governance, pipeline optimization, and Azure cloud infrastructure.
- Experience designing and implementing production-level data engineering solutions.
Nice-to-Have Qualifications
- Experience managing, leading, or mentoring data engineering teams.
- Knowledge of Scala and PowerShell.
- Experience with Unity Catalog, Kafka, MLflow, and Apache Airflow.
- Familiarity with Azure DevOps and CI/CD practices.
- Relevant certifications, including Azure Data Engineer, Databricks Data Engineer, or Azure Solutions Architect.
Apply for this position
This job is hosted externally. Click below to view the full posting and apply.
Prepare application
- Draft this with your agent
- Open in Claude
- Open in ChatGPT
Good distractions
Loading talks and stories from around this roleβ¦