Senior / Lead Data Engineer

• Architect, design, and implement scalable data platforms on Microsoft Azure and Databricks
• Build and optimize data ingestion, transformation, and processing pipelines for batch and real-time data
• Develop high-performance workflows using ADLS, Delta Lake, and Apache Spark (PySpark)
• Design, build, and maintain complex ETL/ELT pipelines with strong reliability and performance standards
• Create and manage conceptual, logical, and physical data models for analytics and operational use cases
• Work with relational and lakehouse systems including PostgreSQL and Delta Lake
• Define and enforce data governance, security, data quality, and architectural best practices
• Collaborate with data architects, data scientists, analysts, and business stakeholders to translate requirements into scalable solutions
• Troubleshoot production issues and continuously optimize platform performance
• Mentor junior engineers and contribute to engineering standards, reusable frameworks, and best practices

We are seeking an experienced Senior / Lead Data Engineer with 8+ years of hands-on expertise in building scalable, high-performance data platforms within the Azure ecosystem.

This role is ideal for a technically strong data engineering professional who has deep experience with Azure Databricks, Apache Spark, modern lakehouse architectures, and large-scale data pipelines. You will play a key role in designing end-to-end data solutions, influencing architectural decisions, and ensuring reliable, secure, and high-quality data systems., * 8+ years of hands-on data engineering experience in enterprise environments

• Strong expertise in Azure Data Platform, including:
• Azure Databricks
• Azure Data Factory (preferred)
• Azure Functions
• Advanced proficiency in Apache Spark (PySpark)
• Strong command of SQL, query optimization, and performance tuning
• In-depth knowledge of ETL/ELT frameworks, orchestration, and scheduling
• Hands-on experience with Delta Lake (ACID transactions, optimization, schema evolution)
• Strong background in data modelling (normalized, dimensional, and lakehouse models)
• Experience with batch and real-time/streaming data processing (Kafka, Event Hub, or similar)
• Solid understanding of cloud-native architectures, distributed systems, and data engineering best practices
• Ability to design end-to-end technical solutions and evaluate architectural trade-offs
• Excellent analytical, problem-solving, and communication skills
• Proven ability to collaborate across teams and lead technical discussions

Preferred / Nice-to-Have Skills

• Experience with CI/CD pipelines using Azure DevOps and Git
• Familiarity with Infrastructure as Code (Terraform, ARM templates)
• Exposure to data governance and cataloging tools such as Azure Purview
• Experience supporting BI, analytics, or machine learning workloads on Databricks

Compensation will be competitive and may vary based on factors such as location, experience, skill set, education, and role level. The organization offers a reasonable and market-aligned compensation range for roles hired across

DataEngineer #SeniorDataEngineer #LeadDataEngineer #AzureJobs #Databricks #BigDataJobs #PySpark #TechHiring #USJobs #GeorgiaJobs