Lead Azure Data Engineer- TECHNICAL LEAD

Source Select Group
United States
about 1 month ago
Apply on www.indeed.com
Prepare application

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Experience level
Expert
Experience required
1 year minimum
Working hours
Regular working hours
Job source

Tech stack

Java (Programming Language) Airflow Business Analytics Applications Microsoft Azure Bash Shell Big Data Code Review Information Engineering Extract Transform Load (ETL) Data Systems Data Warehousing Database Design
+15 more
Python (Programming Language) Machine Learning Microsoft SQL Server NoSQL Performance Tuning Azure Data Lake Transact-SQL Feature Engineering Azure Data Factory Apache Spark Microsoft Fabric Pyspark Machine Learning Operations Azure Service Fabric Databricks

Job description

This is a full-time, remote position leading a team of Data Scientists and Azure Data Engineers in the design, development, and delivery of machine learning models, enterprise data pipelines, and analytics solutions on Microsoft Azure and Fabric.

  • This full-time remote role is for a Lead Azure Data Engineer/Architect responsible for designing, building, and optimizing data solutions on Microsoft Azure.
  • he Lead Azure Data Engineer/Architect will collaborate closely with stakeholders to gather requirements, translate them into technical designs, and ensure data quality, security, and governance standards are met.
  • Day-to-day responsibilities include hands-on development, performance tuning, code reviews, and mentoring other data engineers., * Architect and build scalable pipelines using:
  • Databricks
  • Apache Airflow
  • Fabric Data Factory
  • Azure Date Engineering
  • Microsoft Fabric (Lakehouse, OneLake, Semantic Models)
  • Implement medallion architecture and modern data warehousing
  • Ensure scalability, performance, and resilience
  • Own full lifecycle: feature engineering * model design * training * validation * deployment * monitoring
  • Drive model selection, tuning, and evaluation strategies
  • Deliver predictive analytics tied to project and financial data
  • REQUIRED SKILLS

Technical (Hands-On Leadership Required)

  • Expert in Python, advanced T-SQL, strong Spark/PySpark
  • Deep experience in:
  • Azure Data Factory (ADF) for ML pipelines (required)
  • Data engineering - ETL/ELT, data warehousing, medallion architecture
  • Advanced experience with:
  • Azure ecosystem
  • Databricks
  • Microsoft Fabric
  • Apache Airflow, * Lead the design and implementation of comprehensive data architectures leveraging Azure Data Lake, Azure Data Factory, and other Azure cloud services to support enterprise analytics needs.
  • Develop and optimize scalable ETL (Extract, Transform, Load) processes using tools such as Informatica, Spark, and custom scripting in Java, Python, or Bash.
  • Architect and maintain large-scale data warehouses utilizing Microsoft SQL Server, and NoSQL databases
  • Collaborate with cross-functional teams to define data modeling standards, ensuring efficient database design and integration of linked data sources.

Requirements

  • 7+ years in Azure Data Engineering
  • 3+ years of work experience with Azure Data Factory
  • 1+ years of work or strong knowledge of Azure Service Fabric
  • 4+ years of work experience with Azure Databricks
  • 3+ years in technical leadership roles
  • Bachelor’s degree required, * Lead Data engineer: 6 years (Required)
  • Azure Data Lake: 3 years (Required)
  • Python: 3 years (Required)

Benefits & conditions

401(k), Health insurance, Paid time off, Vision insurance, Dental insurance, Life insurance Full-time Remote, * 401(k)

  • Dental insurance
  • Health insurance
  • Life insurance
  • Paid time off
  • Vision insurance

Application Question(s):

  • While this position is remote the company does require you to be on-site for the final interview, first week of hired and the annual meeting in Florida. Are you able to do this? No exceptions?

Apply for this position

This job is hosted externally. Click below to view the full posting and apply.

Apply on www.indeed.com
Prepare application

Good distractions

Talks and stories from around this role — technically off-topic, practically not.

3:03 min

Career evolution in data engineering and AI platforms

Maria Apazoglou · Coffee With Developers

4:32 min

Harnessing Spark with Python using PySpark and Py4J

Ayon Roy · LIVE

2:15 min

Empowering domain teams with an open data platform

Sandhya Menon Sandhya Menon · World Congress 2026 Europe

2:37 min

Comparing traditional SQL tables versus NoSQL non-tabular databases

Stanimira Vlaeva · JS Congress

3:37 min

Scaling machine learning pipelines from prototypes to petabytes

Julian Joseph · LIVE

3:05 min

Audience questions on AI agents and pipeline vectorization

Joy Joy · World Congress 2024

Videos

See all

Related articles

See all