Azure Data Engineer

TechDigital Corporation
Pleasanton, CA, United States
1 day ago
Apply on www.thejobnetwork.com
Prepare application

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Working hours
Regular working hours

Tech stack

Microsoft Azure Cloud Computing Databases Continuous Integration Data Validation Data Integration Extract Transform Load (ETL) Data Transformation Data Systems Data Warehousing Dimensional Modeling Apache Hive
+21 more
Performance Tuning Scrum Methodology Query Optimization Azure DevOps Pipelines Azure Data Lake SQL Stored Procedures SQL Databases Data Streaming Transact-SQL Parquet Azure Data Factory Apache Spark Git Build Management Data Lakes Pyspark Microsoft Tfs Software Version Control Data Pipelines Key Vault Databricks

Job description

” Design, develop, and maintain data pipelines using Azure Databricks (PySpark/Spark SQL) and Delta Lake. β€œ Build and orchestrate ETL/ELT workflows in Azure Data Factory, including linked services, datasets, and triggers. β€œ Write and optimize complex SQL for data transformation, aggregation, and performance tuning. β€œ Collaborate with cross-functional teams to deliver reliable, high-quality data solutions. β€œ Implement data quality checks, monitoring, and follow engineering best practices. β€œ Work in an Agile environment and contribute to sprint planning and retrospectives.

Requirements

We are seeking a skilled Azure Data Engineer with strong hands-on expertise in Azure Databricks (ADB), Azure Data Factory (ADF), and SQL. The ideal candidate will design and build scalable data pipelines and transformations on the Azure platform, with solid experience in Spark (PySpark), the Databricks Lakehouse, and ETL/ELT orchestration. Exposure to Delta Lake, data modeling, and CI/CD for data workloads is a plus., β€œ Databricks: Strong proficiency in Azure Databricks, PySpark, Spark SQL, and notebook development. β€œ Data Integration: Hands-on experience with Azure Data Factory (pipelines, mapping data flows, triggers). β€œ Database: Expert-level SQL (mandatory) with T-SQL/Spark SQL; query optimization and stored procedures. β€œ Storage & Format: Experience with Delta Lake, Parquet, and Azure Data Lake Storage (ADLS Gen2). β€œ Data Modeling: Understanding of dimensional modeling and data warehousing concepts. β€œ Version Control: Git/Azure Repos or similar tools. β€œ CI/CD: Familiarity with Azure DevOps pipelines for data deployments. β€œ Cloud: Working knowledge of the broader Azure ecosystem (Key Vault, Synapse is a plus).

Apply for this position

This job is hosted externally. Click below to view the full posting and apply.

Apply on www.thejobnetwork.com
Prepare application

Good distractions

Loading talks and stories from around this role…