Azure Data Engineer
Infinity Quest
UK
1 day ago
- Discuss this with your agent
- Open in Claude
- Open in ChatGPT
Apply on www.adzuna.co.uk
Prepare application
- Draft this with your agent
- Open in Claude
- Open in ChatGPT
Role details
Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Compensation
Β£47,522.0
Working hours
Regular working hours
Job source
Tech stack
Business Analytics Applications
Microsoft Azure
Continuous Integration
Data Validation
Extract Transform Load (ETL)
Data Warehousing
Python (Programming Language)
Key Management
Standard Sql
Azure Data Lake
SQL Databases
Data Processing
+11 more
Azure Data Factory
Apache Spark
Git
Data Lakes
Pyspark
Deployment Automation
Restful APIs
Stream Processing
Azure Synapse Analytics
Data Pipelines
Databricks
Job description
We are seeking an experienced Azure Databricks Data Engineer to design, develop, and maintain scalable data pipelines and analytics solutions on the Azure cloud platform. The ideal candidate should have expertise in Databricks, Spark, Azure Data Services, ETL/ELT development, and data warehousing concepts., * Design, develop, and optimize data pipelines using Azure Databricks and Apache Spark.
- Build and maintain ETL/ELT processes for ingesting data from multiple sources.
- Develop scalable batch and real-time data processing solutions.
- Work with Azure Data Factory (ADF), Azure Data Lake Storage (ADLS), and Azure Synapse Analytics.
- Implement Delta Lake architecture for reliable and high-performance data processing.
- Create and optimize Spark jobs using PySpark, Scala, or SQL.
- Monitor, troubleshoot, and improve data pipeline performance.
- Implement data quality checks, governance, and security best practices.
- Collaborate with business stakeholders, data analysts, and architects to understand requirements.
- Support CI/CD implementation and deployment automation.
Requirements
Azure Services
- Azure Databricks
- Azure Data Factory (ADF)
- Azure Data Lake Storage Gen2 (ADLS)
- Azure Synapse Analytics
- Azure Key Vault
- Azure Monitor
Technical Skills
- PySpark (Mandatory)
- Apache Spark
- Python
- SQL
- Delta Lake
- Data Modeling
- ETL/ELT Development
- REST APIs
- Git/Azure DevOps
Apply for this position
This job is hosted externally. Click below to view the full posting and apply.
Apply on www.adzuna.co.uk
Prepare application
- Draft this with your agent
- Open in Claude
- Open in ChatGPT
Good distractions
Loading talks and stories from around this roleβ¦