Senior Azure Data Engineer (ID: 4146)
STAFIDE
Amsterdam, Netherlands
3 days ago
- Discuss this with your agent
- Open in Claude
- Open in ChatGPT
Apply on www.adzuna.nl
Prepare application
- Draft this with your agent
- Open in Claude
- Open in ChatGPT
Role details
Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Experience level
Expert
Experience required
8 years minimum
Working hours
Regular working hours
Job source
Tech stack
Microsoft Azure
Big Data
Cloud Database
Information Engineering
Data Integration
Extract Transform Load (ETL)
Data Security
Data Systems
Python (Programming Language)
Standard Sql
SQL Databases
Data Processing
+5 more
Azure Data Factory
Delivery Pipeline
Pyspark
Data Pipelines
Databricks
Job description
- Design, develop, and maintain complex data pipelines using Azure Data Factory (ADF) and Azure Databricks.
- Build, implement, and optimize large-scale data processing solutions using PySpark and Databricks.
- Develop robust ETL processes using Python for data extraction, transformation, and loading.
- Design and implement CI/CD pipelines using Azure DevOps to automate testing, integration, and deployment.
- Work with SQL to query, process, transform, and manage large datasets.
- Monitor, troubleshoot, and optimize data pipelines to ensure performance, scalability, and reliability.
- Collaborate with cross-functional teams to understand business and data requirements and deliver effective data solutions.
- Ensure data security, quality, and compliance with established standards and best practices.
- Stay up to date with emerging technologies and best practices in Azure cloud data engineering., * Exposure to modern cloud data technologies, including Azure Data Factory, Azure Databricks, and PySpark.
- An opportunity to contribute to large-scale data processing and automation initiatives.
Requirements
- 8-10 years of overall experience in IT, with strong experience in data engineering and development.
- Hands-on experience with Azure Data Factory for data integration, orchestration, and pipeline automation.
- Strong expertise in Azure Databricks for scalable data processing and engineering solutions.
- Proficiency in Python and PySpark for big data processing and ETL development.
- Experience implementing CI/CD pipelines using Azure DevOps.
- Good knowledge of SQL and experience working with large datasets.
- Experience developing, maintaining, and optimizing complex data pipelines.
- Understanding of data security, data quality, and cloud data engineering best practices.
You should possess the ability to:
- Design and develop scalable, reliable, and high-performance data engineering solutions.
- Troubleshoot complex data pipeline issues and improve processing efficiency.
- Work independently as well as collaboratively within cross-functional teams.
- Analyze technical and business requirements and translate them into effective data solutions.
- Apply problem-solving and analytical skills to overcome technical challenges.
- Communicate technical concepts clearly and maintain proper technical documentation.
- Adapt to evolving Azure technologies and modern data engineering practices.
Apply for this position
This job is hosted externally. Click below to view the full posting and apply.
Apply on www.adzuna.nl
Prepare application
- Draft this with your agent
- Open in Claude
- Open in ChatGPT
Good distractions
Loading talks and stories from around this roleβ¦