Azure Data Engineer - Databricks & Unity Catalog
Pauwels Consulting
Brussel, Belgium
4 days ago
- Discuss this with your agent
- Open in Claude
- Open in ChatGPT
Apply on europa.eu
Prepare application
- Draft this with your agent
- Open in Claude
- Open in ChatGPT
Role details
Contract type
Temporary contract
Employment type
Full-time (> 32 hours)
Experience level
Expert
Experience required
5 years minimum
Working hours
Regular working hours
Languages
Dutch, French
Job source
Tech stack
Advanced Message Queuing Protocol
Microsoft Azure
Big Data
Continuous Integration
Data Architecture
Information Engineering
Data Sharing
SQL Azure
Power BI
SAP NetWeaver Business Warehouse
SQL Databases
Data Ingestion
+12 more
Azure Data Factory
Apache Spark
Data Lakes
Pyspark
Infrastructure Automation Frameworks
SAP S/4HANA
Restful APIs
Terraform
Stream Processing
Data Pipelines
Key Vault
Databricks
Job description
- Design and develop scalable data pipelines using Azure Data Factory and Databricks.
- Implement modern data architectures including Data Lake, Delta Lake, and Lakehouse patterns.
- Manage Databricks Unity Catalog for data sharing, permissions, and robust governance.
- Automate data ingestion, transformation, and deployment processes to enhance platform efficiency.
- Monitor and optimize resource consumption and performance within Azure and Databricks environments.
- Collaborate with data scientists and business stakeholders to industrialize new analytics use cases.
Requirements
- 5+ years of experience in data engineering with a focus on Azure cloud environments.
- You possess deep expertise in Azure Databricks and Unity Catalog, including Lakehouse design and security.
- You bring advanced skills in Spark, PySpark, and Delta Lake for large-scale data processing.
- You have strong proficiency in SQL and Python for data manipulation and platform automation.
- You possess experience with Azure Data Factory, Azure Storage, Azure SQL, and Key Vault.
- You have knowledge of CI/CD principles and Azure DevOps applied to data engineering.
- You bring familiarity with application integration patterns such as REST API and AMQP.
- You’re proactive and possess an analytical mindset with strong communication skills.
- You are fluent in French or Dutch with professional proficiency in English.
Nice to Haves
- Possession of Microsoft Azure or Databricks certifications.
- Experience with Power BI administration and governance.
- Knowledge of Terraform or Infrastructure as Code.
- Familiarity with real-time data streaming and Azure Monitor.
- Basic knowledge of SAP data sources like S/4HANA or BW/4HANA.
Apply for this position
This job is hosted externally. Click below to view the full posting and apply.
Apply on europa.eu
Prepare application
- Draft this with your agent
- Open in Claude
- Open in ChatGPT
Good distractions
Loading talks and stories from around this role…