Data Engineer - Databricks & Python

Editx Bv

Brussel, Belgium

5 days ago

Apply on europa.eu

Prepare application

Role details

Contract type

Temporary contract

Employment type

Full-time (> 32 hours)

Experience level

Experienced

Experience required

3 years minimum

Working hours

Regular working hours

Languages

Dutch, English

Job source

europa.eu

Tech stack

Java (Programming Language) Application Programming Interfaces (APIs) Big Data Databases Data Infrastructure Extract Transform Load (ETL) Data Transformation Statistical Hypothesis Testing Python (Programming Language) Azure Data Lake Pulumi Apache Spark

+8 more

Infrastructure as Code (IaC) Data Lakes Pyspark Data Management Restful APIs Terraform Data Pipelines Databricks

Job description

Explore and validate new data and AI technologies through short-cycle feasibility projects.
Design and implement scalable ETL processes and data pipelines.
Integrate diverse data sources such as databases and APIs into the central analytics platform.
Collaborate in multidisciplinary teams to test hypotheses and demonstrate technical potential.
Maintain data consistency, quality, and reliability across the entire pipeline lifecycle.
Implement Infrastructure as Code (IaC) to support and manage data environments.

Requirements

3+ years of experience as a data engineer.
You have experience with Python, Java, or Scala for developing datastreams and ETL processes.
You possess hands-on experience with Databricks, Spark, and PySpark for processing large datasets.
You bring experience with data lake solutions such as Azure Data Lake or Delta Lake.
You have experience using DBT for data transformation and modeling.
You possess experience with Terraform or Pulumi for IaC.
You have experience connecting data sources via REST APIs and various database systems.
You’re proactive and solution-oriented in a fail-fast environment.
You are fluent in Dutch with a C2 level.

Nice to Haves