Senior Data Engineer

Everience Benelux

Brussel, Belgium

6 days ago

Role details

Contract type

Permanent contract

Employment type

Full-time (> 32 hours)

Experience level

Expert

Working hours

Regular working hours

Job source

www.adzuna.be

Tech stack

Query Performance .NET Framework Airflow Big Data C Sharp (Programming Language) Cloud Computing Databases Directed Acyclic Graph (Directed Graphs) Data Deduplication Information Engineering Data Governance Data Infrastructure

+15 more

Data Integrity Extract Transform Load (ETL) Data Warehousing Digital Assets Python (Programming Language) Cloud Services Unstructured Data Software Organization Sql Optimization Apache Spark Git Data Lakes Pyspark Data Management Databricks

Job description

You will be responsible for ensuring high-quality access to data sources, with a strong focus on data quality, standardization, qualification, and governance to enable effective use by Data Analysts and Data Scientists.

You will contribute to the definition and implementation of data policies and to the structuring of the data lifecycle, ensuring compliance with regulatory requirements in collaboration with key data governance stakeholders.

You will oversee data management and processing systems, including Big Data and IoT platforms. You will ensure the integration of structured and unstructured data from multiple sources and maintain high data quality within the Data Lake through rigorous testing, validation, and deduplication processes.

Data Qualification & Data Management:

Capture structured and unstructured data generated by internal applications and external sources.
Integrate and consolidate data from multiple systems and environments.
Structure data through semantic modeling and standardization practices.
Map available data assets and maintain data consistency.
Clean and enrich datasets, including duplicate elimination and quality remediation.
Validate data integrity and business consistency.
Create and maintain data repositories where required.

Requirements

Data Engineering & Development

Proficiency in Python and .NET/C# for script development and ETL implementations.
Strong knowledge of software development best practices and version control using Git.
Expertise in modern data acquisition, preparation, integration, and transformation techniques.

Databases & Data Warehousing

Strong understanding of relational databases and SQL.
Ability to design efficient data models and optimize query performance.
Experience working with enterprise data warehouses and large-scale data platforms.

Big Data & Cloud Technologies

Experience with Databricks, including ETL development, Spark cluster management, and SQL optimization.
Experience with PySpark and Apache Spark ecosystems.
Knowledge of AWS cloud services and data platform solutions.
Experience building and orchestrating ETL workflows using Airflow and DAGs.