Data Engineer

Onesource Consulting
Ixelles, Belgium
yesterday

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Working hours
Regular working hours
Languages
English

Job location

Ixelles, Belgium

Tech stack

Java
Airflow
Amazon Web Services (AWS)
Amazon Web Services (AWS)
Amazon Web Services (AWS)
Big Data
Cloud Database
Databases
Data Architecture
Data Governance
Data Infrastructure
ETL
Data Stores
Identity and Access Management
Python
Standard Sql
Software Engineering
SQL Databases
Data Streaming
Workflow Management Systems
Spark
Cloudformation
Data Lake
Data Management
Api Gateway
Amazon Web Services (AWS)
Data Pipelines
Go
Programming Languages

Job description

  • We are looking for a Data engineer to strengthen the Data platform team.
  • Are you passionate about Cloud data architecture, ELT, data pipelines, software development and building (real-time) data applications? Are you also an analytical and critical thinker? The Client Data platform team is looking for reinforcement to take the next steps in the maturity of the platform and to increase the adoption of the platform within Client. In addition to the development and delivery of new data pipelines, this also means the continuous optimization of existing components and processes and the addition of new use cases

YOUR CHALLENGE

  • You are responsible for building out the data lake and you know how to take Client to the next level by integrating different data sources, but also by collaborating on various use cases for analytics and BI.

Requirements

  • You design, build, test and manage data management systems.
  • You describe and map the data from different source systems to the target data store.
  • You have a good knowledge of data architecture, infrastructure and/or development.
  • You have a good knowledge of data governance principles and know how to apply them concretelyYou have experience with "Infrastructure as code".
  • You have a good knowledge of Python, Go, Spark, SQL and you quickly learn new techniques.
  • You have thorough knowledge of the AWS cloud platform and have experience in using AWS big data services such as EMR, Glue, S3, Athena, RDS, ...
  • You have experience using Apache AirFlow.
  • You set up ELT pipelines to make data from different source systems available on the platform, via tooling or a self-written connector.
  • You can get started with dbt to transform data, so that you can easily help/talk/think along with your colleagues from the teamYou can monitor data quality.
  • You are communicative and a team playerYou are able to translate complex matters into the business in a simple way.
  • Being in possession of an AWS solution architect certification is a plus.
  • Experience within the media sector is a plus., * Demonstrable experience as a Data Engineer
  • Proven experience with AWS components for compute (EC2 / ECS / Lambda)
  • Proven experience with AWS storage/database components (S3 / RDS)
  • Proven experience with programming languages such as Python, Java or Scala for developing data streams and ETL processes
  • Proven experience with Python
  • Proven experience with SQL
  • Proven experience in data architecture within the context of data platforms
  • Proven experience in the media sector
  • Proven experience in designing and implementing Extract, Transform, Load (ETL) processes
  • Proven experience with Apache Spark
  • Proven experience with AWS Cloudformation
  • Proven experience with AWS components for applications (SNS / SQS / API Gateway)
  • Proven experience with AWS components for big data (Glue / Athena / EMR / Kinesis / SageMaker)
  • Proven experience with AWS security and identity components (IAM / KMS)
  • Proven experience with data platforms
  • Proven experience with dbt
  • Proven experience with Go
  • Demonstrable experience with Workflow Orchestration tools (e.g. Apache Airflow)

Apply for this position