Data Engineer

• Contribute to production support and maintenance of the DataHub, correcting incidents, anomalies, and resolving data-related issues while implementing functional and technical developments to ensure stability.
• Modify existing code per business requirements and continuously improve performance and maintainability.
• Ensure performance and security of the data infrastructure, following best practices.
• Design and develop efficient data pipelines to enrich and transform large volumes of data with complex business rules, optimize ingestion, and automate pipelines.
• Integrate data from multiple sources into the raw layer of the DataHub.
• Implement data transformation and quality processes to ensure consistency and accuracy using Scala, SQL, and Spark.
• Configure CI/CD pipelines for automated deployments, unit testing, and development management.
• Write and perform unit and validation tests to ensure code integrity.
• Implement orchestrators and scheduling processes to automate data pipeline execution (Airflow as a Service).
• Migrate Hadoop infrastructure to cloud infrastructure on Kubernetes Engine, COS, Spark as a Service, and Airflow as a Service.
• Write technical documentation (specifications, operational documents).

• Experience with HDFS and structured SQL databases.
• Proficiency in Shell Script.
• Knowledge of Elasticsearch and Kibana.
• Experience with Vault (or HVault).
• Experience with Dremio for data virtualization.
• Fluency in English and French.

Preferred

• Experience with Elasticsearch, Kibana.
• Experience with Vault.

We are an international technology services company founded in 1983 and currently have over 2,000 employees in 5 countries: France, Spain, Romania, Portugal, and Luxembourg.