Data Engineer
softgarden GmbH
Neu-Isenburg, Germany
3 days ago
Role details
Contract type
Permanent contract Employment type
Full-time (> 32 hours) Working hours
Regular working hours Languages
English, GermanJob location
Neu-Isenburg, Germany
Tech stack
Java
API
Agile Methodologies
Azure
Big Data
Cloud Computing
Computer Programming
Data Infrastructure
Data Integration
ETL
Data Transformation
Data Virtualization
Data Warehousing
Relational Databases
Hadoop
Hadoop Distributed File System
HBase
Hive
Python
PostgreSQL
Enterprise Messaging Systems
Microsoft SQL Server
MongoDB
NoSQL
Oracle Applications
Salesforce
SAP Applications
SAP NetWeaver Business Warehouse
Software Engineering
SQL Databases
Sqoop
Data Streaming
Systems Integration
UML
Web Services
Cloud Platform System
Data Ingestion
Azure
Spark
Data Lake
Kubernetes
Information Technology
InfluxDB
Cassandra
Kafka
Apache Nifi
Api Management
Docker
Databricks
Job description
- Design and implement large-scale data solutions using Big Data and Cloud technologies
- Design and develop conceptual data models for integrating, exploring and retrieving data from heterogeneous data sources and IT systems
- Implement and manage ETL processes, data transformations, data flows and service APIs
- Work with data virtualization, integration, and analytics technologies on our data platform
- Implement optimized data representations (views) and aggregations from multiple data sources
- Define and implement data ingestion, retention, lineage and access policies
- Define and implement data service API management and usage policies
- Define and implement data and data service catalog functionalities
- Work in interdisciplinary, cross-functional teams according to agile methodology
- Closely collaborate with business, digitalization and IT functions
Requirements
- Successfully completed studies in Computer Science or related fields
- Proficient understanding of distributed and Cloud computing principles and technologies
- Experience with integration of data from multiple data sources and IT systems, e.g. ERP/SAP, DWH/SAP-BW, Web Services, CRM/Salesforce, MES, etc.
- Relevant knowledge in working with Cloud environments and services, preferably with Microsoft Azure, e.g. Azure Data Factory, Data Lake, IoT Hub, Databricks, etc.
- Strong programming skills in Java and/or Python; Software development experience is a plus
- Experience in data modeling (ER, UML) and proficiency in SQL
- Experience with relational databases such as Microsoft SQL Server and Oracle Postgres as well as experience with NoSQL databases such as MongoDB, Cassandra, InfluxDB, Stardog
- Knowledge of ETL techniques and frameworks, such as ADF, PowerCenter, NiFi, Sqoop
- Experience with Hadoop Ecosystem (Spark, HDFS, Hive, HBase), Containers (Docker, Kubernetes) and Distributed Messaging Platforms (Kafka, NATS.io) is a plus
- Fluent in English spoken and written, German is beneficial