Senior Data Engineer (m/w)
PROBIS Software GmbH
München, Germany
3 days ago
Role details
Contract type
Permanent contract Employment type
Full-time (> 32 hours) Working hours
Regular working hours Languages
English Experience level
SeniorJob location
Remote
München, Germany
Tech stack
.NET
API
Azure
Cloud Computing
Databases
Data as a Services
Data Architecture
Information Engineering
Data Infrastructure
ETL
Data Transformation
Data Security
Data Systems
Data Warehousing
Design of User Interfaces
PostgreSQL
Open Source Technology
Standard Sql
SQL Databases
Systems Integration
Data Storage Technologies
.NET Core
Spark
Data Lake
Data Pipelines
Databricks
Job description
The Senior Data Engineer is responsible for designing, implementing, and maintaining enterprise-scale data architecture solutions. This role designs and develops a greenfield data lakehouse system, including data ingestion pipelines, storage optimization, and high-performance query APIs. The position requires hands-on technical implementation combined with strategic architectural design skills and English speaking individual.
Data Architecture Design
- Understand PROBIS data model, identify bottlenecks and translate into
- actionable designs
- Design a modern data lakehouse architecture for greenfield implementation
- Architect scalable data storage and processing solutions using Azure
- ecosystem and applicable open source components
- Design the best stack for the job, considering data sovereignty and storage
- requirements in target markets
Technical Implementation
- Implement data ingestion pipelines from existing postgres database systems
- and integrations
- Improve the .net based data access of existing postgres system
- Build and verify high-performance query APIs on top of the greenfield design, enabling orders of magnitude growth in data volume, with improved performance
- Develop and optimize data transformation processes
- Set up monitoring and observability for reliable live operation
System Integration
- Integrate the data lakehouse with existing enterprise systems
- Design and implement performant query engines for user interfaces
- Ensure data consistency and quality across integrated systems
Documentation and Compliance
- Execute performance and consistency testing under increased loads
- Collect evidence and document the architecture
- Ensure compliance and support necessary certification processes, * Opportunity to work with cutting-edge AI, cloud technologies and tools
- Full access to our comprehensive toolset
- Flexible remote work options with home office capability
- International, team-oriented atmosphere with flat hierarchies
- Culture of open feedback and quick decision-making processes
- Regular team and company events
- Opportunity to shape the technical direction of a growing PropTech company
Requirements
- Minimum 6 years of experience in data engineering
- Minimum 4 years of hands-on experience with Microsoft Azure data services
- Proven track record of shipping performant data projects on Azure / Postgres / Data Factory / Databricks
Technical Skills
- Expert proficiency in Microsoft Azure data platform (Synapse, Data Factory, SQL Databases, Data Lake)
- Good .NET Core development skills
- Strong SQL and database (psql) optimization capabilities
- Experience with data modeling techniques and methodologies
- Ideally, experience with sovereign data solutions including Spark based data warehousing
- Knowledge of ETL/ELT processes and data pipeline design
About the company
PROBIS is a fast-growing PropTech startup from Munich that is currently redefining financial controlling for real estate projects worldwide with its cloud-based platform - from high-rises in New York to megaprojects in Saudi Arabia. Our software is technically sophisticated, complex in terms of content - and yet should be easy and intuitive for our users to operate.