Data Engineer DBT & Snowflake

Mit dieser Anfrage suchen wir einen Senior Data Engineer DBT & Snowflake (SQL, Python, Airflow, Energy Pricing Pipelines), * Develop & Maintain Data Pipelines: Design, build, and maintain robust data pipelines (using SQL and DBT) to process and transform large volumes of data.

• Implement Requirements in Pipelines: Incorporate new business logic and regulatory requirements into the data pipeline workflows, ensuring compliance and correct calculations.
• Data Quality & Monitoring: Establish and enhance data quality checks, including implementing monitoring and alerting mechanisms to ensure pipeline outputs are accurate and reliable.
• Testing & Quality Assurance: Create and maintain comprehensive unit, integration, and regression tests for data pipelines to guarantee stability and facilitate safe changes.
• Data Modeling: Perform technical data modeling and design data model structures (ETL processes ingesting data from Data Warehouse / Snowflake) to support efficient data storage and retrieval.
• Pipeline Orchestration & Observability: Manage pipeline orchestration (e.g., using Apache Airflow) and ensure technical observability and data lineage tracking across the pipeline infrastructure.
• Performance & Reliability: Proactively optimize the performance of pipelines and ensure their robustness and reliability in production, including troubleshooting and resolving any technical issues.

• SQL & Performance Tuning: Advanced SQL skills with hands-on experience in optimizing query performance on large datasets.
• DBT (Data Build Tool): Strong expertise in DBT, including writing tests, creating macros, and using version control for data transformations.
• Python Programming: Proficiency in Python for implementing complex data transformation logic and scripting tasks within the data pipeline.
• Data Warehousing & ETL: Solid knowledge of data warehousing concepts and ETL processes; experience with Snowflake or similar modern data platforms is highly desirable.
• Pipeline Tools: Familiarity with data pipeline orchestration and monitoring tools (e.g., Airflow, data observability frameworks) and best practices for maintaining data pipeline health.

Soft Skills:

• Communication: Excellent communication skills with the ability to explain technical topics clearly and collaborate effectively in a cross-functional team.
• Teamwork: Strong team player with a collaborative approach and a willingness to share knowledge and support colleagues.
• Language Proficiency: Professional working proficiency in English; German language skills (at least B2 level) are a plus.
• Problem-Solving: Solution-oriented mindset with a proactive attitude towards troubleshooting and continuous improvement of processes