> Markdown version of [/jobs/ext/86111-senior-data-platform-engineer](https://www.wearedevelopers.com/jobs/ext/86111-senior-data-platform-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Senior Data Platform Engineer - **Company:** MY GAMES - **Location:** Germany (Remote available) - **Experience:** Expert - **Contract:** Permanent contract - **Skills:** Airflow, Continuous Integration, Cron, Directed Acyclic Graph (Directed Graphs), Data Governance, Data Infrastructure, Extract Transform Load (ETL), Data Warehousing, Distributed Computing Environment, Hadoop Distributed File System, Python (Programming Language), Prometheus, Standard Sql, DataOps, Runbook, SQL Databases, Toolchain, Grafana, Apache Spark, Data Lakes, Kubernetes, Deployment Automation, Apache Kafka, Data Pipelines - **Published:** May 29, 2026 - **Apply:** https://de.indeed.com/viewjob?jk=334649475643e6e7 ## About the Role Do you have experience in Spark?, * Hands-on experience as a Data Engineer, Data Platform Engineer, DataOps Engineer, or in a similar role * Proven track record building and maintaining ETL/ELT pipelines * Strong command of Python and SQL * Practical experience with Spark or other distributed processing frameworks * Deep knowledge of Airflow internals: scheduler, workers, executors, DAG lifecycle, retries, sensors, pools, queues, SLAs, logs, alerts, and callbacks/listeners * Experience running data workflows in production environments * Ability to set up monitoring, alerting, and dashboards - e.g., Grafana/Prometheus-style observability stacks * Solid understanding of monitoring Airflow, Spark, HDFS, and Kafka: failures, lag, resource usage, data freshness, completeness, queue time, and scheduler health * Experience automating engineering processes: scripts, CLIs, internal tools, deployment automation, and validation checks * Familiarity with DWH, Data Lake, and Lakehouse architectures * Experience working with legacy pipelines and migrating them to a modern stack * Kubernetes proficiency at the application level: containers/images, deployments, jobs, cronjobs, configs, secrets, and resource requests/limits * Ability to drive a technical task end-to-end - from problem scoping to a production-ready solution * You think beyond implementation: reliability, observability, maintainability, and lifecycle are part of your definition of done ## Description * Drive the evolution of our Data Platform and lead migration to a new on-premise stack * Build, maintain, and migrate ETL/ELT pipelines from the legacy environment to the new stack * Work across the full toolchain: Python, SQL, Spark, Airflow, HDFS, Kafka, Trino, Iceberg, and dbt * Own Airflow as a production orchestration layer - DAGs, deployment, retries, sensors, pools, queues, callbacks/listeners, backfills, and reruns * Design and implement DataOps practices: monitoring, alerting, SLA/SLO tracking, runbooks, incident diagnostics, and postmortems * Set up observability across Airflow, Spark, HDFS, Kafka, and other platform components * Build custom listeners, exporters, checkers, and internal tooling for platform health diagnostics * Automate recurring team operations: DAG deployment, pipeline migrations, backfill/retry/recovery flows, and pre-release validation * Advance CI/CD and production-readiness standards for data workflows * Contribute to Data Governance at the engineering level - ownership, naming conventions, metadata, lineage, access patterns, auditability, and privacy-by-design * Work with Kubernetes at the application level: updating images, configuring deployments/jobs/cronjobs, migrating services, and managing configs, secrets, and env variables * Help the team cut down on manual ops, recurring failures, and operational noise ## Related Videos - [From Messy Queries to Scalable Systems - How Data Engineering actually works](https://www.wearedevelopers.com/videos/100203-from-messy-queries-to-scalable-systems-how-data-engineering-actually-works) - [From clicks to cribs - How to find your dream home with web scraping](https://www.wearedevelopers.com/videos/767-from-clicks-to-cribs-how-to-find-your-dream-home-with-web-scraping) - [5 steps for running a Kubernetes environment at scale](https://www.wearedevelopers.com/videos/88-5-steps-for-running-a-kubernetes-environment-at-scale) - [Enjoying SQL data pipelines with dbt](https://www.wearedevelopers.com/videos/823-enjoying-sql-data-pipelines-with-dbt) - [AI Model Management Life Circles: ML Ops For Generative AI Models From Research to Deployment](https://www.wearedevelopers.com/videos/1152-ai-model-management-life-circles-ml-ops-for-generative-ai-models-from-research-to-deployment) - [Is your backend a hodgepodge of queues, event stores and cron jobs? Durable Execution to the Rescue.](https://www.wearedevelopers.com/videos/744-is-your-backend-a-hodgepodge-of-queues-event-stores-and-cron-jobs-durable-execution-to-the-rescue) ## Related Articles - [Where To Find Software Engineering Jobs](https://www.wearedevelopers.com/magazine/396-where-to-find-software-engineering-jobs) - [7 Most Popular Web Developer Jobs in Europe](https://www.wearedevelopers.com/magazine/163-7-most-popular-web-developer-jobs-in-europe) - [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk) - [The Most Popular IT Jobs on the Market](https://www.wearedevelopers.com/magazine/376-the-most-popular-it-jobs-on-the-market) - [Find a Developer Job: 12 Best Job Sites For Developers](https://www.wearedevelopers.com/magazine/165-find-a-developer-job-12-best-job-sites-for-developers) - [The 12 Best Jobs for Software Engineers](https://www.wearedevelopers.com/magazine/401-the-12-best-jobs-for-software-engineers)