Lead Streaming Platform Engineer

We are seeking a highly skilled and motivated Streaming Platform Engineer to join the Data Streaming Platform team. This unique hybrid role combines platform, software, and data engineering to build, scale, and maintain our high-performance real-time data streaming platform. The ideal candidate has a passion for architecting robust, scalable systems to enable data-driven products and services at massive scale.

Your Mission

• Design, build, and maintain the core infrastructure for our real-time data streaming platform, ensuring high availability, reliability, and low latency.
• Implement and optimize data pipelines and stream processing applications using Apache Kafka, Apache Flink, and Spark Streaming.
• Collaborate with software and data engineering teams to define event schemas, ensure data quality, and support integration of new services into the streaming ecosystem.
• Develop and maintain automation and tooling for platform provisioning, configuration management, and CI/CD pipelines.
• Champion the development of self-service tools and workflows that empower engineers to manage their own streaming data needs, reducing friction and accelerating development.
• Monitor platform performance, troubleshoot issues, and implement observability solutions (metrics, logging, tracing) to ensure the platform’s health and stability.
• Stay up-to-date with the latest advancements in streaming and distributed systems technologies and propose innovative solutions to technical challenges.

Qualifications

• Strong production experience with Apache Kafka and its ecosystem (e.g., Confluent Cloud, Kafka Streams, Kafka Connect). Solid understanding of distributed systems and event-driven architectures.
• Experience building and optimizing real-time data pipelines for ML, analytics, and reporting, leveraging Apache Flink, Spark Structured Streaming, and low-latency OLAP systems like Apache Pinot.
• Hands-on experience with major cloud platforms (AWS, GCP, or Azure), Kubernetes, Docker, Terraform, CI/CD (GitHub Actions), and observability tools (New Relic, Prometheus, Grafana).
• Proficiency in at least one programming language: Python, TypeScript, Java, Scala, or Go.
• Familiarity with data platform concepts, including data lakes and data warehouses.

About the Team

You will be part of a talented and diverse team of data engineers, data scientists, and product managers focused on revolutionizing the use of stream-processing across the organization. We’re building innovative data solutions to optimize internal processes, enhance customer experiences, and drive business growth.

On is an Equal Opportunity Employer. We are committed to creating a work environment that is fair and inclusive, where all decisions related to recruitment, advancement, and retention are free of discrimination.

We are seeking a highly skilled and motivated Streaming Platform Engineer to join the Data Streaming Platform team. This unique hybrid role combines platform, software, and data engineering to build, scale, and maintain our high-performance real-time data streaming platform. The ideal candidate has a passion for architecting robust, scalable systems to enable data-driven products and services at massive scale., * Strong production experience with Apache Kafka and its ecosystem (e.g., Confluent Cloud, Kafka Streams, Kafka Connect). Solid understanding of distributed systems and event-driven architectures.

• Experience building and optimizing real-time data pipelines for ML, analytics, and reporting, leveraging Apache Flink, Spark Structured Streaming, and low-latency OLAP systems like Apache Pinot.
• Hands-on experience with major cloud platforms (AWS, GCP, or Azure), Kubernetes, Docker, Terraform, CI/CD (GitHub Actions), and observability tools (New Relic, Prometheus, Grafana).
• Proficiency in at least one programming language: Python, TypeScript, Java, Scala, or Go.
• Familiarity with data platform concepts, including data lakes and data warehouses.