AWS EKS & Kafka Platform Lead Engineer

What You''''ll Do As a lead member of our team, you''''ll focus on designing, deploying, and maintaining our core streaming and orchestration platforms. This is a hands-on leadership role.

• Lead the setup, configuration, and ongoing management of Kubernetes clusters on AWS EKS, ensuring high availability and performance for data workloads.
• Design Apache Kafka operations at massive scale: create and manage topics, scale nodes/brokers, implement tagging/classification, tune for 1M+ records/second throughput, and handle petabyte-level storage.
• Deploy Kafka using modern tools like Strimzi (Kafka operator for Kubernetes), Confluent components, and Helm charts.
• Design and build integration and orchestration with tools such as Apache Spark, Airflow, Apache Beam, and emerging Flink pipelines for batch and streaming workloads.
• Contribute to building and enhancing our Data Lake and Data Mesh architecture, incorporating open table formats like Apache Iceberg for reliable, versioned data storage.
• Collaborate with data engineers and application teams to provision resources, troubleshoot issues, and implement best practices for observability, security, and cost efficiency.
• Automate infrastructure and deployments using IaC principles (e.g., Helm, Terraform where applicable) to enable self-service for teams.

• You''''re a hands-on architect who can delegate, director, and mentor other talented team members.
• Strong primary expertise in AWS, especially EKS (Kubernetes cluster management).
• Solid production experience with Apache Kafka at scale-topic creation/management, cluster scaling, performance tuning, and related ecosystem tools.
• Familiarity with Kafka deployment on Kubernetes via Strimzi, Confluent, or Helm.
• Working knowledge of related data processing tools: Spark, Airflow, Apache Beam, and ideally exposure to Flink or similar streaming engines.
• Experience or interest in modern data lake technologies like Apache Iceberg, Data Lake / Data Mesh patterns.
• Comfortable in a mixed on-prem and cloud (AWS) environment.
• Strong problem-solving skills, attention to detail, and a collaborative mindset.

If you''''re excited about high-throughput streaming on AWS EKS, Kubernetes-native platforms, and contributing to a growing Data Mesh/Data Lake initiative-while enjoying great perks and a balanced lifestyle-we''''d love to hear from you!

We''''re architecting and building a modern, high-scale data platform to power real-time analytics and a next-generation Data Lake / Data Mesh architecture. Our environment handles massive throughput with Kafka processing over 1 million records per second and petabytes of total data, running on AWS EKS (Kubernetes) with a mix of on-prem and cloud components. Join in a leadership role on a collaborative team where your work directly enables cutting-edge data streaming and processing. We value people who like to get their hands dirty and keep their skills sharp. Perks * Free GrubHub food deliveries to keep you fueled * Free downtown parking * Frequent team parties every 2 weeks - we celebrate wins and enjoy time together * Work on high-impact, large-scale systems without after-hours support demands * Opportunity to contribute to evolving initiatives like Data Lake, Data Mesh, Apache Iceberg, and real-time processing pipelines