Senior Big Data Engineer
Role details
Job location
Tech stack
Job description
Join the Media Products Analytics Engineering division focused on building platforms and tools to manage and process terabytes of data for both internal and external stakeholders. Our team is responsible for developing robust data infrastructure and high-performance pipelines that power critical media analytics and projection datasets., We're looking for a Senior Big Data Engineer who can work independently and proactively to design, implement, and maintain scalable data processing pipelines. You will contribute to a high-performing team that owns infrastructure critical to analytics, modeling, and media performance reporting., * Design and develop large-scale data pipelines and infrastructure to process high-volume datasets
- Build and maintain scalable, production-grade ETL/ELT workflows using Apache Spark, Scala, and SQL
- Work with distributed file systems (HDFS) and optimize data processing performance
- Write and test reusable and reliable code using functional programming libraries like Cats and fs2
- Develop and test solutions in containerized environments (e.g., Docker, Kubernetes)
- Collaborate with analysts, platform teams, and product stakeholders to deliver data-driven solutions
- Take ownership of services with minimal supervision, demonstrating initiative and autonomy
Requirements
- Strong programming experience with Scala and distributed data frameworks like Apache Spark
- Proficiency in SQL and experience handling large datasets in HDFS environments
- Working knowledge of Python for scripting and data manipulation tasks
- Hands-on experience developing and testing in Docker and/or Kubernetes environments
- Familiarity with Cats, fs2, or other advanced Scala functional programming libraries
- Ability to work autonomously in a dynamic, fast-paced environment and take full ownership of projects
We offer
- Opportunity to work on bleeding-edge projects
- Work with a highly motivated and dedicated team