> Markdown version of [/jobs/ext/878973-data-engineer-scala](https://www.wearedevelopers.com/jobs/ext/878973-data-engineer-scala). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Data Engineer, Scala - **Company:** Magnite, Inc. - **Location:** Los Angeles, CA, United States - **Experience:** Expert - **Salary:** $150,000.0 - $170,000.0 - **Contract:** Permanent contract - **Skills:** Java (Programming Language), Airflow, Amazon Web Services, Amazon Elastic Compute Cloud, Amazon S3, Big Data, Cloud Computing, Continuous Integration, Information Engineering, Data Infrastructure, Software Deployment, Software Engineering, Data Streaming, Data Ingestion, Apache Spark, Amazon Relational Database Service, Core Data, Information Technology, Build Tools, Data Delivery, Terraform, Data Pipelines, Docker, Jenkins, Databricks - **Published:** June 9, 2026 - **Apply:** https://www.dice.com/job-detail/6bf6836a-e967-48f2-a93b-2cb80b1d4c37 ## About the Role * Proficiency in Scala and Spark to build and optimize large-scale batch and streaming data pipelines * With 5+ years of software development experience with a BS/MS Computer Science or equivalent work experience * Who wants to take ownership. We are responsible for the entire software development lifecycle, from requirements gathering to production support * To engage in critical and creative thinking, and constructive brainstorming. We expect engineers to present and discuss tradeoffs to solve the problems we face. * With either existing experience designing and building systems that work with large-scale data volumes and data ingestion systems at scale in a cloud-first setting, OR established technical excellence with a desire to learn and crush the data engineering world ## Description We are seeking someone with proficiency in Scala and Spark to build and optimize large-scale batch and streaming data pipelines. The Core Data Engineering team sits within the data engineering group and maintains technical involvement across the applications and infrastructure that make up Magnite's data pipelines. Our pipelines collectively handle 400+ billion events per day and generate terabytes of data per hour across both cloud-based and on-prem infrastructure. This data underpins the various business lines - including client reporting, internal data science, account managers, and product + business teams - and as such we need to build systems that remain scalable and efficient at this volume of data while also ensuring data consistency and reliability. We value communication, discussion, and sharing of ideas to come to the best technical solutions to our large-scale data challenges. We are looking for people who want to get things done and value open collaboration (including constructive feedback when brainstorming). The team's mandate is technical development across the three platform-specific data engineering teams in the group. As such they get exposure to all aspects of our data engineering infrastructure and applications including Spark jobs, Java-based real-time event processing, and large-scale data warehousing. In this role you will: * Get to work on handling internet-scale data problems * Help architect and build systems to process our data volume to empower all consumers * Have hands-on involvement across the group's various data pipelines and related systems for data delivery * Be a part of and promote our culture of collaboration and mentorship Typical challenges we face in this role include: * Highly-scalable infrastructure: Our traffic has patterns with peak- and off-peak times and seasonal shifts, and our data infrastructure needs to be able to respond accordingly * Cost optimization: Improving profit margins by lowering the infrastructure cost basis through building efficient systems * Technical architecture considerations given various SLAs for data delivery, * Spark (batch + streaming) data pipelines, currently based both on Databricks and on-prem Spark infrastructure, primarily using Scala * Streaming event processing data pipeline, using Java * Terraform, Docker, Jenkins for CI/CD / infra / application deployment * Airflow for job orchestration * AWS-based cloud infrastructure including RDS, EC2, S3, Kinesis, ECS, Magnite does not accept agency resumes and is not responsible for any fees related to unsolicited resumes. ## Related Videos - [From Messy Queries to Scalable Systems - How Data Engineering actually works](https://www.wearedevelopers.com/videos/100203-from-messy-queries-to-scalable-systems-how-data-engineering-actually-works) - [The Road to MLOps: How Verivox Transitioned to AWS](https://www.wearedevelopers.com/videos/1050-the-road-to-mlops-how-verivox-transitioned-to-aws) - [Docker Compose: Rediscovered](https://www.wearedevelopers.com/videos/1978-docker-compose-rediscovered) - [Enjoying SQL data pipelines with dbt](https://www.wearedevelopers.com/videos/823-enjoying-sql-data-pipelines-with-dbt) - [Modern Data Architectures need Software Engineering](https://www.wearedevelopers.com/videos/1030-modern-data-architectures-need-software-engineering) - [Our GitOps approach for deploying an Identity Provider and an API Gateway in a SaaS company](https://www.wearedevelopers.com/videos/776-our-gitops-approach-for-deploying-an-identity-provider-and-an-api-gateway-in-a-saas-company) ## Related Articles - [How We Built a Worry-Free System That Runs for 10+ Years – And What We’d Do Again](https://www.wearedevelopers.com/magazine/751-how-we-built-a-worry-free-system-that-runs-for-10-years-and-what-we-d-do-again) - [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers) - [Dev Digest 120 - Apple and peers](https://www.wearedevelopers.com/magazine/455-dev-digest-120-apple-and-peers) - [Top Big Data Technologies That You Need to Know](https://www.wearedevelopers.com/magazine/108-top-big-data-technologies-that-you-need-to-know) - [MLops – Deploying, Maintaining And Evolving Machine Learning Models in Production](https://www.wearedevelopers.com/magazine/115-mlops-deploying-maintaining-and-evolving-machine-learning-models-in-production) - [MLOps – What’s the deal behind it?](https://www.wearedevelopers.com/magazine/125-mlops-what-s-the-deal-behind-it)