> Markdown version of [/jobs/ext/3948877-senior-sw-engineer-data-optmization](https://www.wearedevelopers.com/jobs/ext/3948877-senior-sw-engineer-data-optmization). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Senior SW Engineer, Data & Optmization

- **Company:** ROBERTS RECRUITING, LLC
- **Location:** Boston, MA, United States
- **Experience:** Expert
- **Contract:** Permanent contract
- **Skills:** Java (Programming Language), Akka (Toolkit), Amazon Web Services, Big Data, Code Review, Computer Programming, Distributed Systems, Elasticsearch, Apache Hadoop, Systems Development Life Cycle, Ansible, Scala (Programming Language), Software Engineering, Apache Solr, Data Streaming, Apache Zookeeper, Datadog, Data Processing, Apache Spark, Electronic Medical Records, HybridCloud, Git, Low Latency, Cassandra, Apache Kafka, Sbt (Software), Functional Programming, Puppet
- **Published:** October 27, 2026
- **Apply:** https://www.careerbuilder.com/job-details/senior-sw-engineer-data-optmization-boston-ma--e6c5d56c-bd35-4776-954e-daa0c83765b1

## About the Role

Software Engineering, Software Engineering, Forecasting, Lambda Architecture, Big Data, Scalable System Development, Distributed Applications, Hybrid Cloud, Data Processing, Amazon Web Services (AWS), * Minimum 5 years software engineering experience

 * Experience with large scale distributed systems and big data processing
 * Familiarity with real time data streaming technologies like Spark, Kafka etc
 * Strong verbal and communication skills
 * BS, MS, or equivalent experience
 * Experience with GIT version control

## Description

We are looking for a Senior Software Engineer with a real passion for building highly-scalable, distributed Big Data applications based on the Lambda Architecture. We want people who take ownership of a project. As part of engineering, you'll work on an agile team designing and building a Forecasting and Optimization platform capable of:

 * Processing 2 Billion Bid requests a day
 * 1.5TB of data processed daily
 * Running in a distributed environment with 500+ servers across a hybrid cloud environment
 * Data processing in AWS
 * 200k QPS at low latency

The optimizer and forecaster take in mathematical models in order to make as well as predict the outcome of "fuzzy" decisions related to inventory management, bidding, and performance of advertising campaigns.
Tools We Like You To Have:

 * Scala or Java
 * SBT, Akka
 * Kafka, Zookeeper, Spark, Hadoop, EMR
 * Solr, Cassandra, ElasticSearch
 * Ansible/Puppet, DataDog

What You'll Be Doing..

 * Hands-on programming capabilities in Java/Scala, Spark, Hadoop etc..
 * You will Work with a team of Data Scientists in revamping our Forecaster and Optimization systems feeding directly into the bidder
 * Design/architect and implement new features and systems with a focus on high quality well tested code
 * Collaborate with other team members, participating in architecture discussions and code reviews.
 * You will provide production support, monitoring the performance of ourForecaster and Optimization systems and troubleshoot/fix issues that arise with deployed software