Scala Senior Machine Learning Infrastructure Engineer

We’re looking for a Senior Machine Learning Infrastructure Engineer to help build and run Etsy’s ML orchestration platform. Every ML request at Etsy goes through it, including search, recommendations, ads, trust and safety, and generative AI use cases. The platform handles online inference and model evaluation, and it connects product teams to the models, features, and serving infrastructure behind them. This is a high-scale, latency-sensitive, high-availability system, and its reliability directly affects the buyer and seller experience. You’ll join ML Enablement, a platform team with a product mindset.

This is a full time role reporting to the Engineering Manager.

What’s this team like at Etsy?

• We own the single entry point for ML at Etsy. Our design decisions affect every team that ships a model.
• We treat the platform as a product. Our users are Applied Scientists and Software Engineers, and our goal is to make going from experiment to production safe, fast, and self-service.
• We work closely with Applied Scientists, Data Enablement, model serving and LLM hosting teams, and product engineering partners across Etsy.
• We use generative AI and other productivity tools thoughtfully, with a focus on learning and intentional contribution.

What does the day-to-day look like?

• Design, build, and operate the services that route, orchestrate, and execute online ML inference requests at scale, with a focus on latency, throughput, reliability, and observability.
• Build orchestration capabilities such as multi-model request graphs, feature retrieval, fan-out and fan-in, caching, batching, fallbacks, timeouts, and graceful degradation.
• Enable safe model rollout and evaluation: shadow traffic, canaries, A/B and experiment integration, and online and offline evaluation workflows.
• Support both predictive models and generative (open-source LLM) workloads, including the orchestration patterns they need, such as streaming, long-running requests, and GPU-backed serving.
• Build self-service interfaces (APIs, SDKs, configuration, tooling) so scientists and engineers can onboard and ship models without platform expertise.
• Define and uphold SLOs and capacity plans. Lead monitoring, alerting, incident response, and postmortems for a tier-one system.
• Partner with users to gather feedback, drive adoption of paved paths, and plan migrations away from legacy integrations.
• Contribute to technical excellence through design reviews, code reviews, and mentorship.
• Of course, this is just a sample of the kinds of work this role will require! You should assume that your role will encompass other tasks, too, and that your job duties and responsibilities may change from time to time at Etsy’s discretion, or otherwise applicable with local law.

• 5+ years of software engineering experience building and operating distributed, low-latency production services.
• Experience with Scala or another functional programming language and comfort with functional patterns such as immutability, composition, and effect or concurrency libraries.
• A solid understanding of how ML inference works in production: model serving, feature retrieval, batching, latency and throughput trade-offs, and how predictive and transformer-based models are deployed and evaluated.
• Hands-on experience operating services on Kubernetes, including deployments, scaling, resource management, and debugging in production.
• Experience designing service APIs and request-routing or orchestration systems (e.g., gRPC, queues).
• Strong operational skills: metrics, tracing, alerting, on-call ownership, and performance profiling.
• A product mindset and strong communication skills. You enjoy working with internal users to understand their needs.

Nice to have

• Experience with cloud infrastructure, ideally GCP.
• Experience with model serving frameworks and LLM inference infrastructure (e.g., vLLM, Triton, KServe).
• Experience with experimentation platforms, feature stores, or online evaluation systems.
• Experience with GPU capacity management and cost optimization.
• Experience with Scala libraries and frameworks such as Cats Effect, ZIO, Akka/Pekko, or http4s.

In addition to salary, you’ll be eligible for an equity package, an annual performance bonus, and our competitive benefits that support you and your family. Base salary is determined by your location, skills, and experience.

Etsy is the global marketplace for unique and creative goods. We build, power, and evolve the tools and technologies that connect millions of entrepreneurs with millions of buyers around the world. As an Etsy employee, you will tackle unique, meaningful, and large-scale problems alongside passionate coworkers, all the while making a rewarding impact and Keeping Commerce Human.

We believe exceptional companies are built by exceptional teams, and we’re intentional about it. That means hiring great people, setting them up for success from day one, and giving them real reasons to grow their careers here. We invest in development that goes beyond promotions, and we foster the trust and relationships that help people do their best work together.