> Markdown version of [/jobs/ext/3894469-senior-software-engineer-ai-inference](https://www.wearedevelopers.com/jobs/ext/3894469-senior-software-engineer-ai-inference). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Senior Software Engineer - AI Inference

- **Company:** Bloomberg L.P.
- **Location:** New York, NY, United States
- **Experience:** Expert
- **Salary:** $160,000.0 - $240,000.0
- **Contract:** Permanent contract
- **Skills:** Nvidia CUDA, Software Debugging, Distributed Systems, InfiniBand, Object-Oriented Software Development, Software Engineering, Load Balancing, Cloud Platform System, Pytorch, Caching, Kubernetes, Information Technology, Machine Learning Operations, Model Inference
- **Published:** October 23, 2026
- **Apply:** https://www.manhattanjobs.com/job.asp?id=3466612739&tx=KK5353FFI&pt=1&aff=0B19D771-A501-4A5E-8338-2A822B784D54&utm_source=Job%20Feed&utm_medium=textkernel&utm_campaign=DE&utm_term=0B19D771-A501-4A5E-8338-2A822B784D54

## About the Role

* 5+ years of professional software engineering experience.
 * Experience designing, building, and operating production distributed systems.
 * Strong systems intuition and a track record of debugging and optimizing performance-critical services.
 * Ability to own problems end-to-end and quickly ramp up in unfamiliar technical areas.
 * 4+ years of demonstrated experience working with an object-oriented programming language.
 * A degree in Computer Science, Electrical Engineering, or equivalent practical experience.

We'd love to see:

 * Experience deploying and operating machine learning systems at scale.
 * Experience with inference optimization techniques such as batching, caching, request scheduling, or memory-aware serving.
 * Familiarity with PyTorch and GPU software stacks such as CUDA and NCCL.
 * Exposure to high-performance interconnects and distributed computing technologies such as NVLink, InfiniBand, or MPI.
 * Experience with Kubernetes and cloud-native infrastructure.
 * Experience with load balancing, request routing, or traffic management systems.