> Markdown version of [/jobs/ext/4621227-sr-principal-software-engineer](https://www.wearedevelopers.com/jobs/ext/4621227-sr-principal-software-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Sr. Principal Software Engineer

- **Company:** Cerence Inc.
- **Location:** Burlington, MA, United States
- **Experience:** Expert
- **Contract:** Permanent contract
- **Skills:** Nvidia CUDA, Data Centers, JavaScript Pagination Plugin, Graphics Processing Unit (GPU), Large Language Models, Discretization, Low Latency, Reasoning Engines, Machine Learning Operations, TensorRT, Mixed Precision, Model Inference
- **Published:** November 7, 2026
- **Apply:** https://www.juju.com/job/28_25197_9842c8593c778fc1cbd6ffda07c392bb

## About the Role

footprint, and throughput***KV Cache Optimization**** Optimize key-value cache performance through:* Paging* Prefix caching* Cache-aware memory layout design* Reduce memory pressure while sustaining high throughput***Latency & Throughput Optimisation**** Design and tune:* Batching strategies* Continuous batching* Speculative decoding* Optimize tail latency and tokens/sec under real production traffic patterns**What Success Looks Like*** Models deploy efficiently on edge and embedded devices, not just servers* Tokens/sec significantly outperform baseline implementations* End-to-end latency is minimized and predictable* Inference cost per request is materially reduced* The company is no longer dependent on partners for inference optimization**Required Experience & Skills*****Strongly Required**** Proven experience optimizing **ML inference performance in production*** Deep understanding of **GPU architecture and memory hierarchies*** Hands-on experience with CUDA and low-level performance

## Description

Principal Software EngineerSkip to main contentWe use cookies to understand how you use our site and to improve your experience.
This includes personalizing content and advertising., We're looking for an exceptional Senior Principal Software Engineer who is ready to drive the future of mobility with us! **Job Description:****What You Will Work On*** Optimize and deploy **high**-**performance LLM inference pipelines*** Own inference runtimes across **data center, edge, and embedded platforms*** Push model performance through quantization, kernel fusion, and cache optimization* Drive latency and throughput improvements that directly impact production products* Enable efficient, reliable deployment without external vendor dependency**Core Responsibilities***Inference Engines & Runtime** Build deep expertise and ownership of:* vLLM* TensorRT-LLM* llama.cpp* QAIRT* Extend and tune inference engines using **custom CUDA kernels*** Adapt runtimes for constrained and embedded deployment environments***Quantization & Numerical Optimisation**** Implement and evaluate quantisation strategies:* INT8, INT4, FP4, FP8, mixed precision* AWQ* GPTQ* Balance accuracy, latency, memory