> Markdown version of [/jobs/ext/4650180-software-engineer-ai-systems-performance](https://www.wearedevelopers.com/jobs/ext/4650180-software-engineer-ai-systems-performance). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Software Engineer, AI Systems & Performance

- **Company:** COTIDAL LLC
- **Location:** Palo Alto, CA, United States
- **Contract:** Permanent contract
- **Skills:** Compilers, Nvidia CUDA, Shard (Database Architecture), Distributed Computing Environment, Linux Kernel, Regression Testing, Inference Optimization, Tensorflow, Reinforcement Learning, Graphics Processing Unit (GPU), Pytorch, Parallel Computation, Build Management, AWS Trainium, SGLang
- **Published:** November 9, 2026
- **Apply:** https://www.juju.com/job/28_25197_8d7bc088f95d5ea64185da55850d3f4a

## About the Role

* ML frameworks and compilers such as PyTorch, JAX, or XLA, and hands-on work with GPUs, TPUs, Trainium, or other accelerator architectures.
 * RL and post-training systems, including rollout generation, weight synchronization, and frameworks such as verl.
 * Distributed training and execution, including sharding, model parallelism, collective communication, and overlapping computation with communication.
 * Kernel development or compiler optimization using Pallas, Triton, CUDA, or similar tools.
 * Inference optimization with vLLM, SGLang, or similar systems, including batching, KV caches, and quantization.

## Description

You'll define and build Cotidal's workload software stack for training, reinforcement learning, and inference. You'll make architectural decisions and carry systems from their first implementation through production scale as models, workloads, and accelerator architectures evolve.

Working directly with customers and teammates, you'll identify the problems that matter, set technical priorities, and decide how to solve them. We're looking for engineers with depth in one area, the judgment to navigate open-ended problems, and the curiosity to work across the stack.

What You Will Do

Model & Framework Integration: Design and build the framework integrations and distributed execution systems that bring training, RL, and inference workloads to Cotidal's accelerators. Extend ML frameworks and libraries, resolve compatibility and compilation issues, and shape support for new model architectures and workflows.

Performance Engineering: Set performance goals, build representative benchmarks, and identify the changes that will have the greatest impact on real workloads. Improve model code, compilers, runtimes, and kernels to increase training efficiency, shorten RL iteration time, and improve inference throughput and latency while preserving correctness and model quality.

Production Reliability: Own the reliability of the systems you build as workloads scale. Define validation and regression tests, observability, checkpointing, and recovery mechanisms. Turn production experience into reusable capabilities and upstream improvements that make new workloads easier to support and existing ones easier to operate.

What We Are Looking For

 * You've built software for training, inference, or accelerated computing and stayed close to how it behaves in practice-through performance limits, failures, and changing requirements.
 * You follow a problem wherever it leads: from model code and distributed execution into the runtime or hardware, then back to whether the improvement helps the overall workload.
 * You have strong software engineering fundamentals and understand how compute, memory, and communication shape system performance. You're comfortable reading unfamiliar code and using experiments to test your understanding.
 * You turn open-ended goals into a technical plan, decide what to prioritize, and carry the work through to a reliable system. You take responsibility for the outcome and adjust your approach as you learn.
 * You work directly with users and teammates, make your reasoning clear, and seek out the expertise needed to move a problem forward. You have opinions about how systems should be built and have changed your mind when the evidence called for it.