> Markdown version of [/jobs/ext/4620947-inference-software-engineer](https://www.wearedevelopers.com/jobs/ext/4620947-inference-software-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Inference Software Engineer

- **Company:** Etched, LLC
- **Location:** San Jose, CA, United States
- **Contract:** Permanent contract
- **Skills:** C++ (Programming Language), Compilers, Software Debugging, Distributed Systems, InfiniBand, Linux Kernel, Software Systems, Graphics Processing Unit (GPU), Pytorch, Low Latency, Transformer Architectures
- **Published:** November 7, 2026
- **Apply:** https://www.juju.com/job/28_25197_5192fd73d606533f40284954daa96c44

## About the Role

* Proficiency in C++ or Rust.
 * Understanding of performance-sensitive or complex distributed software systems like Linux internals, accelerator architectures (e.g. GPUs, TPUs), Compilers, or high-speed interconnects (e.g. NVLink, InfiniBand).
 * Familiarity with PyTorch or JAX.
 * Ported applications to non-standard accelerator hardware or hardware platforms.

Strong candidates may also have experience with (Nice-to-have qualifications)

 * Developed low-latency, high-performance applications using both kernel-level and user-space networking stacks.
 * Deep understanding of distributed systems concepts, algorithms, and challenges, including consensus protocols, consistency models, and communication patterns.
 * Solid grasp of Transformer architectures, particularly Mixture-of-Experts (MoE).
 * Built applications with extensive SIMD (Single Instruction, Multiple Data) optimizations for performance-critical paths.

## Description

* Support porting state-of-the-art models to our architecture. Help build programming abstractions and testing capabilities to rapidly iterate on model porting.
 * Build, enhance, and scale Sohu's runtime, including multi-node inference, intra-node execution, state management, and robust error handling.
 * Optimize routing and communication layers using Sohu's collectives.
 * Utilize performance profiling and debugging tools to identify bottlenecks and correctness issues.