> Markdown version of [/jobs/ext/2706311-inference-software-engineer](https://www.wearedevelopers.com/jobs/ext/2706311-inference-software-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Inference Software Engineer

- **Company:** Etched, LLC
- **Location:** San Jose, CA, United States
- **Contract:** Permanent contract
- **Skills:** C++ (Programming Language), Compilers, Software Debugging, Distributed Systems, InfiniBand, Linux Kernel, Software Systems, Graphics Processing Unit (GPU), Pytorch, Low Latency
- **Published:** September 4, 2026
- **Apply:** https://startup.jobs/inference-software-engineer-etched-7805026

## About the Role

* Proficiency in C++ or Rust.
 * Understanding of performance-sensitive or complex distributed software systems like Linux internals, accelerator architectures (e.g. GPUs, TPUs), Compilers, or high-speed interconnects (e.g. NVLink, InfiniBand).
 * Familiarity with PyTorch or JAX.
 * Ported applications to non-standard accelerator hardware or hardware platforms.

Strong candidates may also have experience with (Nice-to-have qualifications)

 * Developed low-latency, high-performance applications using both kernel-level and user-space networking stacks.
 * Deep understanding of distributed systems concepts, algorithms, and challenges, including consensus protocols, consistency models, and communication patterns.
 * Solid grasp of Transformer architectures, particularly Mixture-of-Experts (MoE).
 * Built applications with extensive SIMD (Single Instruction, Multiple Data) optimizations for performance-critical paths.

## Description

* Support porting state-of-the-art models to our architecture. Help build programming abstractions and testing capabilities to rapidly iterate on model porting.
 * Build, enhance, and scale Sohu's runtime, including multi-node inference, intra-node execution, state management, and robust error handling.
 * Optimize routing and communication layers using Sohu's collectives.
 * Utilize performance profiling and debugging tools to identify bottlenecks and correctness issues.


## Related Videos

- [The Gashlycrumb Tinies of AI Networking You Must Know (or Languish!)](https://www.wearedevelopers.com/videos/2067-the-gashlycrumb-tinies-of-ai-networking-you-must-know-or-languish)
- [Photonic Computing: Programming a New Class of AI Accelerators (incl. Live Coding)](https://www.wearedevelopers.com/videos/100196-photonic-computing-programming-a-new-class-of-ai-accelerators-incl-live-coding)
- [Coffee with Developers - Stephen Jones - NVIDIA](https://www.wearedevelopers.com/videos/1303-coffee-with-developers-stephen-jones-nvidia)
- [Just-in-time Compilation in JVM](https://www.wearedevelopers.com/videos/240-just-in-time-compilation-in-jvm)
- [Tour de Force: Open-Source LLM Inference Optimization from Simple to Sophisticated](https://www.wearedevelopers.com/videos/100099-tour-de-force-open-source-llm-inference-optimization-from-simple-to-sophisticated)
- [Serverless deployment of (large) NLP models ](https://www.wearedevelopers.com/videos/158-serverless-deployment-of-large-nlp-models)

## Related Articles

- [How We Built a Worry-Free System That Runs for 10+ Years – And What We’d Do Again](https://www.wearedevelopers.com/magazine/751-how-we-built-a-worry-free-system-that-runs-for-10-years-and-what-we-d-do-again)
- [Dev Digest 120 - Apple and peers](https://www.wearedevelopers.com/magazine/455-dev-digest-120-apple-and-peers)
- [Dev Digest 102 - Race conditions](https://www.wearedevelopers.com/magazine/386-dev-digest-102-race-conditions)
- [Dev Digest 121 - AI goes offline](https://www.wearedevelopers.com/magazine/456-dev-digest-121-ai-goes-offline)
- [The Best X (Twitter) Accounts for Developers](https://www.wearedevelopers.com/magazine/294-the-best-x-twitter-accounts-for-developers)
- [Résumé-Driven Development: How IT trends affect the job market for software developers](https://www.wearedevelopers.com/magazine/59-resume-driven-development-how-it-trends-affect-the-job-market-for-software-developers)