> Markdown version of [/jobs/ext/1393067-ai-runtime-low-level-software-engineer](https://www.wearedevelopers.com/jobs/ext/1393067-ai-runtime-low-level-software-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # AI Runtime / Low-Level Software Engineer - **Company:** Kalray - **Location:** Montbonnot-Saint-Martin, France - **Experience:** Expert - **Contract:** Permanent contract - **Skills:** Application Programming Interfaces (APIs), Artificial Intelligence, C++ (Programming Language), Profiling, Software Quality, Computer Engineering, Concurrent Computing, Continuous Integration, Software Debugging, Linux, Embedded Software, Firmware, Linux Kernel, Linux-Powered Devices, PCI Express, Performance Tuning, Quick EMUlator (QEMU), Reduced Instruction Set Computing, Data Streaming, System Programming, Large Language Models, Information Technology, Bare Metal, Codebase, Hardware Acceleration - **Published:** July 23, 2026 - **Apply:** https://www.kalrayinc.com/jobs/ai-runtime-low-level-software-engineer/ ## About the Role * Strong experience in C and/or C++ systems programming on Linux, within large and complex codebases. * Strong knowledge of parallel and concurrent programming, including threads, memory movement, profiling and performance optimization. * Experience with embedded software, firmware or bare-metal development, ideally with technologies such as OpenAMP, remoteproc, rpmsg, virtio and shared-memory communication models. * Good understanding of Linux internals, including concepts such as mmap, sysfs and the Linux device model, as well as PCIe fundamentals (BARs, DMA). * Experience with RISC-V (or similar ISA), cross-compilation and low-level debugging across the hardware/software boundary. Nice to have: * Experience with the GenAI / LLM inference ecosystem, such as vLLM, or AI compilation technologies such as MLIR/LLVM. * Experience developing kernel drivers (PCIe, remoteproc) and working with QEMU or hardware emulation environments. * Experience with CI/CD systems targeting hardware or emulation platforms. * Experience with high-performance data-plane development, including DMA and zero-copy techniques, * MSc or Engineering degree (BAC+5) in Computer Science, Embedded Systems, Computer Engineering or related field. * 5+ years of experience in low-level software development, embedded systems, runtime development or systems programming. * Strong ownership mindset and autonomy with the ability to drive a complex software component end-to-end and make sound technical decisions. * Strong systems-thinking ability, understanding the complete chain from AI serving and compiler layers down to runtime, firmware and hardware. * Performance-oriented mindset with intuition for identifying where cycles, memory copies and bottlenecks impact execution. * Rigorous approach to software quality, automation, reproducibility and reliable engineering practices. * Strong collaboration and communication skills, with the ability to work closely with multidisciplinary teams. * Comfortable working in an international, fast-evolving startup environment. ## Description As an AI Runtime / Low-Level Software Engineer, you will own the offload runtime layer that enables inference workloads to run efficiently on our custom RISC-V AI accelerator from an x86 host. You will join our AI & Compute team, which is building a full-stack GenAI inference platform, from serving to silicon: LLM Serving * AI Compilation * Runtime / Offload * Optimized AI Kernel Libraries You will develop the Low-Level software stack responsible for device lifecycle management, scheduling and workload dispatch, high-performance host-to-device communication, and the runtime APIs exposed to compiler and serving layers. You will build this stack end-to-end on an open software foundation and collaborate closely with hardware, compiler and AI serving teams to drive the solution from emulation to production silicon. Your main responsibilities will include: Leading and contributing to: * Develop and optimize the runtime responsible for executing offloaded inference workloads, from the host offload API to the accelerator firmware. * Own the performance-critical path, including scheduling, dispatch, memory movement and data flow between the host and the accelerator. * Profile, analyze and optimize runtime performance by identifying bottlenecks related to latency, memory transfers and execution efficiency. * Design and maintain clean runtime APIs exposed to AI compiler and serving layers, enabling efficient workload execution on the accelerator. * Integrate and validate the real PCIe communication path while maintaining emulation environments as CI-tested development platforms. * Develop engineering foundations including CI/CD pipelines, automated smoke tests on emulated hardware and strong quality standards. * Collaborate with AI Inference, hardware and architecture teams to influence technical decisions and improve real-world AI performance. * Contribute to the continuous improvement and reliability of the complete AI acceleration software stack. ## Related Videos - [Docker network without Docker](https://www.wearedevelopers.com/videos/1418-docker-network-without-docker) - [Getting to Know Your Legacy (System) with AI-Driven Software Archeology](https://www.wearedevelopers.com/videos/1437-getting-to-know-your-legacy-system-with-ai-driven-software-archeology) - [Playing Pong on a shoulder press machine](https://www.wearedevelopers.com/videos/100140-playing-pong-on-a-shoulder-press-machine) - [Developing an AI.SDK](https://www.wearedevelopers.com/videos/198-developing-an-ai-sdk) - [Agent Smith Gets Hardware: Autonomous IoT Hacking From Debug Port to Cloud API](https://www.wearedevelopers.com/videos/100258-agent-smith-gets-hardware-autonomous-iot-hacking-from-debug-port-to-cloud-api) - [Why your codebase lies to AI?](https://www.wearedevelopers.com/videos/100281-why-your-codebase-lies-to-ai) ## Related Articles - [How to Become an AI Engineer](https://www.wearedevelopers.com/magazine/331-how-to-become-an-ai-engineer) - [MLOps And AI Driven Development](https://www.wearedevelopers.com/magazine/82-mlops-and-ai-driven-development) - [What Are Large Language Models?](https://www.wearedevelopers.com/magazine/304-what-are-large-language-models) - [Navigating the AI Shift](https://www.wearedevelopers.com/magazine/629-navigating-the-ai-shift) - [Stephan Gillich - Bringing AI Everywhere](https://www.wearedevelopers.com/magazine/489-stephan-gillich-bringing-ai-everywhere) - [Dev Digest 121 - AI goes offline](https://www.wearedevelopers.com/magazine/456-dev-digest-121-ai-goes-offline)