Senior Software Engineer, Google Distributed Cloud AI

Experteer Overview In this role you will shape the LLM inference serving layer of Google Distributed Cloud (GDC), driving the design, development, and performance optimization of critical components. You will work with cross-functional teams to ensure scalable model lifecycle, data loading, request routing, and intelligent load balancing. You will advance serving capabilities for advanced LLM techniques and collaborate with core teams on Kubernetes, networking, and hardware acceleration. This role offers impact across the platform, from billing and observability to security and quota management, in a fast-paced, edge-to-hybrid cloud environment. Compensation / Benefits * Lead the design, development, and optimization of LLM inference serving components on GDC * Oversee model lifecycle management, efficient data loading, dynamic request routing, and load balancing * Drive horizontal integration with billing, logging, observability, security, and quota management * Enhance serving capabilities for disaggregated serving, speculative decoding, quantization, and model sharding across distributed hardware * Collaborate with teams building LLM frameworks, Kubernetes/GKE, networking, and hardware acceleration to deliver a cohesive platform Tasks * Bachelor’s degree or equivalent practical experience * 5 years of software development experience * 3 years of testing, maintaining, or launching software products * 1 year of software design and architecture experience * 3 years of experience with large-scale infrastructure, distributed systems, or related compute/storage knowledge * Experience programming in Go for software development including AI/ML applications Key requirements * bonus target * equity * benefits * competitive compensation

Google for disaggregated serving, speculative decoding, quantization, and model sharding across distributed hardware * Collaborate with teams building LLM frameworks, Kubernetes/GKE, networking, and hardware acceleration to deliver a cohesive platform Tasks * Bachelor’s degree or equivalent practical experience * 5 years of software development experience * 3 years of testing, maintaining, or launching software products * 1 year of software design and architecture experience * 3 years of experience with large-scale infrastructure, distributed systems, or related compute/storage knowledge * Experience programming in Go for software development including AI/ML applications Key requirements * bonus target * equity * benefits * competitive compensation