Principal Software Engineer, CoreAI

Join Microsoft's AI Core team building high performance runtime systems that power OpenAI chat and multimodal AI models at scale. This role focuses on systems level optimization for largescale LLM inferencing with deep C++ expertise.

• Design and implement high performance microservices and runtime components in C++.
• Optimize AI inferencing systems for latency, throughput, cost, and reliability at large scale.
• Debug and resolve complex production issues related to performance, scaling, and service reliability.
• Collaborate with cross-functional partners to integrate model inference pipelines into scalable infrastructure.
• Contribute to state-of-the-art multimodal inferencing systems supporting text, speech, and vision workloads.
• Drive systems level innovations for realtime and batch inferencing efficiency.
• Participate in code reviews and provide technical mentorship to senior and peer engineers.

Required:

• 6+ years of experience in systems programming with strong expertise in C++.
• Proven experience building, deploying, and operating scalable cloud services.
• Strong debugging skills and experience using performance profiling and diagnostic tools.
• Hands-on experience with distributed systems, Kubernetes, and containerized workloads.
• Experience with largescale LLM inferencing infrastructure, including CUDA.

Other Requirements:

• Ability to meet Microsoft, customer and/or government security screening requirements are required for this role. These requirements include but are not limited to the following specialized security screenings:
• Microsoft Cloud Background Check: This position will be required to pass the Microsoft Cloud background check upon hire/transfer and every two years thereafter., * Experience optimizing AI model inference distributed GPU/CPU stack.
• Exposure to Azure OpenAI or similar largescale AI serving platforms.
• Understanding of service reliability engineering (SRE) principles and operational excellence

Software Engineering IC5 - The typical base pay range for this role across the U.S. is USD $139,900 - $274,800 per year. There is a different range applicable to specific work locations, within the San Francisco Bay area and New York City metropolitan area, and the base pay range for this role in those locations is USD $188,000 - $304,200 per year.

Microsoft is a global technology company headquartered in Redmond, Washington. Our mission is to empower every person and every organization on the planet to achieve more. We develop, license, and support a wide range of software products, services, and devices that help individuals and businesses realize their full potential.

Our flagship products include the Microsoft 365 productivity cloud, Windows operating system, Azure cloud platform, and Dynamics 365 business applications. We are also a leader in areas such as artificial intelligence, cybersecurity, developer tools, and gaming through Xbox and Game Pass.

With operations in more than 190 countries and over 220,000 employees worldwide, Microsoft is committed to responsible innovation, inclusive economic growth, and sustainability. We work closely with governments, industries, and communities to ensure that technology serves the public good and helps address some of the world’s most pressing challenges.

As we celebrate our 50th anniversary in 2025, we continue to look forward—investing in AI, cloud, and quantum computing to shape the future of work, education, and society at large scale.