Senior AI Hardware Architect

• Lead performance analysis, profiling, benchmarking, and analytical modeling across GPU and AI accelerator architectures, identifying bottlenecks, architectural trade-offs, and optimization opportunities across hardware, software, and system layers.
• Analyze end-to-end AI workloads and serving systems, including model execution, runtime behavior, memory systems, communication collectives, and workload mapping strategies to understand performance, scalability, efficiency, and cost drivers.
• Develop performance, efficiency, and system-level models to evaluate new architectural features, memory and interconnect innovations, collective communication mechanisms, and accelerator design choices, driving perf/W and TCO optimization.
• Correlate silicon measurements, software traces, and kernel execution behavior with architectural models and simulators to validate assumptions, improve model fidelity, and guide future architecture decisions.
• Drive kernel-level, runtime-level, and system-level performance optimizations across AI training and inference workloads, translating workload insights into actionable hardware and software improvements.
• Design and develop data analysis, correlation, visualization, and performance modeling tools that improve debugging efficiency, architectural insight, and decision-making velocity.
• Partner closely with architecture, microarchitecture, compiler, runtime, networking, and systems teams to evaluate design trade-offs and influence product roadmaps through quantitative analysis and technical leadership.
• Present performance findings, architectural recommendations, and design trade-offs to senior technical leadership through architecture reviews, technical reports, and strategic planning discussions.

• Master's Degree in Electrical Engineering, Computer Engineering, Mechanical Engineering, or related field AND 3+ years technical engineering experience OR Bachelor's Degree in Electrical Engineering, Computer Engineering, Mechanical Engineering, or related field AND 5+ years technical engineering experience OR equivalent experience.

Other Requirements:

• Ability to meet Microsoft, customer and/or government security screening requirements are required for this role. These requirements include, but are not limited to, the following specialized security screenings:

• Microsoft Cloud Background Check: This position will be required to pass the Microsoft Cloud background check upon hire/transfer and every two years thereafter., * 4+ years of experience in Computer Architecture, AI Systems, or closely related technical domains.

• MS or PhD in Computer Architecture, Computer Systems, Electrical Engineering, Machine Learning, High-Performance Computing, or a related field.
• Strong understanding of GPU and AI accelerator architectures, including compute pipelines, memory hierarchies, interconnects, collective communication, and parallel execution models.
• Experience with analytical performance modeling, architectural simulation, workload characterization, and silicon correlation for accelerator and system design.
• Expertise in performance profiling, benchmarking, and root-cause analysis using hardware counters, software traces, and workload-level measurements.
• Hands-on experience analyzing and optimizing AI kernels, with the ability to connect kernel behavior to architectural and system-level performance.
• Experience developing performance, efficiency, or TCO models to evaluate architectural features, memory systems, networking, and large-scale AI deployments.
• Strong programming skills in Python and C/C++ for performance analysis, tooling, benchmarking, automation, and data analysis.
• Deep understanding of AI and HPC workloads, including training and inference of large-scale transformer-based models.
• Experience running and analyzing end-to-end AI workloads on production-scale systems, with the ability to diagnose bottlenecks across hardware, runtime, networking, and system layers.
• Familiarity with modern AI frameworks and serving stacks, including PyTorch, vLLM, SGLang, and distributed training or inference frameworks.
• Knowledge of modern AI optimization techniques, including quantization, sparsity, sharding strategies, KV-cache management, Flash Attention, and communication-computation overlap.
• Strong written and verbal communication skills, with experience presenting architectural analyses, performance studies, and design recommendations to technical stakeholders and leadership.

Hardware Engineering IC4 - The typical base pay range for this role across the U.S. is USD $119,800.00 - $234,700.00 per year. There is a different range applicable to specific work locations, within the San Francisco Bay area and New York City metropolitan area, and the base pay range for this role in those locations is USD $160,200.00 - $261,000.00 per year.

Microsoft is a global technology company headquartered in Redmond, Washington. Our mission is to empower every person and every organization on the planet to achieve more. We develop, license, and support a wide range of software products, services, and devices that help individuals and businesses realize their full potential.

Our flagship products include the Microsoft 365 productivity cloud, Windows operating system, Azure cloud platform, and Dynamics 365 business applications. We are also a leader in areas such as artificial intelligence, cybersecurity, developer tools, and gaming through Xbox and Game Pass.

With operations in more than 190 countries and over 220,000 employees worldwide, Microsoft is committed to responsible innovation, inclusive economic growth, and sustainability. We work closely with governments, industries, and communities to ensure that technology serves the public good and helps address some of the world’s most pressing challenges.

As we celebrate our 50th anniversary in 2025, we continue to look forward—investing in AI, cloud, and quantum computing to shape the future of work, education, and society at large scale.