Principle Software Engineer at Microsoft

Redmond, Washington, United States -

Full Time

Start Date

Immediate

Expiry Date

20 Feb, 26

Salary

0.0

Posted On

22 Nov, 25

Experience

10 year(s) or above

Remote Job

Yes

Telecommute

Yes

Sponsor Visa

Skills

Software Engineering, AI Systems, Performance Engineering, Hardware/Software Co-Design, Performance Optimization, Cross-Functional Collaboration, C, C++, Python, PyTorch, CUDA, Triton, Distributed AI Workloads, Transformer-Based Models, Benchmarking, Workload Simulation

Industry

Software Development

Description

Model Bring-Up & Characterization Lead the bring-up and functional validation of LLMs on custom AI accelerators and GPUs. Develop and maintain detailed performance characterizations across compute, memory, and interconnect domains. Instrument and profile end-to-end training and inference workloads to identify scaling inefficiencies and performance gaps. Hardware/Software/Model Co-Design Partner with silicon and system architects, compiler/runtime engineers, and model researchers to define co-design strategies that maximize efficiency and utilization. Drive studies and experiments across quantization formats, tensor parallelism, activation checkpointing, memory layouts, and communication topologies. Performance Optimization Analyze kernel- and system-level traces to identify limiting factors in compute, memory, and interconnect. Propose and implement optimizations in scheduling, fusion, and data movement to improve throughput and power efficiency. Guide runtime and compiler improvements informed by workload analysis. Cross-Functional Leadership Collaborate with teams across Azure ML, DeepSpeed, and Maia hardware programs to deliver production-grade AI infrastructure. Present architectural findings and recommendations to senior engineering leadership. Mentor and technically guide engineers working in performance, compiler, and system bring-up domains. Bachelor's Degree in Computer Science or related technical field AND 6+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python OR equivalent experience. 10+ years of experience in AI systems, hardware/software co-design, or performance engineering. Deep understanding of AI accelerator and GPU architectures, including compute pipelines, memory hierarchies, and interconnects. Proficiency with PyTorch, CUDA, Triton, or similar frameworks for performance tuning and kernel development. Proven track record of cross-disciplinary collaboration between hardware, software, and ML model teams. Experience profiling and optimizing large-scale distributed AI workloads. Familiarity with DeepSpeed, Megatron-LM, SGLang, or vLLM training and inference pipelines. Deep understanding of transformer-based model architectures and scaling behaviors. Hands-on experience with AI performance modeling, benchmarking, or workload simulation. Demonstrated technical leadership and communication skills in highly collaborative environments.

Responsibilities

Lead the bring-up and functional validation of LLMs on custom AI accelerators and GPUs. Collaborate with teams across Azure ML, DeepSpeed, and Maia hardware programs to deliver production-grade AI infrastructure.