Senior Researcher - LLM Systems at Microsoft

Bengaluru, karnataka, India -

Full Time

Start Date

Immediate

Expiry Date

16 Apr, 26

Salary

0.0

Posted On

16 Jan, 26

Experience

5 year(s) or above

Remote Job

Yes

Telecommute

Yes

Sponsor Visa

Skills

Distributed Systems, Operating Systems, Large-Scale ML Serving, C++, Python, Queuing Theory, Scheduling Theory, Request Orchestration, Transformer Inference, Caching Techniques, Endpoint Configuration, Performance Modeling, Inference Serving Frameworks, GPU Memory Management, Research Impact, Collaboration

Industry

Software Development

Description

Overview Generative AI is transforming how people create, collaborate, and communicate - redefining productivity across Microsoft 365 and our customers globally. At Microsoft, we run the biggest platform for collaboration and productivity in the world with hundreds of millions of consumer/enterprise users. Tackling AI efficiency challenges is crucial for delivering these experiences at scale. Within our Microsoft wide Systems Innovation initiative, we are working to advance efficiency across AI systems, where we look at novel designs and optimizations across AI stacks: models, AI frameworks, cloud infrastructure, and hardware. We are an Applied Research team driving mid- and long-term product innovations. We closely collaborate with multiple research teams and product groups across the globe who bring a multitude of technical expertise in cloud systems, machine learning and software engineering. We communicate our research both internally and externally through academic publications, open-source releases, blog posts, patents, and industry conferences. Further, we also collaborate with academic and industry partners to advance the state of the art and target material product impact that will affect 100s of millions of customers. We are looking for a Senior Researcher– Systems Researcher to invent, analyze, and productionize the next generation of serving architectures for transformer-based models across cloud and edge. The candidate will focus on algorithmic and systems innovations, including batching, routing, scheduling, caching, deployment safety, and endpoint configuration, that materially improve latency, throughput, cost, and reliability under real-world SLAs for Microsoft Copilots. The ideal candidate brings a strong background in distributed systems, operating systems, and/or large-scale ML serving, plus the ambition to translate research into impact in production environments. This role blends rigorous research (theory + measurement) with hands-on engineering, and includes publishing papers, filing patents, and collaborating across research and product teams to advance the state of the art. Have a look at this link for reading: Efficient AI - Microsoft Research Microsoft’s mission is to empower every person and every organization on the planet to achieve more. As employees we come together with a growth mindset, innovate to empower others, and collaborate to realize our shared goals. Each day we build on our values of respect, integrity, and accountability to create a culture of inclusion where everyone can thrive at work and beyond. Responsibilities - Invent and evaluate algorithms for dynamic batching, routing, and scheduling for transformer inference under multi-tenant SLOs and variable sequence lengths. - Design and implement caching layers (e.g., KV cache paging/offload, prompt/result caching) and memory pressure controls to maximize GPU/accelerator utilization. - Develop endpoint configuration policies (e.g., tensor/pipe parallelism, quantization/precision profiles, speculative decoding, chunked/streaming generation) and safe rollout mechanisms. - Profile and optimize end-to-end serving pipelines: token-level latency, E2E p95/p99, throughput-per-$, cold-start behavior, warm pool strategy, and capacity planning. - Collaborate with model, kernel, and hardware teams to align serving algorithms with attention/KV innovations and accelerator features. - Publish research, file patents, and, where appropriate, contribute to open-source serving frameworks. - Document designs, benchmarks, and operational playbooks; mentor junior researchers/engineers. Qualifications Required Qualifications: Doctorate in relevant field - OR equivalent experience. Demonstrated expertise in queuing/scheduling theory and practical request orchestration under SLO constraints. Proficiency in C++ and Python for high-performance systems; strong code quality and profiling/debugging skills. Proven record of research impact (publications and/or patents) and shipping systems that run at scale. Preferred Qualifications: Deep understanding of transformer inference efficiency techniques (attention, paged KV cache, speculative decoding, LoRA, sequence packing/continuous batching, quantization). Background in cost/performance modeling, autoscaling, and multi-region DR. Hands-on experience with inference serving frameworks (e.g., vLLM, Triton Inference Server, TensorRT-LLM, ONNX Runtime/ORT, Ray Serve, DeepSpeed-MII). Familiarity with GPU/accelerator memory management concepts to co-design cache/throughput policies. Other Requirements: Ability to meet Microsoft, customer and/or government security screening requirements are required for this role. These requirements include but are not limited to the following specialized security screenings: - Microsoft Cloud Background Check: This position will be required to pass the Microsoft Cloud background check upon hire/transfer and every two years thereafterMicrosoft is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to age, ancestry, color, family or medical care leave, gender identity or expression, genetic information, marital status, medical condition, national origin, physical or mental disability, political affiliation, protected veteran status, race, religion, sex (including pregnancy), sexual orientation, or any other characteristic protected by applicable laws, regulations and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or the recruiting process, please send a request via the Accommodation request form. Benefits/perks listed below may vary depending on the nature of your employment with Microsoft and the country where you work. #M365Core #M365Research #Research This position will be open for a minimum of 5 days, with applications accepted on an ongoing basis until the position is filled. Microsoft is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to age, ancestry, citizenship, color, family or medical care leave, gender identity or expression, genetic information, immigration status, marital status, medical condition, national origin, physical or mental disability, political affiliation, protected veteran or military status, race, ethnicity, religion, sex (including pregnancy), sexual orientation, or any other characteristic protected by applicable local laws, regulations and ordinances. If you need assistance with religious accommodations and/or a reasonable accommodation due to a disability during the application process, read more about requesting accommodations.

Responsibilities

The Senior Researcher will invent and evaluate algorithms for dynamic batching, routing, and scheduling for transformer inference, while also designing and implementing caching layers and endpoint configuration policies. They will collaborate with various teams to optimize serving pipelines and contribute to research publications and patents.