Principal AI Engineer at Microsoft

Redmond, Washington, United States -

Full Time

Start Date

Immediate

Expiry Date

03 Mar, 26

Salary

296400.0

Posted On

03 Dec, 25

Experience

10 year(s) or above

Remote Job

Yes

Telecommute

Yes

Sponsor Visa

Skills

AI Engineering, Autonomous Agents, LLM Systems, Cloud Platforms, Python, C#, Java, Microservices, Containers, CI/CD, Technical Leadership, Mentorship, Evaluation Pipelines, Communication Skills, Architecture Design, Cross-Functional Collaboration

Industry

Software Development

Description

- Design and ship autonomous agents that can reason, plan, and execute multi-step workflows, integrating with MCP tools, APIs, action registry, and internal data sources. - Build and evolve the evaluations platform (offline evals, golden datasets, LLM-as-judge/auto-raters) and integrate it into CI/CD so every model, prompt, and agent change is tested before and after release. - Partner cross-functionally with product, design, research, and data science to establish observability and quality guardrails for LLM and agent workloads, including telemetry, tracing, dashboards, and metrics for task success, safety, cost, and latency, and use them to support A/B tests and phased rollouts. - Provide technical leadership and mentorship, setting standards for design, coding, eval practices, and Responsible AI, while driving multi-team initiatives that raise the bar for reliability and user impact. - Bachelor's Degree in Computer Science or related technical field AND 8+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python OR equivalent experience. - Master's Degree in Computer Science or related technical field AND 12+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python OR Bachelor's Degree in Computer Science or related technical field AND 15+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python OR equivalent experience. - Demonstrated experience shipping LLM-powered features or agentic systems in production (e.g., copilots, assistants, workflow agents, tool-calling systems). - Experience with at least one major cloud platform (Azure, AWS, or GCP) and modern infrastructure practices (microservices, containers, orchestration, CI/CD). - Strong proficiency in Python (and/or another backend language such as C# or Java) and experience building production services or platforms. - Proven ability to define architectures, make trade-offs, and lead multi-team projects from design through launch. - Background in evaluation of ML/LLM systems: designing test sets, rubrics, human and auto-rater evaluation pipelines. - Excellent communication skills and the ability to explain complex trade-offs to both technical and non-technical stakeholders. azdat azuredata Software Engineering IC6 - The typical base pay range for this role across the U.S. is USD $163,000 - $296,400 per year. There is a different range applicable to specific work locations, within the San Francisco Bay area and New York City metropolitan area, and the base pay range for this role in those locations is USD $220,800 - $331,200 per year. Certain roles may be eligible for benefits and other compensation.

Responsibilities

Design and ship autonomous agents that can reason, plan, and execute multi-step workflows. Build and evolve the evaluations platform and integrate it into CI/CD for testing model changes.