Principal Software Engineer - CoreAI

Join our team within CoreAI, where we are building the AI data-plane that powers all LLM inferencing workloads across Microsoft and Azure customers—from cutting-edge startups to Fortune 500 enterprises. Our converged AI fabric delivers inference capabilities for all LLMs in Microsoft catalog, including OpenAI, Mistral, Cohere, Llama, and more.

As a Principal Software Engineer - CoreAI, you will shape the future of one of the largest and fastest-growing services in Azure, foundational to Microsoft's AI strategy. Our mission is to serve models at scale—reliably, efficiently, and with ultra-low latency—enabling a rich set of AI-powered product experiences.

This is a rapidly evolving space with immense opportunities to learn, innovate, and drive industry-wide impact. You will be responsible for:

• Designing, implementing and delivering AI services to support product offerings for large-scale LLM serving
• Collaborating closely with product management and partner teams to align technical direction with business goals
• Coaching and growing team members with design, code reviews, and mentorship serving as a role model
• Engaging with customers to gather feedback and resolve complex issues
• Innovating on technical solutions and patterns to improve availability, reliability, efficiency, observability, and performance

The role offers competitive compensation with a base pay range of USD $139,900 - $274,800 per year (higher in SF Bay Area and NYC: USD $188,000 - $304,200). You'll be part of a world-class team committed to technical excellence and career growth, working on critical Azure services that underpin Microsoft's AI vision.

You'll have the opportunity to drive innovation on product shape and offers, defining how developers leverage cutting-edge AI technologies to build world-class applications. The position involves working in a fast-paced environment where ideas to production happens in weeks, making real impact on Microsoft's AI infrastructure.