Sr. Software Engineer--GPU Inference Optimization

Microsoft empowers every person and organization on the planet to achieve more through technology and innovation.
Backend
Senior Software Engineer
Hybrid
4+ years of experience
AI

Description For Sr. Software Engineer--GPU Inference Optimization

Microsoft's Search Ads Understanding team is seeking a Senior Software Engineer specializing in GPU Inference Optimization to join their innovative team. This role focuses on optimizing GPU inference for large language models (LLMs) and small language models (SLMs) to support various Ads tasks including query rewrite, Ad relevance, and Ad creative generation.

The position offers an exciting opportunity to work with cutting-edge AI technology and hardware, developing fundamental abstractions, programming models, runtimes, libraries, and APIs. The team is responsible for building an intelligent system that matches advertisers' "Ad display" with users' "queries" using advanced AI models and sophisticated engineering systems.

The ideal candidate will have strong expertise in GPU optimization, C/C++ programming, and deep learning frameworks. You'll be working in a fast-paced environment, collaborating with researchers and developers to solve complex technical challenges in building a full end-to-end AI stack.

Microsoft offers a comprehensive benefits package, including industry-leading healthcare, educational resources, parental leave, and investment opportunities. The company maintains a strong commitment to diversity and inclusion, fostering a culture where everyone can thrive and contribute to their mission of empowering every person and organization on the planet.

Working in Beijing, China, with a hybrid work arrangement (up to 50% work from home), you'll be part of a team that drives user satisfaction, advertiser ROI, and Bing revenue through innovative solutions and technical excellence.

Last updated 18 days ago

Responsibilities For Sr. Software Engineer--GPU Inference Optimization

  • Software development in C/C++, Python, and GPU languages such as CUDA, ROCm, or Triton
  • Work with cutting-edge hardware stacks and fast-moving software stack to deliver best-of-class inference and optimal cost
  • Engage with key partners to implement inference and training optimization for state-of-the-art LLMs and other models

Requirements For Sr. Software Engineer--GPU Inference Optimization

Python
  • Bachelor's degree in computer science or related technical field AND 4+ years technical engineering experience
  • 3+ years practical experience working on GPU applications and optimization
  • Experience writing new GPU kernels
  • Cross-team collaboration skills
  • Experience with C/C++, CUDA, or ROCm programming
  • Experience in low-level performance analysis and optimization
  • Proficiency using GPU profiling tools such as NVIDIA Visual Profiler and NVIDIA Nsight Compute

Benefits For Sr. Software Engineer--GPU Inference Optimization

Medical Insurance
Education Budget
Parental Leave
  • Industry leading healthcare
  • Educational resources
  • Discounts on products and services
  • Savings and investments
  • Maternity and paternity leave
  • Generous time away
  • Giving programs
  • Opportunities to network and connect

Interested in this job?

Jobs Related To Microsoft Sr. Software Engineer--GPU Inference Optimization

Senior Software Backend Engineer- Kusto

Senior Backend Engineer role at Microsoft's Kusto team, building highly available control plane services for Azure Data Explorer, with hybrid work model in Israel.

Senior Software Engineer

Senior Software Engineer role at Microsoft's AI Platform team, building cloud services and AI workload management solutions using Kubernetes and modern cloud technologies.

Senior Software Engineer

Senior Software Engineer role at Microsoft AI, building scalable infrastructure for Copilot, Bing, and other AI services, offering competitive compensation and hybrid work.

Senior Software Engineer- Kusto

Senior Software Engineer position at Microsoft working on the Kusto team, building Azure Data Explorer service with focus on big data analytics and cloud computing.

Senior Software Engineer

Senior Software Engineer role at Microsoft Teams, focusing on video rendering and real-time collaboration technologies for Windows and Mac clients.