Principal Software Engineer - AI Inference NVIDIA NIM

NVIDIA is the platform upon which every new AI-powered application is built, leading in GPU technology and AI innovation.
$272,000 - $425,500
Machine Learning
Principal Software Engineer
In-Person
5,000+ Employees
12+ years of experience
AI · Enterprise SaaS
This job posting may no longer be active. You may be interested in these related jobs instead:
Senior Software Architect, AI Networking

Senior Software Architect role at NVIDIA focusing on developing groundbreaking AI networking technologies, offering competitive compensation and the opportunity to shape the future of AI infrastructure.

Principal Software Engineer - Enterprise AI Platform

Principal Software Engineer role at NVIDIA focusing on building next-generation AI platforms and products for enterprise efficiency.

Principal Data Scientist - Oracle Labs

Principal Data Scientist position at Oracle Labs in Zurich, focusing on cutting-edge AI, LLMs, and AI Agents for enterprise applications. 10+ years experience required.

Machine Learning Architect - LLM & Generative AI (Image/Video)

Lead Machine Learning Architect position at Apple focusing on LLMs and generative AI, offering competitive compensation and the opportunity to shape next-generation AI technologies.

Machine Learning Compiler Architect

Senior compiler architecture role focused on developing and optimizing machine learning compilers for Apple's Neural Engine hardware accelerator.

Description For Principal Software Engineer - AI Inference NVIDIA NIM

NVIDIA, the leading platform for AI-powered applications, is seeking a Principal Engineer for NVIDIA Inference Microservices (NIMs). This role combines deep technical expertise in AI Inference with containerized software development to create efficient, scalable architectures. The position involves designing inference microservices for various applications, from multi-modal systems to protein folding and weather prediction. As a principal engineer, you'll lead the architecture of NIM software, working with cutting-edge GPU technology and AI inference solutions. The role requires extensive collaboration across teams, technical leadership in microservices development, and expertise in performance optimization. You'll be responsible for defining APIs, improving inference performance, and mentoring other engineers. The ideal candidate brings 12+ years of experience in distributed systems, deep knowledge of AI inference, and strong leadership capabilities. NVIDIA offers a competitive compensation package including a base salary range of $272,000-$425,500, equity, and comprehensive benefits. This is an opportunity to shape the future of AI inference technology while working with industry-leading professionals in a collaborative environment.

Last updated 4 months ago

Responsibilities For Principal Software Engineer - AI Inference NVIDIA NIM

  • Design and architect NIM software to support modularity and high performance using GPU-enhanced technology
  • Build scalable and maintainable microservices, define APIs for different inference use cases
  • Collaborate with multiple AI model teams to build efficient architecture
  • Define metrics and drive improvements based on user feedback
  • Provide technical leadership and mentorship to team members
  • Design inference microservices spanning multi-modal, protein folding, and weather prediction applications

Requirements For Principal Software Engineer - AI Inference NVIDIA NIM

Python
Kubernetes
  • Experience with large-language models and generative AI
  • Deep expertise in microservices, PyTorch, ONNX, REST, gRPC APIs
  • Technical leadership experience in building scalable microservices
  • BS or MS in Computer Science, Computer Engineering or related field
  • 12+ years of experience in building and optimizing distributed services
  • Strong communication and collaboration skills
  • Experience with multiple inference backends

Benefits For Principal Software Engineer - AI Inference NVIDIA NIM

Equity
  • Competitive base salary
  • Equity compensation
  • Comprehensive benefits package

Interested in this job?