Senior Software Engineer, Quantized Training

We are now looking for a Senior Software Engineer for Quantized Training. We are a team committed to developing next-generation quantized training recipes for Hopper and future GPUs. We are seeking software engineers to help rethink and create tailored solutions to accelerate the discovery of new recipes. This is a coding-heavy role focused on building infrastructure, tooling, and visualizations.

The candidate's work directly supports NVIDIA's production SW systems including Megatron-LM and Transformer Engine. The candidate will be part of a core team of engineers and researchers working in lock step to improve quantized training convergence and efficiency.

What you'll be doing:

• Create well-tested SW systems and PoCs in support of quantized training
• Build visualization tools to track and assess the health of model training
• Benchmark internal and external methods for quantized training
• Build an insights platform for tracking model metrics and benchmarks
• Architect CI/CD systems for versioning training recipes
• Participate in code reviews

What we need to see:

• A Masters Degree or PhD or meaningful equivalent experience in Computer Science/Computer Engineering or a related field.
• 5+ years of relevant software development experience.
• Strong software engineering background with a focus on building concise and well-tested code in C++ and Python
• Experience working with ML accelerators and PyTorch or similar frameworks
• Good foundation in ML training and quantization
• Strong written and oral communication skills

Ways to stand out from the crowd:

• Experience with CUDA, performance optimization and debugging
• Proficient in precision and numerics for ML

GPU computing is the most productive and pervasive platform for deep learning and AI. NVIDIA offers highly competitive salaries and a comprehensive benefits package. This opportunity offers you the ability to collaborate with some of the most forward-thinking and hard-working people in the world, shaping the future of AI in a creative and autonomous work environment that encourages innovation.