Manager, LLM Accuracy Evaluation
Description For Manager, LLM Accuracy Evaluation
NVIDIA is seeking a visionary Manager to lead their LLM Accuracy Evaluation team, focusing on pioneering new methodologies for evaluating cutting-edge deep learning models, including LLMs, RAG, agents, and vision models. This role offers a unique opportunity to shape the future of AI at one of the industry's leading companies. The position involves working with flagship models like Gemma and Llama-3, optimizing them as NVIDIA Inference Microservices (NIM). The role combines technical leadership with team management, requiring both deep AI expertise and strong leadership skills. The successful candidate will work with state-of-the-art GPU clusters and have early access to unreleased hardware, directly influencing NVIDIA's roadmap and the broader AI landscape. The position requires a blend of technical expertise in AI/ML, leadership experience, and strong communication skills, with opportunities to work on groundbreaking projects that will shape the future of AI technology. The role offers exposure to cutting-edge AI developments and the chance to work with both internal teams and external partners in delivering high-impact solutions.
Responsibilities For Manager, LLM Accuracy Evaluation
- Lead and mentor a team of highly skilled engineers
- Drive accuracy evaluation of flagship models as optimized NVIDIA Inference Microservices (NIM)
- Collaborate with internal customers across the organization
- Supervise development and implementation of new methodologies for evaluating AI models
- Build a culture of continuous improvement
Requirements For Manager, LLM Accuracy Evaluation
- BS, MS, or PhD in Computer Science, AI, Applied Math, or related field
- 8+ years of industry experience, including 3+ years in leadership/management role
- Proven experience leading engineering teams
- Strong understanding of AI technologies, including NLP, LLMs, and deep learning
- Exceptional communication and interpersonal skills
