AIML - Sr Engineering Program Manager, Evaluation- Special Projects

We are seeking a highly skilled Engineering Program Manager (EPM) to lead user studies, data collection efforts, and evaluations for our AI models across multiple modalities, including text, images, and audio. The successful candidate will play a critical role in designing, implementing, and refining methodologies to assess LLM performance, while also driving user studies and large-scale data collection initiatives to support model development and improvement.

Key responsibilities include:

• Develop, implement, and refine robust evaluation frameworks that align AI model performance with user needs and goals.
• Lead and manage user research programs, leveraging diverse methodologies to gather actionable user insights.
• Design and execute large-scale data collection efforts, ensuring the acquisition of high-quality, diverse datasets critical for AI model development.
• Partner with key stakeholders, including designers, data scientists, and engineers to ensure that evaluation and data collection efforts are well-integrated into the overall AI model development lifecycle.
• Stay at the forefront of AI evaluation and data collection techniques, continuously innovating and refining methodologies to maintain state-of-the-art model performance and user experience.

This role offers the opportunity to have a significant impact on the quality and effectiveness of our AI models by leading efforts in both evaluation and user research, ensuring we meet the highest standards of performance and user satisfaction.