Taro Logo

LLM Ops Engineer

Pattern is a technology company specializing in AI and machine learning solutions.
Machine Learning
Mid-Level Software Engineer
Hybrid
3+ years of experience
AI

Description For LLM Ops Engineer

Pattern is seeking an LLM Ops Engineer to join their Engineering team in a hybrid work environment based in Pune, India. This role focuses on building and maintaining AI/LLM workflows in production environments, with a particular emphasis on developing a self-serve LLM Ops platform using Opik by CometML. The ideal candidate will bridge the gap between data science and engineering teams, optimizing AI system performance while ensuring reliability and efficiency.

The position requires a mix of technical expertise in Python programming, AI/ML technologies, and cloud platforms (AWS), along with strong collaborative skills. You'll be responsible for implementing performance metrics, running experiments, and creating evaluation processes that maintain high-quality AI system performance. The role involves working with cutting-edge AI technologies including OpenAI, Anthropic, and Bedrock, while applying statistical concepts and NLP tools to measure and improve system performance.

This is an excellent opportunity for a mid-level engineer with 3-5 years of experience in MLOps or related fields who wants to work at the intersection of AI operations and engineering. You'll have the chance to impact how AI systems are deployed and monitored at scale, while working with cross-functional teams to create robust AI workflows. The hybrid work environment offers flexibility while maintaining collaborative opportunities with colleagues.

The role combines technical depth with strategic thinking, as you'll need to balance system performance with cost optimization while maintaining high accuracy standards. If you're passionate about AI operations, have strong Python skills, and enjoy collaborative problem-solving, this position offers significant growth potential in the rapidly evolving field of AI/ML operations.

Last updated 2 days ago

Responsibilities For LLM Ops Engineer

  • Monitor, evaluate, and optimize AI/LLM workflows in production environments
  • Build out a self-serve LLM Ops platform for engineering and data science departments
  • Integrate LLM Ops platform (Opik by CometML) for existing AI workflows
  • Identify performance metrics and create evaluation processes
  • Create datasets and benchmarks for new AI workflows
  • Run experiments and optimize performance via model changes and prompt adjustments
  • Debug and troubleshoot AI workflow issues
  • Optimize inference costs and latency while maintaining accuracy
  • Develop automations for LLM Ops platform integration

Requirements For LLM Ops Engineer

Python
  • Strong Python programming skills
  • Experience with generative AI models and tools (OpenAI, Anthropic, Bedrock, etc)
  • Knowledge of fundamental statistical concepts and NLP tools
  • Familiarity with AWS
  • Understanding of prompt engineering concepts
  • Strong collaboration and people skills
  • 3-5 years of experience in LLM/AI Ops, MLOps, Data Science, or MLE

Jobs Related To Pattern LLM Ops Engineer