Taro Logo

Software Engineer II, AI/ML, Cloud AI Infrastructure

A global technology company that develops AI, search, cloud computing, software and online advertising technologies.
Machine Learning
Mid-Level Software Engineer
In-Person
5,000+ Employees
1+ year of experience
AI · Enterprise SaaS · Cloud

Description For Software Engineer II, AI/ML, Cloud AI Infrastructure

Join Google's ML, Systems, & Cloud AI (MSCA) organization as a Software Engineer II focused on AI/ML infrastructure. In this role, you'll be at the forefront of developing and optimizing machine learning systems that power Google's services and Google Cloud. You'll work with cutting-edge technologies including TPUs, Vertex AI, and Gemini models, while ensuring security, efficiency, and reliability across hyperscale computing systems.

The position offers unique opportunities to impact billions of users through Google's services and enterprise customers through Google Cloud's AI platforms. You'll be responsible for optimizing AI/ML model performance, troubleshooting complex distributed systems, and collaborating with world-class engineers to advance Google's AI infrastructure.

This role is perfect for engineers passionate about machine learning infrastructure who want to work at scale. You'll need strong programming skills, experience with distributed systems, and ideally knowledge of ML frameworks and cloud technologies. The role combines technical depth in AI/ML with practical engineering skills to build robust, scalable systems.

Working at Google means joining a company at the forefront of AI innovation, with access to unique computing resources and the opportunity to solve challenging technical problems. You'll be part of a supportive team culture that values diversity, collaboration, and technical excellence while helping shape the future of AI infrastructure at global scale.

Last updated 10 days ago

Responsibilities For Software Engineer II, AI/ML, Cloud AI Infrastructure

  • Measure and optimize AI/ML model performance on Google Cloud infrastructure
  • Identify and resolve performance issues and collaborate with internal infrastructure teams
  • Develop and deliver high-quality training and demos for customers and internal teams
  • Contribute to ongoing product improvement
  • Conduct performance profiling, debugging, and troubleshooting of training and inference workloads

Requirements For Software Engineer II, AI/ML, Cloud AI Infrastructure

Python
Java
JavaScript
  • Bachelor's degree or equivalent practical experience
  • 1 year of experience with software development in one or more programming languages
  • Experience with Generative AI, Large Language Models (LLM), or Machine Learning infrastructure preferred
  • Experience with distributed computing leveraging GPUs or TPUs preferred
  • Experience with cloud services in Compute, Storage, or Networking preferred
  • Ability to collaborate effectively with cross-functional teams
  • Ability to thrive in a dynamic environment where AI technologies are continuously advancing

Benefits For Software Engineer II, AI/ML, Cloud AI Infrastructure

Medical Insurance
Dental Insurance
Vision Insurance
401k
Parental Leave
  • Comprehensive health benefits
  • Retirement plans
  • Parental leave
  • Professional development opportunities

Interested in this job?

Jobs Related To Google Software Engineer II, AI/ML, Cloud AI Infrastructure