Senior Software Development Engineer, AI/ML, AWS Neuron, Model Inference

The Annapurna Labs team at Amazon Web Services (AWS) builds AWS Neuron, the software development kit used to accelerate deep learning and GenAI workloads on Amazon's custom machine learning accelerators, Inferentia and Trainium. This role is part of the Inference Enablement and Acceleration team, which is at the forefront of running a wide range of models and supporting novel architecture alongside maximizing their performance for AWS's custom ML accelerators.

The position involves working across multiple technology layers - from frameworks and kernels to compiler, runtime, and collectives. You'll be responsible for development, enablement, and performance tuning of various LLM model families, including massive scale large language models like the Llama family, DeepSeek, and beyond.

As a Senior Software Development Engineer, you'll architect and implement business-critical features, mentor experienced engineers, and work in a unique learning culture where innovation and experimentation are encouraged. The role combines deep hardware knowledge with ML expertise to push the boundaries of AI acceleration technology.

Key responsibilities include building distributed inference support for PyTorch in the Neuron SDK, tuning models for highest performance, and maximizing efficiency on AWS Trainium and Inferentia silicon and servers. You'll collaborate with cross-functional teams, work directly with customers, and contribute to future architecture designs.

The team operates in a startup-like environment where you'll always work on the most important initiatives. We emphasize collaboration, technical ownership, and continuous learning. Our inclusive culture supports knowledge-sharing and mentorship, with opportunities for career growth through increasingly complex technical challenges.

The position offers competitive compensation ranging from $129,300 to $223,600 per year based on geographic location, plus equity, sign-on payments, and comprehensive benefits. Join us to solve some of the most interesting and impactful infrastructure challenges in AI/ML today.