Senior Software Engineer

We're working with
Annapurna Labs (AWS)
on this opportunity.

Senior Software Development Engineer – AI/ML (AWS Neuron, Model Inference)

Cupertino, CA — Remote/Hybrid

$151,300 - $261,500

AWS's Annapurna Labs team builds
Neuron
— the software stack powering Inferentia and Trainium. They're hiring a Senior SDE to work at the bleeding edge of
LLM inference performance
, optimizing models like Llama, DeepSeek and more across custom ML accelerators.

What you'll work on:

• Build and optimize distributed inference for PyTorch on Trainium & Inferentia
• Tune large-scale LLMs for latency, throughput, and hardware efficiency
• Design high-performance kernels, runtime features, and infrastructure
• Profile, debug, and resolve performance bottlenecks across the entire ML stack
• Work directly with customers enabling their models on AWS accelerators
• Partner with compiler, runtime, and hardware teams to shape future architecture

What you'll need:

• Strong Python + systems-level programming skills
• Deep experience with ML frameworks (PyTorch or JAX)
• Background in model optimization, distributed inference, or HPC
• Ability to optimize across the full stack — from model architecture down to hardware

This is an extremely rare chance to work at the intersection of
ML, systems engineering, and custom silicon
, directly influencing the future of AI acceleration inside AWS.

Apply now via Haystack.