Founding Backend Engineer

LiteLLM is an open-source LLM Gateway with 18K+ stars on GitHub and trusted by companies like Rocket Money, Samsara, Lemonade, and Adobe. We provide an open source Python SDK and Python FastAPI Server that allows calling 100+ LLM APIs (Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic) in the OpenAI format.

We have raised a $1.6M seed round from Y Combinator, Gravity Fund and Pioneer Fund. As a Founding Backend Engineer, you'll be responsible for ensuring LiteLLM unifies the format for calling LLM APIs in the OpenAI spec. This involves writing transformations to convert API requests from OpenAI spec to various LLM provider formats.

Key projects you'll work on:

• Migrating key systems from httpx to aiohttp for 10x higher throughput
• Adding support for Anthropic and Bedrock Anthropic 'thinking' parameter
• Handling provider-specific quirks like OpenAI o-1 streaming limitations
• Scaling aggregate spend computation for 1M+ logs
• Implementing cost tracking and logging for Anthropic API

Our tech stack includes Python, FastAPI, JavaScript/TypeScript, Redis, Postgres, S3, GCS Storage, Datadog, and Slack API. You'll work directly with our CEO and CTO in a small, fast-paced team.

We're looking for someone who:

• Has 1-2 years of backend/full-stack development experience with production systems
• Is passionate about open source and engaging with users
• Has experience maintaining high-performance infrastructure
• Thrives in a small, accountable team environment
• Is eager to shape the infrastructure of a growing developer tool

The interview process includes:

1. Solving a GitHub issue to demonstrate familiarity with LiteLLM
2. A 2-week work sprint to experience real collaboration
3. Final decision and offer

Join us in San Francisco to help build the future of LLM API standardization and integration.