Data Infrastructure Engineer

OpenAI is an AI research and deployment company dedicated to ensuring that general-purpose artificial intelligence benefits all of humanity.
$200,000 - $385,000
Data
Senior Software Engineer
Hybrid
4+ years of experience
AI

Description For Data Infrastructure Engineer

You'll join the team that's behind OpenAI's data infrastructure that powers critical engineering, product, alignment teams that are core to the work we do at OpenAI. The systems we support include our data warehouse, batch compute infrastructure, streaming infrastructure, data orchestration system, data lake, vector databases, critical integrations, and more.

The Applied Data Platform team designs, builds, and operates the foundational data infrastructure that enables products and teams at OpenAI.

You are comfortable with work such as scaling Kubernetes services, OLAP systems, debugging Kafka consumer lag, diagnosing distributed kv store failures, designing a system to retrieve image vectors with low latency.

You are well versed with infrastructure tooling such as Terraform, worked with Kubernetes, and have the SRE skill sets.

This role is based in San Francisco, CA. We use a hybrid work model of 3 days in the office per week and offer relocation assistance to new employees.

In this role, you will:

  • Design, build, and maintain data infrastructure systems such as distributed compute, data orchestration, distributed storage, streaming infrastructure while ensuring scalability, reliability, and security
  • Ensure our data platform can scale reliably to the next several orders of magnitude
  • Accelerate company productivity by empowering your fellow engineers & teammates with excellent data tooling and systems, providing a best in case experience
  • Bring new features and capabilities to the world by partnering with product engineers, trust & safety and other teams to build the technical foundations
  • Like all other teams, we are responsible for the reliability of the systems we build. This includes an on-call rotation to respond to critical incidents as needed

You might thrive in this role if you:

  • Have 4+ years in data infrastructure engineering OR
  • Have 4+ years in infrastructure engineering with a strong interest in data
  • Take pride in building and operating scalable, reliable, secure systems
  • Are comfortable with ambiguity and rapid change
  • Have a voracious and intrinsic desire to learn and fill in missing skills—and an equally strong talent for sharing learnings clearly and concisely with others

Some of the technologies you'll be working with include Apache Spark, Clickhouse, Python, Terraform, Kafka, Azure EventHub, Vector DBs.

OpenAI is committed to providing reasonable accommodations to applicants with disabilities. We are an equal opportunity employer and do not discriminate on the basis of race, religion, national origin, gender, sexual orientation, age, veteran status, disability or any other legally protected status.

Last updated 16 days ago

Responsibilities For Data Infrastructure Engineer

  • Design, build, and maintain data infrastructure systems such as distributed compute, data orchestration, distributed storage, streaming infrastructure
  • Ensure scalability, reliability, and security of data platform
  • Scale data platform reliably to the next several orders of magnitude
  • Accelerate company productivity by empowering fellow engineers & teammates with excellent data tooling and systems
  • Partner with product engineers, trust & safety and other teams to build technical foundations
  • Participate in on-call rotation to respond to critical incidents as needed

Requirements For Data Infrastructure Engineer

Kubernetes
Python
Kafka
  • 4+ years in data infrastructure engineering OR 4+ years in infrastructure engineering with a strong interest in data
  • Comfortable with scaling Kubernetes services, OLAP systems, debugging Kafka consumer lag, diagnosing distributed kv store failures
  • Well-versed with infrastructure tooling such as Terraform
  • Experience with Kubernetes
  • SRE skill sets
  • Ability to take pride in building and operating scalable, reliable, secure systems
  • Comfortable with ambiguity and rapid change
  • Voracious and intrinsic desire to learn and fill in missing skills
  • Strong talent for sharing learnings clearly and concisely with others

Benefits For Data Infrastructure Engineer

Medical Insurance
Dental Insurance
Vision Insurance
Mental Health Assistance
401k
Parental Leave
Education Budget
Equity
Relocation Benefits
  • Medical, dental, and vision insurance for you and your family
  • Mental health and wellness support
  • 401(k) plan with 50% matching
  • Unlimited time off and 13 company holidays per year
  • Paid parental leave (24 weeks) and family-planning support
  • Annual learning & development stipend ($1,500 per year)
  • Equity
  • Relocation assistance

Interested in this job?

Jobs Related To OpenAI Data Infrastructure Engineer

Senior Software Engineer, YouTube Commerce, Applied Data

Senior Software Engineer role at Google, focusing on YouTube Commerce and Applied Data, developing tools and workflows for paid product teams.

Senior Software Engineer, Metrics and Business Intelligence

Senior Software Engineer role at Google for Metrics and Business Intelligence, focusing on data-driven decision making and infrastructure.

Senior Research Scientist

Senior Research Scientist position at Google Research, focusing on AI and machine learning, with competitive salary and benefits.

Senior Data Scientist, Generative AI, Google Cloud

Senior Data Scientist role at Google Cloud, focusing on Generative AI and data-driven product improvement.

Senior Data Scientist, Research

Senior Data Scientist position at Google, focusing on product improvement through advanced analytics and modeling.