Taro Logo

Staff Software Engineer - Data Platform

Databricks is the data and AI company that helps organizations unify and democratize data, analytics and AI. More than 10,000 organizations worldwide rely on their Data Intelligence Platform.
Data
Staff Software Engineer
In-Person
5,000+ Employees
12+ years of experience
Enterprise SaaS · AI

Description For Staff Software Engineer - Data Platform

Databricks is seeking a Staff Software Engineer to join their Data Platform team, focusing on building the Data Intelligence Platform. This role is crucial in automating decision-making across the entire company through developing sophisticated data infrastructure. The position involves working with cutting-edge technology to handle massive scale - managing millions of virtual machines and processing exabytes of data daily.

The role combines technical leadership with hands-on development, requiring expertise in building robust data pipelines, metrics stores, and data governance platforms. You'll work directly with various teams including Product, Data Science, and Applied AI, using the latest Databricks products and contributing to their evolution through real-world usage and feedback.

Key responsibilities include designing and maintaining the company-wide metrics store, developing the Data Intelligence Platform, and creating efficient tooling for managing Databricks at scale across multiple clouds. The position requires strong technical expertise in data infrastructure, ETL frameworks, and large-scale distributed systems.

The ideal candidate brings 12+ years of industry experience, with significant technical leadership background in similar large-scale data projects. Experience with workflow orchestration, messaging systems, and building reliable multi-geo data pipelines is essential. This role offers the opportunity to impact how Databricks operates at scale while helping shape the future of their product offerings.

Working at Databricks means joining a company at the forefront of data and AI technology, serving over 10,000 organizations worldwide, including major Fortune 500 companies. The position offers the chance to work on challenging technical problems while contributing to a platform that enables data teams to solve critical world problems.

Last updated 5 hours ago

Responsibilities For Staff Software Engineer - Data Platform

  • Design and run the Databricks metrics store for business units and engineering teams
  • Design and run the cross-company Data Intelligence Platform
  • Develop tooling and infrastructure to manage and run Databricks at scale
  • Design the base ETL framework used by all pipelines
  • Partner with engineering teams for product vision and requirements
  • Build reliable data pipelines and solve data problems
  • Establish conventions and create APIs for telemetry and logging
  • Represent Databricks at conferences & events

Requirements For Staff Software Engineer - Data Platform

Kafka
RabbitMQ
  • 12+ years of industry experience
  • 6+ years of technical leadership on large projects
  • Experience building and operating reliable multi-geo data pipelines at scale
  • Experience with workflow/orchestration frameworks like Airflow and DBT
  • Experience with large-scale messaging systems like Kafka or RabbitMQ
  • Excellent cross-functional and communication skills
  • Passion for data infrastructure

Interested in this job?

Jobs Related To Databricks Staff Software Engineer - Data Platform