Staff Software Engineer - Distributed Data Systems

At Databricks, we are building the next generation of distributed data systems that power some of the world's most sophisticated data and AI workloads. As a Staff Software Engineer on the Runtime team, you'll be working on cutting-edge distributed systems that process exabytes of data daily across millions of virtual machines. You'll be developing systems that combine the performance of specialized SQL engines with the flexibility to support diverse workloads from ETL to data science.

The role involves working on critical projects like Apache Spark™, Delta Lake, and Delta Pipelines, where you'll build systems that provide ACID transactions, time travel capabilities, and sophisticated orchestration for thousands of data pipelines. You'll be tackling complex challenges in distributed systems, working with cloud storage backends like AWS S3 and Azure Blob Store, and developing high-performance query optimizers.

This is an opportunity to join a team that's pushing the boundaries of data and AI technology while maintaining the resilience and scale necessary for enterprise customers. You'll be working alongside the original creators of Apache Spark™ in a rapidly growing company that serves thousands of organizations, including Fortune 100 companies.

The position offers competitive compensation ($192,000 - $260,000), comprehensive benefits, and the chance to work on technology that's solving real-world problems from security threat detection to cancer drug development. If you're passionate about distributed systems, have strong experience in Java/Scala, and want to work on technology that processes data at massive scale, this role offers the perfect challenge.