Pinterest is seeking a Sr. Staff Software Engineer for Batch & Stream Processing to lead the strategy and technical direction of their Spark platform. This role involves working with cutting-edge big data technologies like Spark, Kafka, and Kubernetes at a petabyte scale to help Pinners discover and do what they love.
Key responsibilities include:
- Leading Pinterest's Spark platform strategy and technical direction
- Improving and customizing open-source Spark internals
- Building and scaling batch processing frameworks for petabyte-scale datasets
- Collaborating with internal teams on critical business use cases
- Providing thought leadership on efficient data processing at scale
- Contributing to the team's technical vision and long-term roadmap
The ideal candidate will have:
- 8+ years of industry experience with a proven track record
- 5+ years of experience building and supporting large-scale big data infrastructure
- 2+ years of experience contributing to open-source Spark
- Deep knowledge of big data technologies (Spark, Hadoop, Parquet/ORC, Flink)
- Experience leading cross-team engineering efforts
- Proficiency in Java, Scala, or Python
- Experience with Kubernetes and AWS technologies
Pinterest offers a flexible work model called PinFlex and provides an equitable, inclusive work environment. The position offers a competitive salary range of $175,488 - $360,985 USD, along with equity opportunities. Pinterest is committed to diversity and equal opportunity employment.