Sr. Staff Software Engineer, Batch & Stream Processing

Pinterest is a visual discovery platform where millions of people find inspiration and plan for what matters most in their lives.
$175,488 - $360,985
Data
Staff Software Engineer
Hybrid
1,000 - 5,000 Employees
8+ years of experience
This job posting may no longer be active. You may be interested in these related jobs instead:
Senior Data Engineer (IC + Team Lead)

Senior Data Engineer role combining technical leadership and team management at Ravio, helping companies optimize compensation through data-driven solutions.

Data Engineer (L5) - Security

Senior Data Engineer position at Netflix focusing on security and fraud detection, building data pipelines and products using big data technologies.

Technical Architect, Data Cloud

Technical Architect position focusing on Salesforce Data Cloud implementation, solution architecture, and customer success

Sr. Manager, People Data Science

Lead Adobe's People Data Science team, developing predictive models and insights to drive strategic HR decisions while managing a team of analysts and data scientists.

Staff Software Engineer, Workflow Orchestration

Staff Software Engineer position at Airbnb focusing on workflow orchestration and data infrastructure development.

Description For Sr. Staff Software Engineer, Batch & Stream Processing

Pinterest is seeking a Sr. Staff Software Engineer for Batch & Stream Processing to lead the strategy and technical direction of their Spark platform. This role involves working with cutting-edge big data technologies like Spark, Kafka, and Kubernetes at a petabyte scale to help Pinners discover and do what they love.

Key responsibilities include:

  • Leading Pinterest's Spark platform strategy and technical direction
  • Improving and customizing open-source Spark internals
  • Building and scaling batch processing frameworks for petabyte-scale datasets
  • Collaborating with internal teams on critical business use cases
  • Providing thought leadership on efficient data processing at scale
  • Contributing to the team's technical vision and long-term roadmap

The ideal candidate will have:

  • 8+ years of industry experience with a proven track record
  • 5+ years of experience building and supporting large-scale big data infrastructure
  • 2+ years of experience contributing to open-source Spark
  • Deep knowledge of big data technologies (Spark, Hadoop, Parquet/ORC, Flink)
  • Experience leading cross-team engineering efforts
  • Proficiency in Java, Scala, or Python
  • Experience with Kubernetes and AWS technologies

Pinterest offers a flexible work model called PinFlex and provides an equitable, inclusive work environment. The position offers a competitive salary range of $175,488 - $360,985 USD, along with equity opportunities. Pinterest is committed to diversity and equal opportunity employment.

Last updated 6 months ago

Responsibilities For Sr. Staff Software Engineer, Batch & Stream Processing

  • Lead the strategy and technical direction of Pinterest's Spark platform
  • Improve and customize the internals of open-source Spark to meet challenges at scale
  • Build and scale batch processing frameworks and infrastructure to process petabytes-scale datasets
  • Work with internal customers on critical business use cases that rely on batch processing
  • Provide thought leadership on data processing and storage at scale
  • Contribute to the team's technical vision and long-term roadmap

Requirements For Sr. Staff Software Engineer, Batch & Stream Processing

Java
Scala
Python
Kafka
Kubernetes
  • 8+ years of industry experience with a proven track record of technical excellence
  • 5+ years of experience building and supporting large-scale big data infrastructure/platform
  • 2+ years of experience contributing to open-source Spark
  • Deep knowledge of big data technologies (e.g. Spark, Hadoop, Parquet/ORC, Flink)
  • Experience in leading cross-team engineering efforts
  • Proficiency in one or more programming languages (Java, Scala, Python)
  • Experience in Kubernetes and AWS technologies

Benefits For Sr. Staff Software Engineer, Batch & Stream Processing

Equity
  • Equity

Interested in this job?