Software Engineering LMTS- Big Data Engineer
Job Description
Join Salesforce's Unified Intelligence Platform (UIP) team to revolutionize how we understand and optimize services through data. As a Lead Big Data Engineer, you'll work on a cutting-edge data platform processing petabytes of data daily. The role combines modern cloud technologies with advanced data lake engines, incorporating tools like Spark, Trino, Airflow, and Kubernetes. You'll lead architecture and development of mission-critical data services, drive self-service pipelines, and ensure robust data quality and governance. The position offers the opportunity to work with diverse technologies across multiple cloud environments, solving complex distributed systems challenges. You'll collaborate with product teams, implement advanced data frameworks, and champion service ownership principles. This role is perfect for those passionate about big data, distributed systems, and building scalable solutions that power AI and machine learning use cases. The team culture emphasizes collaboration, continuous learning, and engineering excellence, making it an ideal environment for growth and innovation.
Responsibilities For Software Engineering LMTS- Big Data Engineer
- Lead architecture, design, development, and support of mission-critical data and platform services
- Drive multiple self-service, metadata-driven data pipelines and applications
- Collaborate with product management and client teams
- Architect robust data solutions for ingestion, processing, quality, and discovery
- Promote service ownership model with extensive telemetry
- Develop data frameworks for consistency and best practices
- Implement advanced data quality services
- Build Salesforce-integrated applications
- Establish and refine CI/CD processes
- Oversee UIP technology stack including Airflow, Spark, Trino, Iceberg, and Kubernetes
- Collaborate with third-party vendors
- Architect data pipelines for multi-cloud environments
Requirements For Software Engineering LMTS- Big Data Engineer
- Strong foundation in software engineering
- Experience with big data challenges in distributed systems
- Ability to work across multiple roles (Platform Engineer, Data engineer, Backend engineer, DevOps)
- Excellent collaboration and communication skills
- Experience with data platforms and cloud technologies
- Knowledge of modern data lake engines like Spark and Trino
- Familiarity with tools like Airflow, DBT, Jupyter Notebooks, Sagemaker, Iceberg
- Experience with multi-cloud environments (AWS, GCP)
