Taro Logo

Lead Data Engineer – Big Data

S&P Global is the world's foremost provider of credit ratings, benchmarks, analytics and workflow solutions in the global capital, commodity and automotive markets.
$118,094 - $237,100
Data
Staff Software Engineer
In-Person
5,000+ Employees
10+ years of experience
Finance

Description For Lead Data Engineer – Big Data

S&P Global is seeking a Lead Data Engineer to join their Rating Organization's Ingestion Pipelines Engineering Team. This role offers a unique opportunity to shape the future of S&P Ratings' next-generation ingestion pipelines platform. The position requires an experienced professional with 10+ years of data engineering expertise and deep knowledge of cloud technologies.

The role involves designing and implementing sophisticated data pipelines using modern technologies like AWS, Databricks, and various big data tools. You'll be responsible for building robust ELT processes, ensuring data quality, and maintaining high-performance data environments. The position requires expertise in multiple programming languages including Python, Scala, and Java, along with strong experience in cloud architectures and microservices.

As a Lead Data Engineer, you'll work within a collaborative team environment, partnering closely with data science teams and other stakeholders. The role offers competitive compensation ($118,094 - $237,100) and comprehensive benefits including healthcare, retirement planning, and educational support.

S&P Global, as a world-leading provider of credit ratings and analytics, offers an environment where you can make a significant impact on global financial markets. The company's commitment to progress and innovation, combined with its global reach of over 35,000 employees, provides an excellent platform for career growth and development.

The ideal candidate will bring a combination of technical expertise, leadership skills, and business acumen. You'll need to demonstrate strong analytical thinking, excellent communication abilities, and a track record of delivering complex data solutions. This role is perfect for someone who wants to work at the intersection of finance and technology, helping to build and maintain critical data infrastructure that powers global markets.

Working at S&P Global's New York office, you'll be part of a company that values integrity, discovery, and partnership. The role offers the opportunity to work on challenging problems while contributing to the company's mission of providing Essential Intelligence® that helps build a more prosperous future.

Last updated 14 days ago

Responsibilities For Lead Data Engineer – Big Data

  • Design & Build ingestion pipelines using ELT and schema on read in Databricks delta lake
  • Design & Develop Transformations aspects using ELT framework
  • Provide technical expertise in Ratings Data Ingestion pipelines with AWS cloud technologies
  • Build and maintain data environment for speed, accuracy, consistency and up time
  • Work closely with data teams & Data Science team
  • Ensure data governance principles and data quality checks
  • Evaluate new technologies and stay current with emerging trends
  • Ensure compliance with enterprise standards

Requirements For Lead Data Engineer – Big Data

Python
Java
  • BE, MCA or MS degree in Computer Science or Information Technology
  • 10+ years of experience as Data Engineer
  • 5+ years hands-on experience with AWS/Azure cloud technologies
  • 3+ years expertise in data stream processing tools
  • Strong expertise in SDLC cycle
  • Experience with Python, Scala development
  • Experience in microservices and API design
  • Experience with CI/CD tools like Jenkins and Azure DevOps
  • UNIX/Linux environment experience
  • Strong understanding of cloud native architectures
  • Knowledge of Agile approaches
  • Excellent analytical thinking and communication skills

Benefits For Lead Data Engineer – Big Data

Medical Insurance
Dental Insurance
Vision Insurance
401k
Parental Leave
Education Budget
  • Health care coverage
  • Generous time off
  • Continuous learning resources
  • Retirement planning
  • Company-matched student loan contribution
  • Financial wellness programs
  • Family friendly benefits
  • Retail discounts
  • Referral incentive awards

Jobs Related To S&P Global Lead Data Engineer – Big Data