Taro Logo

Lead Software Engineer (Lead ETL PySpark Developer)

Global leader in trusted and transformative intelligence, providing enriched data, insights, analytics and workflow solutions across knowledge, research and innovation.
Data
Staff Software Engineer
Hybrid
5+ years of experience
Enterprise SaaS

Description For Lead Software Engineer (Lead ETL PySpark Developer)

Clarivate is seeking a Lead ETL Developer to join their C3 Data team in Bangalore. This role focuses on the Clarivate Customer Cloud (C3) platform, making rich content available across Clarivate's ecosystem. The position offers an exciting opportunity to work with cutting-edge big data technologies including Python, PySpark, Spark, Databricks, ECS, AWS, and Airflow.

The role requires a seasoned professional with 5+ years of experience in ETL pipeline development and strong expertise in PySpark. As a technical leader, you'll be responsible for designing and implementing data lake platforms, establishing best practices, and mentoring team members. You'll work within a global team of 50+ experts dedicated to the Clarivate Customer Cloud.

The ideal candidate will bring deep technical expertise in Python3, Databricks, Apache Spark, and both SQL and NoSQL databases. Additional experience with AWS services, OpenSearch, Snowflake, and Docker would be advantageous. This position offers the opportunity to work with a leading global company that provides crucial intelligence and analytics solutions across various domains.

Working in a hybrid environment in Bangalore, you'll be part of a company that values innovation and transformation in knowledge and research. Clarivate's mission involves fueling world-changing breakthroughs by harnessing human ingenuity, making this an excellent opportunity for those passionate about making a significant impact through data engineering.

Last updated 2 hours ago

Responsibilities For Lead Software Engineer (Lead ETL PySpark Developer)

  • Providing technical leadership and guidance, evaluating various technologies to balance business needs and cost-effectiveness
  • Collaborating with business and IT teams to design and implement a data lake platform
  • Maintaining the overall solution architecture for the data lake platform
  • Establishing technical best practices for big data management and solutions
  • Ensuring all components of the data lake platform meet SLAs
  • Leading technical investigations, POCs, and hands-on coding
  • Mentoring and coaching team members to enhance their technical and professional skills

Requirements For Lead Software Engineer (Lead ETL PySpark Developer)

Python
  • At least 5+ years of experience in building ETL pipelines and proficiency in PySpark
  • At least 2 years of proficient in Python3 and experience with Databricks and Apache Spark
  • Strong SQL and non-SQL database skills
  • Passionate about code and software architecture
  • Experience with AWS services: EC2, ECS, RDS, S3 (preferred)
  • Experience with OpenSearch, Snowflake (preferred)
  • Experience with Oracle, PostgreSQL (preferred)
  • Experience with Docker (preferred)

Interested in this job?

Jobs Related To Clarivate Lead Software Engineer (Lead ETL PySpark Developer)

Lead Software Engineer- Data Engineer

Lead Data Engineer position at Clarivate, focusing on building big data platforms and healthcare data pipelines using Python, Spark, and AWS technologies.

Staff Data Engineer

Staff Data Engineer position at Zinnia, leading data infrastructure development and optimization using modern tools like Big Query and Apache Airflow in a hybrid work environment.

Senior Analytics Engineer (L5) - Studio Metrics & Strategy DSE

Senior Analytics Engineer position at Netflix, focusing on data-driven insights for Content Operations & Innovation, offering $170k-$720k compensation with comprehensive benefits.

Market Risk Data Scientist / Python Developer - VP

VP-level Market Risk Data Scientist/Python Developer role at Citi London, focusing on risk monitoring and analysis tools development for the Global Rates business.

Data Engineer, Data Quality and Governance(VP)

Senior Data Engineering role at State Street focusing on data quality, governance, and cloud data lake development.