Taro Logo

Data Engineer- Lead Data Engineer

India's leading digital payments and financial services platform processing billions of transactions daily.
Data
Staff Software Engineer
In-Person
7+ years of experience
Finance

Job Description

Paytm, India's premier digital payments and financial services platform, is seeking a Lead Data Engineer to join their Data Engineering team. This role is crucial for the company's infrastructure, focusing on designing and maintaining large-scale, real-time data streams that process billions of transactions daily.

The position requires an experienced professional who can architect and implement reliable data streaming solutions where precision and correctness are fundamental requirements. The role directly impacts millions of users across merchant payments, peer-to-peer transfers, bill payments, and financial services, making data accuracy crucial for maintaining customer confidence and operational excellence.

Key technical challenges include maintaining data consistency across distributed systems, implementing real-time validation frameworks, optimizing query performance on large datasets, and ensuring complete data lineage across multiple business domains. The ideal candidate will have 7+ years of experience in data engineering, with expert-level proficiency in Python and Java, and extensive knowledge of big data technologies like Apache Spark, Kafka, and Airflow.

The role offers an opportunity to work with cutting-edge technology at scale, processing terabytes to petabytes of data daily. You'll collaborate with cross-functional teams including data scientists, product managers, and risk engineers to deliver solutions for real-time fraud detection, personalized recommendations, and regulatory compliance reporting.

Working at Paytm means being at the forefront of India's digital payment revolution, with the chance to impact millions of users while building robust, scalable data systems. The company offers competitive compensation, including equity, comprehensive health benefits, professional development opportunities, and flexible working arrangements. This is an ideal position for a senior data engineer looking to make a significant impact in the fintech industry while working with some of the most challenging and interesting data problems at scale.

Last updated a month ago

Responsibilities For Data Engineer- Lead Data Engineer

  • Design and implement reliable, scalable data streams handling high-volume transaction data
  • Build real-time processing systems using modern data engineering frameworks
  • Develop robust data ingestion systems from multiple sources
  • Implement comprehensive data quality frameworks
  • Manage and optimize distributed data processing platforms
  • Design data lake and data warehouse architectures
  • Monitor and optimize streaming performance
  • Implement efficient data storage strategies
  • Mentor junior data engineers
  • Participate in architectural reviews and establish data engineering standards

Requirements For Data Engineer- Lead Data Engineer

Java
Python
Kafka
MongoDB
MySQL
PostgreSQL
Redis
  • Bachelor's or Master's degree in Computer Science, Engineering, or related technical field
  • 7+ years of hands-on data engineering experience
  • Expert-level proficiency in both Python and Java; experience with Scala preferred
  • Experience with Apache Spark, Kafka, Airflow
  • Strong SQL skills and experience with relational and NoSQL databases
  • Experience with AWS (EMR, Glue, Redshift, S3, Lambda) or equivalent
  • Experience with Docker, Kubernetes, Git, GitLab/GitHub
  • Knowledge of data formats: Parquet, Avro, JSON, REST APIs, GraphQL
  • Experience with monitoring tools like Prometheus, Grafana
  • Understanding of dimensional modeling and data vault methodologies

Benefits For Data Engineer- Lead Data Engineer

Medical Insurance
  • Competitive salary and equity compensation
  • Comprehensive health and wellness benefits
  • Professional development opportunities and conference attendance
  • Flexible working arrangements