Very hard interviews.

I am a seasoned big data lead who drives multiple teams, and I still found the level of questions in the interviews pretty tough. I thought they would ask questions around Spark features, joins, optimization, etc.

However, in Tech Round 1, I was asked deep case study-based questions around memory management, multithreading, resource constraints in streaming, pooling, etc. The take-home assignment was also quite heavy, having questions from SQL, pipeline optimization, software engineering, and MLflow. It took the juice out of me, but I found them to be more relevant to the job compared to random LeetCode problems.

Tech Round 2 was almost like a stress interview with two people shooting questions at me about an architecture and debating every answer. I felt almost no one was getting convinced. Later, they told me they were just grilling me to see how I handle tough situations.

The leadership interviews were more focused on the breadth of my experience.

The best thing was that the recruiter was amazing, frequently available to texts and calls to guide me.

Optimized a multi-hour Spark job. I was able to reduce it to 1.5 minutes but couldn't make it faster, however hard I tried.

Created an MLFlow pipeline after doing EDA.

Challenging case study of a messy big data environment involving cloud, batch, streaming, etc.

Trade-offs between SQL-based data engineering versus Spark-based.