The interview process went smoothly overall. However, during the final technical interview, I was asked to explain a Spark concept to someone with no prior experience using Spark. Given the prompt, I avoided delving too deeply into the technical aspects, focusing instead on a high-level explanation. Unfortunately, I was later informed that my responses were deemed ‘too shallow,’ which led to my rejection.

How to improve the code produced on the Hackerank step. Implicit inner join in Spark. Broadcast join. Which is the most effective way to comment my code? Serialization and deserialization in Python.