I'm a Data Engineer. Within the data engineering realm, there are a lot of tools, just like in the software engineering realm. The modern data stack is pretty popular these days. It includes things like Spark for ETL at scale, Docker for virtualized environments, Airflow for orchestration, dbt (data build tool) for transformations in SQL, Fivetran for automated data connectors, Snowflake for data warehousing, and more.
I'm far from knowing all of these tools well, primarily because I use very few of them in my day job. The main reason I want to change jobs is because of this.
I'm worried I'm caught in a catch-22 situation where I don't know the tools so I can't get jobs that have them, which I guess is similar to the new-grad cold start problem.
My question is, how should I think about learning new tools for job interviews? My current instinct is to learn via failure. That is, I have almost all of the above tools on my resume. If someone asks me about them and I'm not able to give a good answer, I will learn that part about the tool so if I'm in the same situation I can answer properly.
Another approach I can think of is to do Udemy courses of them so I have a deeper understanding of how they work. I've learned to be wary of course not tied to projects, though, so I'm hesitant.
I guess I could do projects to learn more about them, but those take time and right now I'm focused on applying to jobs.
I imagine some answers might focus on what my current problem is: can I get interviews or am I failing interviews? I don't think my issue is with failing interviews right now, and certainly not because of specific knowledge people have called me out for for not knowing these tools. I think my issue is more with sourcing interviews currently.
If there's general advice regarding how to think about prepping for an interview when you only have some of the requirements on the Job Description, would love to hear that too.
My current instinct is to learn via failure. That is, I have almost all of the above tools on my resume. If someone asks me about them and I'm not able to give a good answer, I will learn that part about the tool so if I'm in the same situation I can answer properly.
I love how creative this is, but I feel like it won't go well 😂
The thing is that interaction where you don't have the right answer is like what, 2 minutes? Patching together 2 minute awkward failures in interviews won't actually teach you the stack. You're probably better off just Googling for something like "Top 10 Spark Interview Questions", going through the first 5 articles, memorizing all the Q&A across those, and stitching together that raw memorization as your "learning".
I think the better approach is to figure out the power law and understand which of those technologies you mentioned is most sought-after. I really doubt they all have equal weight. You can do this by:
Now let's talk about actually learning the technology. You can try some collection of these things:
If you can do #1, you probably don't have to do #2 or #3. Here's some related resources to help with #1:
For "Learning from immediate rejections", do you have any specific recommendations for trying to get a signal on what I'm missing from my resume? Companies don't tell you why they're not interviewing you.