How can I get to deep level of understanding of the Java stack?

In my opinion, concurrency, memory management, serialization, reflection are few of the complex topics that most developers don't understand the internal workings of.

Effective Java by Joshua Bloch (an ex-googler) is a great book for advanced topics. See if you find some codelabs to do online for the topics.

One last bit, I would like to understand your motivation behind getting deeper (to avoid X-Y problem https://en.wikipedia.org/wiki/XY_problem). I have built products using java myself, and rarely I had to do super-deep; and when I had to, I just looked up primers quickly. But if you're passionate about programming languages in general, I understand.

+1 to Effective Java, that was honestly the first programming book I read where the concepts felt logical and well-explained.

I recommend doing simple experiments and then documenting what you learn. I did this for Kotlin (e.g., the lazy delegate), and the process of creating the YouTube videos taught me a ton.

This sounds oversimplified, but it's 100% true: Just write a bunch of Java.

Java is one of the most storied programming languages ever created. It has infinite depth and so many versions (and different companies will use different versions). You don't want to fall into the trap of learning optimistically and picking up a bunch of Java knowledge you don't actually need. Always anchor learning against action as I talk about here: "How to avoid going down the rabbit holes when learning new things?"

If you can't get Java learning at work, then build side projects. That is how I learned Java when I was working on weaker engineering teams (i.e. before I went to Meta). As long as you're acutely aware of problems and actively working to tackle them, you will get better. With Java for example, something I learned to really understand is the NPE or NullPointerException. Since I don't want my Android app users to run into crashes, I was forced to learn how to minimize NPEs properly (i.e. not just suppressing them and kicking the root problem down the road).

Read “Effective Java”, as mentioned. Ok, now do it again and write code samples or find ones in your code to fix to adhere. Alright, now codify the rules in your linter and make it part of the coding standard for your team (with whatever exceptions are a must). Read it again.

Read “Java Concurrency in Practice”. Explain to the most junior engineer you know why the idiomatic solution for double-checked locking works (with volatile, with local cached version, etc). Then explain that this is why you use memozing suppliers instead of hand-writing this. Write different concurrent code with native @synchronized critical sections. Do it with methods instead of a brief section. Use ReentrantLocks instead. Now do it without locks using Atomics, Suppliers, etc. Ok, now with chained futures. Alright, do readers/writers. Ok, now modify criteria with deadline for readers to avoid starvation if many writers are queued.

Learn how to create type safe arrays of a type determines via reflection. Now never do that in production code.

Compile a class file. Use javap to disassemble the class file. Make changes, do it again, see what changes. Dex it for android, then use dexdump to evaluate the dexed version. Repeat all of these steps. Tell me why you can’t have more than 65536 methods in one dex file, and which instruction is the limiting factor.

Write a function in C. Have it add or multiply two numbers. Simple. Call it through JNI.

Read the source code for Java and Guava collections.

Oh, now that you’ve done all that: why do you care what’s under the hood? Are you needing to write your own JVM or a new GC, or needing to tune behaviors for a given workload? Do you just want to write idiomatic Java and know how to debug it? As mentioned above, this is X-Y. What do you want to achieve, and why do you think knowing java under the hood will help?

I've been asking this question about Java myself too so thanks for the question and the answers so far!

Instead of answering this for Java specifically, I want to take a step back and share the different levels of depth that a SWE can harness to solve different types of problems. Perhaps it'll help you clarify what level of depth you want to get to.

• Basic. SWEs with a basic level of understanding can write working code because they know the core language fundamentals. This would include syntax, data types, control structures (branching, functions, classes, etc), simple input/output, and etc. One can get surprisingly far with this amount of understanding.
• Intermediate. Engineers at the intermediate level know how to effectively utilize standard libraries, frameworks, and idiomatic language/design patterns in order architect complex systems. This might involve knowing a subset of inner workings for libraries, frameworks, and the language itself. Graduating from this level to the next requires some combination of reading, writing and re-writing production code. This is the level of expertise I'd hire SWEs at if I want to get something done.
• Advanced. Advanced users of languages know most of the internal workings for the language and frameworks they commonly leverage. This allows them to make choices that impact the performance of the system they are building and/or optimizing. It's usually at this stage that engineers develop an opinion for whether a language or framework fits a system's requirements. They can also dive deep to micro-manage memory, manage multi-threaded processes, and make direct contributions to frameworks and libraries. At this level, an engineer has a precise definition of the boundaries of their knowledge, and can very quickly pick up things they don't know. This is the level of expertise I want engineering leaders to have.
• Mastery. These are the engineers that have formed strong enough opinions about different types of languages that they themselves created a language to make their own tradeoffs as opposed to being confined to the languages that currently exist. Mastery is reserved for a small subset of engineers that are language authors and major contributors. There's a theoretical and philosophical element to this that requires a lot of thought and usually comes from a unique passion or new type of system need.

Resources wise, I do think you're right that the ratio of content for deeper levels of expertise is low when compared to more basic levels, but if you look closely on SWE forums and follow strong developers on social media, you'll find repeated references to what I would call classic materials. The most "classic" materials are the documentation for the original language or framework itself. I actually think these resources are almost always underrated since everyone prefers to read the latest and greatest. If you pivot from quick and snappy internet content to studying the classics and exercise learned concepts through writing and re-writing real code, you'll be able to achieve any level of proficiency you want.