How do I get better at building Large scale systems without working directly on such projects?

This is an interesting question, and it's... tricky. I empathize with the working with individual users aspect as well - This is a big reason why working styles like consulting lead to growth walls. Since you are dealing with individual customers, it's case-by-case (so less fundamentals), and you don't get to work with depth and scale.

Here are some thoughts from me on options:

1. Consume resources like books and articles
2. Side projects
3. Open source
4. Expanding scope of existing projects
5. Switch teams to one that does deal more with scale

Consuming Resources

• These can help, but at the end of the day, software is about doing.
• You mentioned that you want to be "really good" at these things, and I think by definition, if you haven't built something in a space, you can't be good at it.

Side Projects

• I'm biased as I've done these a lot, but I do legitimately think these can help here.
• However, side projects, almost by definition, won't have scale as they're likely going to small (unless you get really lucky and build a side project that gets 1 million+ users). This makes it harder to get practical experience in the fields you mentioned.
• That being said, I do see some clear examples of working with the example topics. It's going to be a more "local" instance of doing these (so it's not really what you're looking for), but I feel like it helps. Here are the examples, tied to mobile apps as mobile is effectively built into my DNA:
  • Parallel processing & concurrency: If you build a music player app, there will be some flow to import many mp3 files into your library. You can use multi-threading to process these files in parallel as opposed to having just 1 thread bring in all 5,000 songs or whatever. This is a nice challenge as you figure out how to write properly to the DB without corrupting/overwriting data, dividing up the work cleanly without duplication, etc.
  • Caching: If you have an app that makes API calls, you can cache things. There are libraries to do this (the Taro app uses built-in caching in Firebase), but you can create the challenge by just not using one of these libraries. I've tried building this myself and it's quite tough figuring out how to do incremental update, writing/reading from DB properly without race conditions, and making sure that the UI stays updated as the data set changes underneath.

Open-source

• This is probably the closest you can get to scale by finding a library without thousands of stars and users.
• However, I feel like the barrier to entry for a library of this scale is really high, so be prepared for that.

Expanding Scope

• Is there some massive, massive customer that you're working with (customers of LiveRamp are entire companies right?) where you can find some of this complexity in their solution?
• The other dimension is time. The ideal case is that you work on an application for a huge customer over a long period of time, so you start hitting issues at scale and get to build that depth.

Switching Teams

• This one is obvious: If you aren't getting what you want, this is always a potential option.
• However, I feel like some engineers use this option to aggressively instead of trying to make the best out of their current situation first. Losing all the trust and working knowledge when you switch teams is really painful.