Not a manager here but I strongly feel like understanding my manager's perspective is essential for me to successfully navigate my tech career.
I understand that if a software engineer wants to become promoted, they need to complete projects with increasing scope involving increased technical depth, ambiguity, impact, etc in addition to many other things.
This is a great question and something more software engineers should think about! If you know what motivates your manager, you can understand their actions better, and that can educate how you work with them.
At some point, I'd love to do a full session with Alex about the life of an eng manager (or someone in the Taro Premium community can lead it), but here's how I think about performance evaluation for a manager:
The number of engineers on the team absolutely does matter. Presumably, a larger team means the manager has more scope, more outcomes, and more influence in the company. All of these are necessary prerequisites for getting promoted.
As you point out, this very often does lead to empire-building. I don't have a great answer on how to prevent that, let me do a bit more thinking.