How do I validate a design for a large scale system before making the decision to invest further?
I am hitting the point in my career where I am responsible for designing newer systems to handle more novel problems. I will use my last project as an example.
I had to redesign our game engine to handle more complex scenarios. I decided to "eat the frog" and come up with a few very complex game interactions to test if any of our designs would satisfy those test cases. After about 1 month of development, we landed on a design. This design also passed all the test cases for our old engine.
I think back and wonder if I could've done it more incrementally. We didn't really ship anything during that time, just tested out different ideas in our feature branches. There are still some outstanding questions for that project that are as low level as "should we use an abstract base class or an interface for this abstraction" to "here's an even more complex interaction we don't know how to handle, but could feasibly happen".
Now I am designing a new animation pipeline and don't want to get stuck in the same 1 month long marathon with only a design doc to show for it. As a newer senior engineer and lead, a solid design doc seems like a win, but it was a tough month.
In one way, this is different than having all the work known and chunking it out. We aren't sure what we'd come up with. Maybe that's the nature of R&D. On the other hand, work is work, and I'm sure there are methodologies for making more consistent progress even in R&D.
Interested in how other people approach this dilemma. I think the crux of the issue is I wasn't sure our design would work until we built and tested it against the test cases, and I have a nagging feeling we got a little lucky.
Discussion
(1 comment)- 410 months agoTech Lead/Manager at Meta, Pinterest, Kosei
David Pan talks about how the best tech leads are able to progressively de-risk a project with every passing week, and it sounds like that's where you didn't feel that happened.
The hard part about system design is that there's no right answer- it really depends on the constraints in your company. Here are some questions I'd ask to determine where you are on the spectrum of R&D vs ship-it-no-matter-what.
- How big is your company? How many people will have to maintain/improve the design in the coming months?
- What is the cost of changing the design or fixing it later?
- How much does this body of work deviate from the past work? In your case, how many times has the company redesigned the game engine, and what additional risk is there here?
It's worth documenting the trade-offs and potential issues, then collecting feedback from the team. The uncertainty you're feeling around whether the design would work is not a great place to be, but the feedback can ensure that no one is surprised (since everyone technically signed off on it).
My personal bias is that you should have receipts for your work as frequently as possible, unless you're truly in an R&D role. I'd push you to have some progress update or deliverable that marks progress every week. A month is far too long in startup-world. I'm not as sure about how the game industry works, but I'd assume similarly for most gaming companies as well.
BTW, there are great discussions about this on the system design topic on Taro.
