System Design

System Design

System design plays a crucial role in ensuring that a system is reliable, performant, and scalable

System design is the process of determining the optimal high-level technical approach to a problem. It's a staple in both the interview process and actually doing your job as a software engineer. We have created a video series to act as a system design primer for software engineers.

Is there a way to "grind" system design or soft skills?

Mid-Level Software Engineer at Twitch profile pic
Mid-Level Software Engineer at Twitch

I'm not sure exactly how to phrase this, but to give an analogy, I love card games (ex: Legends of Runeterra, Race for the Galaxy, Hearthstone, etc). There are a fixed set of rules and a fixed set of cards. I can "grind" games and get better by noticing patterns, picking up new strategies or tactics by playing against a diverse set of players. The outcome of an interaction is usually idempotent (i.e. card 1 interaction with card 2).

In real life, things are quite complicated. Asking a certain question in a certain way to person 1 and person 2 may give wildly different responses, and may even depend on your mood, their mood, your tone, time of day, etc. It's super messy and unpredictable.

I also feel a similar way about system design. The nearly infinite possibility of inputs, outputs, TPS, throughput, scenarios make it difficult to reapply the same set of rules to different scenarios. This is just talking about one component, when we bring in N components, the interaction gets very complicated and the "rules" change" case by case. I'm sure it gets better with practice, but I also feel I have a limited opportunity to learn or practice these on the job.

Has anyone found a way to structure these learnings in terms of a repeated "grind", because oftentimes I feel overwhelmed and don't know where to start. This is a complicated question, so answers regarding either a) soft skills or b) system design separately I will treat as valid answers.

Show more
Editor's Choice
13 days ago

What are the things I should look out for when designing a file storage system?

Anonymous User at Taro Community profile pic
Anonymous User at Taro Community

I am working on a side project. I may use GCP or AWS for it.

tldr of the system : Create a web ui to process input files and generate a tabular output giving the link to the dashboard .

I have a bunch of files generated , about 2GB in size in total , Account ID is the high level directory and inside each account there 50 log files each having specific content .

I want to create a UI using Angular framework . Using the UI I will upload the files to GCS /AWS S3 . Then the backend should trigger complex backend scripts, written will be written in Python or Java and then output file is generated. Using the output file a dashboard with few images embedded is generated.

In the end the UI should have a table which should indicate status of processing to COMPLETE and in another column show point to url of the dashboard.

I want to deal with least amount of security work as I want to avoid studying Oauth 2.0 authentication .

Few additional questions , non-system design questions =>

  1. What are the things I should learn in Angular to cover most bases in this project, I am a backend engineer and going over youtube videos . I want to stick with Angular as I have few work projects going to come in coming months so it will help me to learn it .
  2. I am unfamiliar with typescript . How much effort will it need to learn in order to code for this project. I am most familiar with Python only.
Show more
6 months ago

System design is a necessary process at software companies to design architectures that can be compatible with different requirements.