System design interviews are a unique challenge. Unlike their algorithm coding counterparts, which often follow a more standardized approach, the absence of a consistent industry-wide method for evaluating system design skills only adds to this complexity, making the process seem enigmatic and somewhat daunting.
In system design interviews, the crux of the challenge lies in satisfying the non-functional requirements, primarily scalability and availability. These non-functional requirements, although often seen as secondary considerations, play a crucial role in determining the effectiveness and robustness of any system. For instance, scalability isn't just an attribute to be added later, but rather a core aspect of the design that significantly impacts the entire architecture and performance of a system. As such, to ensure effective scalability, a system often needs to be constructed from scratch with a keen emphasis on accommodating an expanding load.
The art of system design involves managing trade-offs, understanding and applying the right design principles, as well as navigating the vast sea of technology choices. We have come up with a system to tackle system design:
Understanding the common design goals, challenges, and conceptual solutions
First, let's take a look at the core challenges of designing web-scale data-intensive applications:
- Too many concurrent users
- Too much data to store and access
- The application needs to have low latency
- The state needs to be consistent
We will discuss these details in the next section.
Grasping the building blocks ("the lego pieces")
This part of the course will focus on the various components that are often used to construct a system (the building blocks), and the design templates that provide a framework for structuring these blocks.
Core Building blocks
Grasping the building blocks ("the lego pieces")
This part of the guide will focus on the various components that are often used to construct a system (the building blocks), and the design templates that provide a framework for structuring these blocks.
Core Building blocks
At the bare minimum you should know the core building blocks of system design
- Scaling stateless services with load balancing
- Scaling database reads with replication and caching
- Scaling database writes with partition (aka sharding)
- Scaling data flow with message queues
System Design Template
With these building blocks, you will be able to apply our template to solve many system design problems. We will dive into the details in the Design Template section. Here’s a sneak peak:
Additional Building Blocks
Additionally, you will want to understand these concepts
- Processing large amount of data (aka “big data”) with batch and stream processing
- Particularly useful for solving data-intensive problems such as designing an analytics app
- Achieving consistency across services using distribution transaction or event sourcing
- Particularly useful for solving problems that require strict transactions such as designing financial apps
- Full text search: full-text index
- Storing data for the long term: data warehousing
On top of these, there are ad hoc knowledge you would want to know tailored to certain problems. For example, geohashing for designing location-based services like Yelp or Uber, operational transform to solve problems like designing Google Doc. You can learn these these on a case-by-case basis. System design interviews are supposed to test your general design skills and not specific knowledge.
Working through problems and building solutions using the building blocks
Finally, we have a series of practical problems for you to work through. You can find the problem in /problems. This hands-on practice will not only help you apply the principles learned but will also enhance your understanding of how to use the building blocks to construct effective solutions. The list of questions grow. We are actively adding more questions to the list.