Solution

Design a Comment System

Design a comment system, such as Disqus, is a feature integrated into websites that allows users to post comments on published content, interact with other users through replies, upvote or downvote comments, and sort or filter comments. This system must be robust, scalable, and efficient to handle a large volume of users and comments.

Overview

The behavior of comments on social media and content platforms, such as YouTube, perfectly illustrates the delicate balance between maintaining real-time accuracy and ensuring quick system responses. Have you ever noticed that after posting a comment on a YouTube video, it sometimes doesn't appear immediately when you view the video from a different account or in an incognito mode? This occurs because, for many platforms, comments aren't seen as urgent. Therefore, they use a method called "eventual consistency," which might delay the display of some comments to all users instantly.

In this example, we will apply our system design template. If you're interested in a hands-on approach, you can also explore our code demonstration to better understand how eventual consistency functions in practice.

Functional Requirements

  • User Authentication: Users can create an account, log in, and log out. This requires a secure authentication system that can handle 100M Daily Active Users.
  • Comment Submission and Display: Users can submit comments, which are then stored in a database and displayed under the related content. Assuming each user posts 10 comments every day, and each comment is 100 bytes, the system needs to handle 1 billion comments daily.
  • Replying to Comments: Users can reply to other comments, with replies nested under the original comment. This requires a system that can handle complex data relationships.
  • Upvote or Downvote Comments: Users can upvote or downvote comments. This requires a system that can track and update the vote count for each comment in real-time.
  • Moderation Tools: Administrators can delete inappropriate comments or ban users. This requires a robust administration system that can handle a large volume of moderation actions.

Non-Functional Requirements

  • Scalability: The system needs to support 100M Daily Active Users and handle 1 billion comments posted daily.
  • Reliability: The system must ensure data integrity and error handling.
  • Availability: The system must be operational when needed, using techniques like replication to avoid downtime.
  • Consistency: Allow for brief inconsistencies in the system within a few seconds, but the system will reach consistency afterwards (eventual consistency).
  • Latency: The system must have low latency to provide a smooth user experience.
  • Efficiency: The system must minimize redundant operations and optimize resource usage.

Resource Estimation

Assuming a read:write ratio of 100:1.

Using the resource estimator, we get the following results:

System Design Diagram

API Endpoint Design

The system will feature API endpoints designed for submitting and displaying comments, responding to comments, and utilizing moderation tools. Each of these endpoints will have a specific request and response format.

  • POST /comment

    Request body:

    { "content": "" // the content of comment }

    Response body:

    { "status": "success" | "failed", "message": "", // string "comment": { "id": "generated_id", "content": "the-content-of-the-content", "posterId": "the-user-id-of-poster", "posterName": "The display name of poster", "postTime": "2022-01-01T12:00:00Z" } }
  • GET /{topic_id}/comments?cursor={last_timestamp}&size={size}: This API endpoint returns data sorted in descending order, from the most recent to the oldest.

    Path variables: The variable topic_id is typically associated with a specific page in a one-to-one relationship. By inputting the relevant 'topic_id' of the page, you can retrieve the comments related to that page.

    Request Parameters:

    • last_timestamp: This parameter is used to query the timestamp of the last comment from the previous list. If you're retrieving data from the first page, this parameter should not be set.
    • size: This parameter determines the number of lists retrieved at once. The default value is 20, but it can be set to a maximum of 50.

    Response body:

    { "status": "success" | "failed", "total": 12345, // The total count of comments of the specified topic. "data": [ // If there are no comments, then this is an empty array. { "id": "the-id-of-the-comment", "content": "The content of comment", "posterId": "the-user-id-of-poster", "posterName": "The display name of poster", "postTime": "2022-01-01T12:00:00Z", "upvoteCount": 123, "downvoteCount": 12, "repliedCommentId": "the-id-of-replied-comment" | undefined, // If this comment replies to another comment, then this field points to the commented comment. }, ... ] }
  • POST /comment/{comment_id}/reply

    Path variables: comment_id, the id of replied comment.

    Request body:

    { "content": "" // the content of comment }

    Response body:

    { "status": "success" | "failed", "message": "", // string "comment": { "id": "generated_id", "content": "the-content-of-the-content", "posterId": "the-user-id-of-poster", "posterName": "The display name of poster", "postTime": "2022-01-01T12:00:00Z", "repliedCommentId": "the-id-of-replied-comment" } }
  • PUT /comment/{comment_id}/upvote

    Path variables: comment_id, the upvoted comment id.

    Response body:

    { "status": "success" | "failed", "message": "", // string }
  • PUT /comment/{comment_id}/downvote

    Path variables: comment_id, the downvoted comment id.

    Response body:

    { "status": "success" | "failed", "message": "", // string }
  • DELETE /admin/comment/{comment_id}

    Path variables: comment_id, the ID of the comment to be deleted.

    Response body:

    { "status": "success" | "failed", "message": "", // string }

Grasping the building blocks ("the lego pieces")

This part of the guide will focus on the various components that are often used to construct a system (the building blocks), and the design templates that provide a framework for structuring these blocks.

Core Building blocks

At the bare minimum you should know the core building blocks of system design

  • Scaling stateless services with load balancing
  • Scaling database reads with replication and caching
  • Scaling database writes with partition (aka sharding)
  • Scaling data flow with message queues

System Design Template

With these building blocks, you will be able to apply our template to solve many system design problems. We will dive into the details in the Design Template section. Here’s a sneak peak:

System Design Template

Additional Building Blocks

Additionally, you will want to understand these concepts

  • Processing large amount of data (aka “big data”) with batch and stream processing
    • Particularly useful for solving data-intensive problems such as designing an analytics app
  • Achieving consistency across services using distribution transaction or event sourcing
    • Particularly useful for solving problems that require strict transactions such as designing financial apps
  • Full text search: full-text index
  • Storing data for the long term: data warehousing

On top of these, there are ad hoc knowledge you would want to know tailored to certain problems. For example, geohashing for designing location-based services like Yelp or Uber, operational transform to solve problems like designing Google Doc. You can learn these these on a case-by-case basis. System design interviews are supposed to test your general design skills and not specific knowledge.

Working through problems and building solutions using the building blocks

Finally, we have a series of practical problems for you to work through. You can find the problem in /problems. This hands-on practice will not only help you apply the principles learned but will also enhance your understanding of how to use the building blocks to construct effective solutions. The list of questions grow. We are actively adding more questions to the list.

Read this article with a free account