FAANG System Design Interview: Design A Chat System (WhatsApp, Facebook Messenger, Discord, Slack) Transcript, AI Summary & Key Points

00:00 WhatsApp and Facebook Messenger process billions of messages daily. Behind every message is a distributed system delivering text instantly across the globe. How do we design a test system that can handle real-time messaging at this scale? Let's break it down. There's a lot to cover when designing a test system, so let's focus on the core features. Our test system supports one-on-one and group chats up to 100 participants.

00:24 We guarantee message delivery. We store messages for offline users for a limited time. This is different from systems that keep all messages in central storage forever. We also show online presence with that familiar green dot. We're designing for a growing chat app that starts with thousands of users and scales to millions. How do clients and servers communicate in a chat system?

00:48 We could use HTTP for everything. When someone sends a message, the client makes an HTTP post request. The server acknowledges and forwards the message to the recipient. This works well for sending, but receiving is problematic. HTTP is client initiated. Servers can easily push messages directly to clients. How do we get messages to users instantly?

01:10 Polling is an option. The client repeatedly ask any new messages. Most requests return empty. This waste server resources. Long pollings hold connections open longer until messages arrive or the connection times out. This reduces requests, but we can do better. Websocket offers a better solution. After an HTTP handshake, the connection upgrades to a persistent channel.

01:36 The server can push messages instantly. However, websocket connections are more difficult to scale, which we'll address later in the video. We could use websocket for both directions, but we keep things simple. HTTP works fine for sending and scales easily. Websocket handles the real challenge receiving messages instantly. So we choose a hybrid approach.

01:57 HTTP for sending messages and websocket for receiving them. Now we have chosen our protocols. Let's design the overall system. We need multiple components working together to handle a large number of users. We divide the system into three layers. Salus services, staple services and third party integrations. Sailor services handle authentication, user profiles, and message sending through REST APIs.

02:23 Load balances distribute requests across multiple servers. This is a known design pattern that scales easily by adding more servers as needed. Chat servers form the stable core. Each client maintains a persistent websocket connection to one chat server. Modern servers can handle tens of thousands or hundreds of thousands of concurrence websocket connections.

02:45 This capacity determines how many servers we need as we scale. When someone sends you a message, but you are not connected to a chat server, the system needs another way to reach you. Third party services like Apple's APNS or Google's FCM deliver push notifications when users are offline. They maintain persistent connections to millions of devices. This is not something we want to reinvent.

03:12 Our architecture handles online users well, but what about offline scenarios? When someone sends you a message while your phone is off or when network connections fail during transmission, how do we guarantee delivery? The inbox pattern solves this. Every users gets an inbox, a personal queue that stores and deliver messages. When someone sends you a message while you're offline, it waits in your inbox for a limited time.

03:36 Here's how it works. When someone sends you a message, the server tries to deliver it immediately if you're online. Your device gets the message through websocket and sends back an acknowledgement. Done. No storage needed. But if you're offline or the delivery fails, the message goes into your inbox. It waits there until you come back online. The acknowledgement mechanism provides reliability for inbox messages.

04:01 When your device receives a message from the inbox, it sends back an act. The server only removes the message from the inbox after getting this acknowledgement. No act means the server will retry later. When you come back online, your chat server checks your inbox for waiting messages. All accumulated messages download to your device in order, bringing you up to date.

04:22 The inbox patterns works great, but now we have a new challenge. With multiple chat servers handling different users, how do messages travel between them? We need a way to route messages between chat servers when users are connected to different servers. The most scalable approach uses a combination of service discovery and direct server communication.

04:42 Let's say Alice sends a message to Bob. Alice's chat server first checks if Bob is online. The server queries a user presence service that tracks which chat server Bob is connected to. This service acts like a directory mapping users to the current server locations. If Bob is online, Alice's server makes a direct RPC call to Bob's chat server. Bob server gets the message and immediately pushes it through Bob's websocket connection.

05:09 This direct approach minimizes latency. This scales well with service discovery systems that help servers find each other. Chat platforms like Discord uses direct serverto-s server communication pattern. If Bob is offline, the message goes to his inbox for later delivery. And we trigger a push notification through third party services. One-on-one messages works smoothly now, but group chat introduce a new complexity.

05:36 How do we efficiently deliver one message to 100 different users? We use a fan out pattern. When someone sends a group message, the server checks which members are online and make RPC calls to deliver immediately to their devices. Offline members get the message stored in the inboxes. This works well for a requirement of up to 100 members since the fan out workload scales with group size.

06:01 With messaging sorted out, let's work on another feature, online presence. The green dot showing online status seems simple but creates interesting technical challenges. We use heartbeats to track presence. We can rely on websocket connection stay alone. connection can appear alive even when the app is backgrounded or the device is sleeping. Clients send a websocket ping frame every 30 seconds to their chat server.

06:26 The server tracks the timestamp of the last ping it receives for each user. If the chat server gets no ping for 60 seconds, it marks the user offline. This approach smooths out brief disconnection in tunnels, elevators, or areas with poor coverage. We built a working chat system, but success brings new problems. As our user base grows from thousands to millions, which components bricks first connections likely becomes the first bottleneck.

06:54 As user count grows, we need more chat servers to handle the connections. The database can also hit limits as we scale. As millions of users go offline and come back online, the inboxes filled up and empty constantly. We can split users across multiple database servers based on their user ids. For global reach, we deploy chat servers in multiple regions.

07:16 Users connect to their nearest region for low latency, but cross region message routing as complexity. This covers the core chat system architecture. There are other areas worth considering. What about message ordering? When messages travel across distributed servers, they can arrive out of order. Message IDs, timestamps, and vector clocks help solve this.

07:38 Security and encryption add another layer of complexity. End-to-end encryption requires careful key exchange mechanisms. Group chess makes this even more challenging. Media handling is its own beast. Sharing images and videos requires compression, CDNs, and progressive loading to keep things fast. Then there features like red receipts and typing indicators.

08:01 Broadcasting typing status to group members sounds simple but can easily overwhelm your system without careful design. One more, we need ray limiting and abuse prevention. Stopping spams and API abuse while keeping the experience smooth is a delicate balance. We are barely scratching the surface, but this is how we design the core features of a chat system that handles real-time messaging at scale.

08:27 Ready to ace your next technical interview? Join our community where we offer comprehensive courses on system design, coding, behavioral questions, machine learning, and object-oriented design. Learn more at bitebico.com.