Designing a Distributed Real-Time Chat System (Backend Deep Dive)
Most chat applications look simple from the outside.
Under the hood, they are complex distributed systems that must handle real-time communication, ordering, failures, and scale.
To understand this deeply, I built a distributed real-time chat backend in Go with support for multi-server architecture, reliable delivery, and scalable message distribution.
System Goal
The system is designed to guarantee:
Reliable message delivery
Real-time communication
Horizontal scalability
Fault tolerance
Architecture Overview
Client → WebSocket Server → Connection Manager → Redis → PostgreSQL
WebSocket server handles real-time connections
Connection manager tracks active users
Redis handles distribution, presence, and coordination
PostgreSQL ensures durability
WebSocket Server
WebSockets were chosen over HTTP polling because they provide full-duplex communication with low latency.
Each connection is modeled as:
user ID
socket
send channel
Instead of writing directly to the socket, messages are pushed into a channel.
This decouples read and write operations, prevents blocking, and isolates slow clients.
Connection Manager
The connection manager maintains:
map[userID][]Connection
A user can have multiple connections across devices.
A read-write mutex ensures thread-safe access while allowing concurrent reads.
Message Flow
Send flow:
Client → WebSocket → Server → Assign sequence → Store in DB → Fanout
Receive flow:
Channel → Write loop → Socket
This separation ensures clean handling of incoming and outgoing data.
Persistence with PostgreSQL
Messages are stored before being sent.
This guarantees:
Crash safety
No data loss
Stored data includes:
Messages
Sequence numbers
Delivery state
Conversation membership
Read receipts
Message Ordering
In a distributed system, messages can arrive out of order.
To solve this, each conversation has a sequence number.
Redis is used for atomic increments:
INCR conversation::seq
This guarantees consistent ordering across servers.
Using the database for sequencing would introduce latency and contention.
Idempotency
Retries can cause duplicate messages.
To handle this:
Each message includes a client_msg_id
The database enforces uniqueness
Duplicate inserts are ignored
This ensures messages are stored only once.
Delivery Guarantee
The system uses at-least-once delivery.
Flow:
Server sends message
Client sends acknowledgment
Server marks message as delivered
If acknowledgments are lost, messages are retried.
Exactly-once delivery is avoided due to complexity and cost.
Fanout Optimization
A naive approach sends one message per user.
This results in O(n) operations.
Instead, users are grouped by server and messages are published once per server.
This reduces complexity to O(number of servers), which is far more efficient for large groups.
Redis Usage
Redis is used for:
Pub/Sub for message distribution
Presence tracking
Sequence generation
Rate limiting
Its in-memory design and atomic operations make it ideal for distributed coordination.
Presence System
User presence is tracked using:
user_servers: → set of server IDs
TTL and heartbeat mechanisms ensure stale entries are removed automatically.
Group Chat Model
Instead of separate logic for one-to-one and group chats, everything is modeled as a conversation.
This simplifies system design and reduces complexity.
Message History
Messages are fetched using cursor-based pagination:
Query by sequence number
Ordered results
Limited result set
This approach is more efficient and scalable than offset-based pagination.
Read Receipts
Each user maintains a last seen sequence number per conversation.
This avoids storing per-message read states and reduces write overhead.
Typing Indicators
Typing indicators are treated as ephemeral data.
They are not stored and are only transmitted via pub/sub.
Rate Limiting
Rate limiting is implemented using Redis with atomic counters and TTL.
This prevents abuse and ensures fairness across users.
Backpressure Handling
Slow clients can cause buffer buildup.
To prevent this, non-blocking channel writes are used.
If a client cannot keep up, the connection is dropped.
This protects the system from memory exhaustion.
Observability
The system includes:
Structured logging
Metrics collection
Observability is critical for debugging and monitoring real systems.
Running the System
Requirements:
Go
PostgreSQL
Redis
Start services and run the server:
go run main.go
Multiple instances can be started to simulate a distributed environment.
Testing
Connect using a WebSocket client
Authenticate user
Send messages
Verify delivery and ordering
Multiple server instances can be used to test distribution.
Benchmarking
Key metrics:
Messages per second
Latency (p50, p95, p99)
Memory usage
Redis throughput
Load testing tools such as k6 or wrk can be used.
Tradeoffs
Several tradeoffs were made:
Redis Pub/Sub instead of Kafka for simplicity
At-least-once delivery instead of exactly-once
Fanout-on-write strategy for lower latency
These decisions prioritize performance and simplicity.
Failure Scenarios
If a server crashes during message delivery:
Message is already stored in the database
Client reconnects and fetches missed messages
If an acknowledgment is lost:
Retry mechanism resends the message
Idempotency prevents duplication
If Redis fails:
Cross-server communication is affected
System partially degrades
Future Improvements
Replace Redis Pub/Sub with Kafka
Add message encryption
Support media storage
Implement sharding
Add push notifications
Conclusion
Building a distributed chat system reveals the complexity behind real-time applications.
The system must balance performance, reliability, and scalability while handling failures gracefully.
This project focuses not just on making the system work, but on making it work under real-world conditions.