Building a Real-Time Feed System (From Frontend to Backend)

How I accidentally designed a fullstack micro-architecture just to push new posts to the top.

🧠 Background

What started as a simple frontend project to display posts in a feed turned into an unexpected journey into system design. I wanted one thing: when a new post is made, users should see a “🔄 New posts available” button, just like Twitter.

The simplest idea? Polling. The better idea? Server-Sent Events. The fun idea? Let's architect a real-time event-driven system with RabbitMQ.

🏗️ Overview of the Architecture

Let's walk through what happens when someone posts a feed item:

Client sends a POST /feeds request.
API inserts into PostgreSQL and publishes an event to RabbitMQ.
A separate consumer service picks up that event and broadcasts it via Server-Sent Events (SSE) to connected clients.

It's decoupled, real-time, and scalable. Here's how I built it.

🧩 Tech Stack

Frontend: Next.js (App Router), TanStack Query, Tailwind, shadcn/ui
Backend API: Go + net/http
Database: PostgreSQL
Event Queue: RabbitMQ
Real-time Layer: Server-Sent Events (SSE)
Infra: Docker (RabbitMQ, Postgres)

🔧 Feed Insertion Flow

API (Go) handles POST /feeds, inserts into the DB, and publishes a FeedCreated event to RabbitMQ:

_ = queue.PublishFeedCreated(queue.FeedEvent{
  ID:      feedID,
  Content: req.Content,
  Author:  req.AuthorID,
  Time:    createdAt.Unix(),
})
go

This decouples DB writes from broadcasting logic.

📨 Event Broadcasting with RabbitMQ + SSE

A separate consumer service listens to the feed.created queue and uses an in-memory SSE pool to push to all connected clients:

// Simplified
func ConsumeAndBroadcast() {
  for msg := range rabbitmq.Consume("feed.created") {
    var event FeedEvent
    json.Unmarshal(msg.Body, &event)
    broadcaster.Broadcast(event)
  }
}
go

This design lets us horizontally scale consumers later.

📺 Frontend: Receiving New Feeds

We are going to use TanStack Query for data fetching, generally this is how TanStack Query works:

Using TanStack Query with useInfiniteQuery, we listen for SSE updates:

const eventSource = new EventSource('http://localhost:8080/events');
eventSource.onmessage = () => setHasNewFeed(true);
ts

Then we show a pulse button:

{hasNewFeed && (
  <Button onClick={refetch} className="animate-pulse">
    🔄 New posts available
  </Button>
)}
ts

📦 Pagination with Cursor

We use a base64-encoded composite cursor:

// backend
func EncodeCursor(ts int64, id string) string {
  return base64.URLEncoding.EncodeToString([]byte(fmt.Sprintf("%d|%s", ts, id)))
}
go

And a smart SQL query with a tie-breaker:

WHERE (created_at < $1 OR (created_at = $1 AND id < $2))
ORDER BY created_at DESC, id DESC
LIMIT $3
sql

This ensures no items are skipped across pages.

🎯 Why RabbitMQ?

Separation of concerns: API focuses only on DB operations.
Reliability: Even if the broadcaster is down, messages are buffered.
Scalability: Multiple consumers can process events (analytics, push, etc.).

📡 Why SSE over WebSocket? • Easier to implement • Reconnects out of the box •

Great for one-way communication like feeds

WebSocket is great too, but SSE won here for simplicity and performance.

🧠 What I Learned • Real-time UX requires backend orchestration.

Message queues simplify scaling and reliability.
Cursor pagination is trickier than you think.
Simplicity matters: SSE > WS for one-way streams.

The Repo

Here's the repo for this project:

Frontend: https://github.com/lordronz/news-feed/
Backend: https://github.com/lordronz/news-feed-backend/

🧵 Final Thoughts

This started with just a scrollable list. But like most engineering journeys, solving a UI problem exposed gaps in architecture—and that's where the real fun begins.

So next time someone asks: “Can we show new posts instantly like Twitter?” — you'll know it's not just a button. It's a system.

System Design with News Feed Use Case