There are some great articles about how an effective feed can completely reshape the internet. Facebook’s News Feed turned social media into a battle royale of attention and FOMO. This isn’t that kind of feed.

One of the late-stage features we built on Comend was the Feed. The Feed was our way of turning a messy mix of product activity into one clean, usable stream. Instead of only showing user actions, it was designed to pull together internal events, system-generated updates, and third party activity into a single timeline that could still respect scope, deduplicate reliably, and stay easy to query and render. I was particularly excited about tracking relevant content from sources like FDA Alerts, new PubMed papers, and activity from clinicaltrials.gov.

The goal was to make the frontend feel simple without making the backend naïve. We wanted it to be performant without having to add any new technology (stick to our existing stack of Next.js and PostgreSQL), and still have the confidence that it can scale to thousands of posts from hundreds of sources. We wanted a feed that could scale across many event types and sources, normalize them into one stable model, and let the UI focus on presenting the right card rather than trying to interpret raw activity itself.

Frontend for feed features is pretty simple:

• fetch recent activity
• sort chronologically
• render cards in a list
• add pagination

This works for very simple feeds, but it stops working once the activity in our product stops being uniform.

In our case, activity came from a mix of places:

• user actions inside the product
• internal system events
• ingest jobs
• outside sources (third party APIs, RSS feeds, etc.)
• events that belonged to a specific organization
• events that were broader than any one organization

Some of that third party activity was not pushed to us in real time. Instead, we used a daily cron job (Vercel makes this easy) to poll and ingest new items from outside sources, then run them through the same normalization pipeline as everything else. Once we have that mix, a feed stops being “a list of recent things” and starts becoming a normalization problem.

The pattern that ended up working

The useful pattern was to split the system into layers instead of letting every feature write directly into a user-facing timeline.

At a high level, it looked like this:

1. Capture raw activity
2. Translate it into a canonical feed shape
3. Enforce scope and idempotency
4. Store the final feed item
5. Query it through a simple paginated API
6. Render by item type on the frontend

That separation gave us a lot more flexibility than treating the feed as just another table in the app.

1. Capture raw activity

One of the best decisions was not writing directly into the feed table.

Instead, raw events were first written into a RawIngestEvent model with fields like:

• provider
• eventType
• externalId
• occurredAt
• payload
• receivedAt
• dedupeKey
• status
• error

The dedupe key was generated by hashing the provider, event type, and either the external ID or the payload itself. The write path then used an upsert, so retries did not automatically create duplicate ingest rows. This layer was especially useful for third party sources. Some of those were ingested by a daily cron job that polled external APIs and RSS feeds, wrote new entries into RawIngestEvent, and let the rest of the system treat them the same way as product-generated activity.

That raw layer ended up doing a few important jobs for us:

• Durability: we had a record of what came in before it was interpreted
• Idempotency: retries were much safer
• Debuggability: we could inspect pending, failed, processed, and skipped events
• Flexibility: producers did not need to know the final feed shape

That distinction mattered a lot.

A raw event is just:

something happened

>

A feed item is:

this is how we want to present that event to a user

>

Those aren’t the same thing, and treating them as separate concepts made the system much easier to extend. This separation also acts as a safety buffer between what we collect and what we show, and opens up the possibility for tracking activity that we want for ourselves but not for the feed.

2. Canonicalization is how the system becomes reusable

The next step was a processor that pulled pending raw events in batches and resolved a mapper based on the event’s provider and eventType. Each mapper translated the source payload into one or more canonical feed objects, which were then validated before being written into FeedItem.

This was probably the most reusable part of the whole design.

Without canonicalization, every new event source risks leaking its irrelevant attributes deeper into the system. The frontend ends up understanding too much and query logic starts spawning special cases. Adding a new source becomes more expensive than it should be. With canonicalization, the messy source-specific logic stays at the outskirts of the system.

Internally, the system just needs to understand:

• a feed item has a type
• it has content
• it has a source
• it occurred at a certain time
• it belongs to some scope
• it has a stable identity

This is a much better foundation than letting the whole product depend on the raw shape of each incoming event.

3. Scope rules belong in the write path

Another thing that helped was enforcing audience rules early.

In the processor, some feed types were considered global and some were organization-scoped. The processor explicitly checked that:

• global feed types must not have an orgId
• organization-scoped feed types must have an orgId

If those rules were violated, the event failed before it was stored as a feed item.

If we leave scope implicit, it tends to get reimplemented inconsistently:

• one query filters correctly
• another forgets
• one component assumes global
• another assumes org-scoped
• eventually the feed starts showing the right data to the wrong audience

It’s much cleaner to make those semantics part of the write path rather than relying on every read path to remember them.

4. We dedupe twice

There are really two (corresponding to the difference between raw ingest events and feed items) different duplication problems in a feed system:

1. Did we receive this source event more than once?
2. Should the user see this feed item more than once?

These are related, but aren’t the same question.

So we handled duplication at two levels:

• RawIngestEvent had a dedupe key for the source event
• FeedItem had its own idempotencyKey for the canonical, user-facing item

The processor then used an upsert keyed on that feed-level idempotency key.

That separation was helpful because:

• one raw event can produce multiple feed items
• multiple raw events can sometimes collapse into the same visible concept
• transport-level retries should not necessarily create UI-level duplicates

It made the system more resilient without forcing those concerns to collapse into one crude “is this a duplicate?” check.

5. The read side stays simple

Because the ingest and normalization pipeline did the heavy lifting, the read API could stay fairly clean. The feed route did a few straightforward things:

• checked the authenticated session
• rejected unauthorized requests
• verified organization membership when an org-specific feed was requested
• parsed a cursor from createdAt and id
• fetched a page
• serialized timestamps
• returned a nextCursor if more results existed

By the time our API returns items, most of the important decisions had already been made.

One small technical detail; the cursor used both:

• createdAt
• id

instead of relying on timestamp alone. This is because feeds often have multiple items with the same timestamp granularity, and timestamp-only pagination can get unstable. Using the timestamp plus a tie-breaker gives us a much cleaner continuation point.

6. The frontend is just a registry of UI

On the client side, the feed list component mostly just had to:

• fetch pages from /api/feed
• append them locally
• support infinite scrolling
• provide a manual “Load more” fallback
• render each item with the right card component

The list itself used an IntersectionObserver to auto-load when the sentinel entered view, but still kept an explicit button for fallback and accessibility; typical feed best practices.

The rendering side used a registry that mapped feed item types to specific card components. This lets different item types have distinct presentations without turning the feed into one giant conditional renderer. More specialized cards could even be lazy-loaded when needed, while unknown types fall-back to a generic card. This registry pattern aged much better than a single feed card component full of branching logic would have.

The final pattern

If I had to boil the whole thing down, the reusable pattern is this:

1. Capture events in raw form
2. Normalize them into a canonical contract
3. Enforce audience and scoping rules
4. Deduplicate at both the event and feed layers
5. Store a stable feed model
6. Keep the read API simple
7. Render by type, not by source

This approach takes a little more discipline up front, but it buys us:

• cleaner boundaries
• easier debugging
• safer retries
• more predictable querying
• more extensible rendering (adding new feed content is a cinch)
• less coupling between producers and presentation

What helped a lot while building this was the time I spent many years ago studying design patterns on Refactoring Guru’s website. Some patterns I used just for this implementation (yes I asked codex to tell me what were used) include:

• mapper lookup by type/provider → Strategy
• feed card registry by item type → Strategy
• canonical event creation → often Factory Method or Builder
• staged processing flow → loosely Template Method
• generic fallback wrappers around specialized renderers → sometimes Decorator
• feed composed of many item renderers → lightly Composite
• infinite feed traversal/pagination → Iterator
• event-driven UI updates / subscriptions, if present elsewhere → Observer

Comend’s Feed looked like a timeline on the surface, but the more reusable pattern underneath was really about normalization: taking messy, heterogeneous activity and turning it into something consistent, scoped, and understandable. Although it was never the core feature we emphasized to users, it was still my favourite feature of all the things we built.