Recommendation System
AdvancedOverview
A Recommendation System (like those used by YouTube, Netflix, or TikTok) curates a highly personalized feed of content for a user. The core challenge is distilling a catalog of millions of items down to a few dozen highly relevant choices in under a few hundred milliseconds. Modern architectures solve this using a two-stage funnel: a lightweight Candidate Generation stage followed by a heavy, feature-rich Ranking stage.
Functional Requirements
- Return a personalized, ordered list of items (e.g., videos, products) for a user.
- Record user interactions (clicks, watch time, likes, purchases) to inform future recommendations.
- Support a mix of historical personalization, trending content, and serendipitous discovery.
- Ensure fresh content (newly uploaded videos) surfaces quickly.
Non-Functional Requirements
- Strict latency limits: the feed must load in < 200ms.
- Massive scale: handle billions of items and hundreds of millions of users.
- High availability: degradation should fall back to cached or trending feeds rather than failing.
- Real-time event processing to capture immediate shifts in user intent.
Capacity Estimation
Assume a video platform with 500M DAU and an active catalog of 100M videos.
- Traffic: 500M DAU checking their feed 10 times a day = 5 Billion feed requests/day ≈ 58,000 QPS.
- Event Ingestion: Every view, click, and pause is logged. Assuming 100 events per user per day = 50 Billion events/day ≈ 600,000 QPS into the telemetry pipeline (e.g., Kafka).
- Model Serving: Evaluating 500 candidates per feed request requires 29M model inferences per second (58k * 500). Heavy GPU/TPU serving fleets are required for the Ranking stage.
High-Level Architecture
The system is divided into two primary phases. The Candidate Generator (Retrieval) narrows the 100M catalog down to ~500 candidates using fast, coarse techniques (e.g., Two-Tower neural networks or collaborative filtering via ANN search). These 500 candidates are passed to the Ranker, a heavy deep neural network that scores each item using hundreds of real-time features (user history, time of day, device, exact watch times). The scored items are then passed through a Re-ranking/Filter layer to remove already-watched content and ensure diversity before being served to the user.
Data Model
| Entity | Fields / Schema | Storage Choice |
|---|---|---|
| user_profile | user_id (PK), demographics, long_term_interests (vector), short_term_history | NoSQL Key-Value Store (Redis / DynamoDB) |
| item_metadata | item_id (PK), category, tags, embedding (vector), stats (views, likes) | Wide-column Store or Document DB |
| user_interaction_events | event_id, user_id, item_id, event_type, timestamp, duration | Event Stream (Kafka) -> Data Lake (Iceberg / S3) |
Detailed Design
Stage 1: Candidate Generation (Retrieval)
Because running a complex model on 100M items is computationally impossible, retrieval must be O(1) or sub-linear. Techniques include:
- Collaborative Filtering: Finding similar users or item-to-item similarities.
- Two-Tower Neural Networks: One tower embeds the user's features, the other embeds the item's features. The dot product of these vectors represents affinity. These item embeddings are pre-computed and stored in a Vector DB (e.g., Faiss) for fast Approximate Nearest Neighbor (ANN) search.
Stage 2: Ranking
The Ranker takes the ~500 candidates and assigns a precise probability score (e.g., predicted watch time or probability of click). This is typically a large Deep Neural Network (DNN) like DLRM (Deep Learning Recommendation Model). It pulls dense real-time features from a Feature Store, combining user context (location, current session clicks) with item context (recent virality).
Real-Time Event Processing
A user's most recent clicks strongly dictate what they want *right now*. Interaction events flow into Kafka and are processed by a stream processing engine (Apache Flink), which immediately updates the user's short-term history in the Feature Store (Redis), allowing the Ranker to adapt within seconds of a click.
Bottlenecks & Solutions
The biggest bottleneck is the Network I/O for Feature Retrieval during the Ranking stage. Fetching hundreds of features for 500 items per request requires extremely low-latency lookups. The solution is caching heavily and using a specialized, in-memory Feature Store (like Redis Cluster or specialized C++ services) co-located with the inference servers.