Recommender System Architecture
The multi-stage funnel that takes a massive catalog of millions of items and efficiently filters it down to the top 10 most relevant items for a specific user.
Why Does This Exist?
When you open Netflix, the system needs to recommend 10 movies out of a catalog of 10,000. When you open YouTube, the system needs to recommend 10 videos out of a catalog of 800 million.
You cannot run a heavy, highly accurate Deep Learning model to score all 800 million YouTube videos every time a user refreshes the homepage. It would take hours and cost millions of dollars in server fees.
To solve this, modern recommender systems are built as a Multi-Stage Funnel. They split the workload into two fundamentally different tasks: a blazing fast algorithm to quickly discard the obvious garbage (Retrieval), and a slow, accurate algorithm to carefully judge the survivors (Ranking).
Think of It Like This
Think of It Like This
Imagine you are hiring a new CEO, and 10,000 people applied.
Stage 1: Retrieval (Candidate Generation) You cannot interview 10,000 people. You run a quick, cheap keyword filter on the resumes. If they don't have "Executive Experience", they go in the trash. You instantly reduce the pile from 10,000 to 100.
Stage 2: Ranking (Scoring) Now that you only have 100 candidates, you can afford to do something very slow and expensive: a 2-hour in-person interview. You meticulously score the remaining 100 candidates and offer the job to the top 1.
This is exactly how YouTube and TikTok work.
The Funnel Architecture
Every major tech company uses some variation of this pipeline.
Stage 1: Retrieval (Candidate Generation)
- Goal: Reduce 1,000,000 items to 500 items in less than 50 milliseconds.
- How it works: This stage heavily prioritizes speed over accuracy. It uses fast algorithms like Collaborative Filtering or Two-Tower Neural Networks combined with Approximate Nearest Neighbor (ANN) search.
- The philosophy: It is perfectly fine if the Retrieval stage includes some mediocre items in the 500. The only unforgivable sin is if it accidentally drops the best item. It must have high recall.
Stage 2: Filtering (Business Logic)
- Goal: Remove items the user shouldn't see.
- How it works: This is usually a set of hardcoded rules, not machine learning. Out of the 500 items from Stage 1, we remove items the user has already bought, items that are currently out of stock, or items that violate safety guidelines. We are left with 400 items.
Stage 3: Ranking (Scoring)
- Goal: Meticulously sort the surviving 400 items to find the absolute best 10 items.
- How it works: This stage heavily prioritizes accuracy over speed. Because there are only 400 items left, we can afford to run a massive, complex neural network. This model looks at hundreds of features (the exact time of day, the user's micro-interactions from 5 minutes ago, the exact text of the item description) to calculate a final probability score (e.g., "There is a 4.2% chance they will click this").
Stage 4: Re-Ranking (Diversity)
- Goal: Ensure the final 10 items aren't completely identical.
- How it works: If the Ranking stage outputs 10 videos about "How to bake a cake", the user will get bored. Re-ranking applies a penalty to similar items to enforce diversity, ensuring the final carousel has a mix of genres.
Show Me the Code
You rarely build this entire funnel from scratch. In Python, you can use the merlin framework from NVIDIA, or orchestrate it yourself using different models for each stage.
# A conceptual representation of the recommender funnel
def recommend_homepage(user_id, total_catalog): # 1. RETRIEVAL: Fast ANN search (e.g., FAISS or Two-Tower) # Reduces 1,000,000 items to 500 candidates = fast_retrieval_model(user_id, total_catalog, top_k=500) # 2. FILTERING: Hard business rules # Reduces 500 to ~400 candidates = [c for c in candidates if c.in_stock and not c.already_purchased(user_id)] # 3. RANKING: Heavy Deep Learning model (e.g., XGBoost or DLRM) # Scores the remaining 400 items scored_candidates = [] for item in candidates: score = heavy_ranking_model.predict_probability_of_click(user_id, item) scored_candidates.append({'item': item, 'score': score}) # Sort by score descending ranked_list = sorted(scored_candidates, key=lambda x: x['score'], reverse=True) # 4. RE-RANKING: Ensure diversity among the Top 10 final_10 = apply_diversity_penalty(ranked_list[:30], top_k=10) return final_10Watch Out For
Watch Out For
The Retrieval stage is the bottleneck. If your highly sophisticated Deep Learning Ranking model is struggling to improve metrics, the problem is usually your Retrieval stage. A Ranking model can only sort the 500 items it is given. If the true best item is trapped in the 999,500 items that were thrown away by the Retrieval stage, the Ranking model has exactly 0% chance of finding it.
The Quick Version
- You cannot run heavy ML models on millions of items simultaneously.
- Recommender systems use a funnel to narrow down the choices.
- Retrieval prioritizes extreme speed to slash the catalog down to hundreds of candidates.
- Ranking prioritizes extreme accuracy to score the remaining candidates and pick the winners.
- Re-Ranking ensures the final presentation is diverse and visually appealing.