Search / Feed Ranking System (Learning-to-Rank)

Advanced

Overview

While basic search relies on keyword matching (TF-IDF/BM25), a modern Search or Feed Ranking system uses Machine Learning (Learning-to-Rank) to order candidates based on probability of relevance, engagement, or conversion. This system takes a list of candidate items, enriches them with hundreds of real-time features, and scores them using ML models (like XGBoost or deep neural networks) in milliseconds.

Architecture of a Real-Time Learning-to-Rank (LTR) Search System

Functional Requirements

Given a user query (or context) and a list of candidates, return a ranked list.
Support continuous A/B testing of different ranking models without downtime.
Incorporate real-time features (e.g., item popularity in the last 15 minutes).
Log features and outcomes (clicks) identically to prevent train-serve skew.

Non-Functional Requirements

Ultra-low latency: the ranking phase must complete in < 50-100ms.
High throughput: handle tens of thousands of search requests per second.
Consistency between offline training data and online serving data.
Graceful degradation: fall back to heuristic/BM25 sorting if the ML ranker times out.

Capacity Estimation

Assume an e-commerce platform with 10,000 search queries per second (QPS).

Scoring Load: If the retrieval stage returns 500 candidates per query, the ranker must score 5,000,000 items per second.
Feature Fetching: If the model uses 200 features per item, the Feature Store must serve 1 Billion feature lookups per second. This necessitates an ultra-fast, distributed in-memory cache (e.g., Redis Cluster).
Telemetry Storage: Logging the features for 5M items/sec generates gigabytes of telemetry data per second, requiring heavy stream processing and batching before writing to the Data Lake.

High-Level Architecture

The system consists of the Search Gateway, the Retrieval Engine (Elasticsearch/Solr), the ML Ranking Service, and the Feature Store. When a query arrives, the Gateway asks the Retrieval Engine for the top 500 keyword matches. The Gateway passes these IDs to the ML Ranking Service. The Ranker fetches real-time item and user features from the Feature Store, constructs feature vectors, and runs the inference model (e.g., XGBoost). The scored items are sorted and returned. Concurrently, the exact feature vectors used during serving are logged asynchronously to a Kafka topic for future model training.

Data Model

Entity	Fields / Schema	Storage Choice
item_features	item_id (PK), historical_ctr, category_conversion_rate, price_percentile	Feature Store (Redis / Aerospike)
user_features	user_id (PK), past_categories_viewed, device_type, user_intent_score	Feature Store (Redis / Aerospike)
training_logs	request_id, item_id, feature_vector (JSON/Protobuf), label (clicked/purchased)	Data Lake (S3 + Parquet / Iceberg)

Detailed Design

The Feature Store

The Feature Store is the backbone of LTR. It solves Train-Serve Skew by ensuring the exact same code generates features for offline training and online serving. It operates in two modes: an offline store (HDFS/S3) for batch training, and an online store (Redis/Memcached) for low-latency serving. A stream processor (Flink) constantly calculates sliding-window aggregations (e.g., "clicks in last 5 mins") and updates the online store.

Model Inference

While deep learning is popular, Gradient Boosted Decision Trees (GBDTs like XGBoost or LightGBM) are still dominant in tabular/ranking tasks due to their inference speed and handling of dense/sparse features. To achieve < 50ms latency for 500 items, inference is heavily parallelized across CPU cores, and models are compiled down to optimized C++ or ONNX runtimes.

Experimentation (A/B Testing)

Ranking systems are never static. An Experimentation Engine at the Gateway routes a percentage of traffic (e.g., 5%) to a 'Treatment' model while the rest goes to the 'Control' model. Telemetry tags every log with the model version, allowing data scientists to analyze statistically significant shifts in Conversion Rate (CVR) or Click-Through Rate (CTR).

Bottlenecks & Solutions

Feature Fetch Latency is the primary bottleneck. Fetching hundreds of features for hundreds of items over the network adds significant latency. Solutions include caching popular features locally in the Ranker's RAM, compressing features using Protobuf/FlatBuffers, or pushing the ML model directly into the Search Engine node (e.g., Elasticsearch LTR plugin) to avoid network hops entirely.

Interview Follow-up Questions

Q: How do you handle 'Position Bias' in your training data?

Users click the top result simply because it's at the top. If we naively train on this, the model learns that being at the top causes relevance. We mitigate this by using position-debiased learning algorithms, randomizing results for a small percentage of traffic to gather unbiased data, or explicitly adding 'position' as a feature during training but holding it constant during online serving.

Q: What happens if the Feature Store goes down?

The ML Ranker cannot score items without features. The system must degrade gracefully. The Search Gateway detects the timeout or failure from the Ranker and immediately falls back to the raw, unranked list provided by the Retrieval Engine (which is usually sorted by BM25 or recency). High availability is prioritized over ML personalization in failure scenarios.