Posts for: #Search

Faceted Search

Amazon’s left sidebar: “Brand: Nike, Adidas, New Balance. Price: Under $50, $50-$100. Size: 8, 9, 10, 11.” Each filter is a facet. Selecting one narrows results and updates the counts on all other facets. Faceted search is not just filtering — it’s computing counts for all possible filter values simultaneously, on the filtered result set. The Count Problem The hard part is not filtering: “show me Nikes under $50” is a simple index query.
[Read more]

Per-User Search Indexes

Web search builds one massive index shared by all users. Email search is different: every query is private to one user, and results depend on that user’s labels, read status, and folder structure. Gmail’s search index is not one index — it’s one index per user. Per-user search indexes are the right architecture when search results are private, personalized, and must reflect user-specific metadata. Why Not a Shared Index A shared inverted index for email would store every user’s email together: word “invoice” points to all documents across all users containing “invoice.
[Read more]

Multi-Stage Ranking

A search index with a billion documents returns thousands of candidates matching a query. Showing all of them ranked by TF-IDF doesn’t produce great results. The ranking signals needed for good results (machine-learned models, user personalization, freshness, domain authority) are expensive to compute. You can’t compute them for a thousand candidates in under 100ms. The solution: a funnel. Use cheap signals to eliminate most candidates fast, then apply expensive signals to the small remainder.
[Read more]

Query Expansion

A user searches for “couch.” The documents say “sofa.” Without query expansion, no results. With it, the search system understands that “couch” and “sofa” are synonyms and expands the query to match either. This seemingly simple addition improves recall dramatically, but introduces its own set of problems. Synonym Expansion The most basic form: maintain a synonym dictionary. Before executing the query, expand each term with its synonyms. "couch" → ["couch", "sofa", "settee", "loveseat"] Query: "buy couch" becomes: (buy) AND (couch OR sofa OR settee OR loveseat) Synonym dictionaries are domain-specific.
[Read more]

Index Partitioning

An inverted index maps terms to the list of documents containing them. A web-scale inverted index has billions of documents and trillions of term-document pairs. It doesn’t fit on one machine. You have to partition it. Two Partitioning Strategies Document partitioning: split documents across machines. Machine 1 has documents 1-1M, machine 2 has documents 1M-2M, and so on. Each machine builds a complete local inverted index over its documents. A query for “distributed systems” goes to every machine in parallel, each returns its local top-K results, a merger combines and re-ranks.
[Read more]