Skip to content

design-google-search.md

Latest commit

 

History

History
34 lines (25 loc) · 1.55 KB

design-google-search.md

File metadata and controls

34 lines (25 loc) · 1.55 KB
Raw

Design Google Search

Crawl the web, build an index, and return ranked results for a query within milliseconds.

Requirements

  • Crawl and refresh a huge corpus of web pages.
  • Index content for fast lookup.
  • Return relevant, ranked results quickly.
  • Scale to enormous query volume.

Key ideas

  • Three stages: crawling (see the web crawler), indexing, and serving.
  • Inverted index: map each term to the list of documents containing it (see database indexing). The index is sharded across many machines.
  • Query serving: a query fans out to index shards, each returns top matches, and results are merged and ranked, all within tight latency.
  • Ranking combines relevance signals and authority; precompute what you can and cache popular queries.

High-level design

flowchart LR
    Crawl[Crawler] --> Index[Indexer]
    Index --> Shards[(Inverted Index Shards)]
    Query[Query] --> Root[Query Coordinator]
    Root --> Shards
    Shards --> Rank[Merge + Rank]
    Rank --> Results[Results]
Loading

Go deeper