Day 9 B-Building blocks 2026-10-02 ← All lessons

Design a URL shortener: encoding, hashing, 301 vs 302

One hundred million long URLs arrive every day and each must shrink to seven characters, survive ten years, and redirect in milliseconds. This lesson builds the shortener: count the load, pick the redirect, then mint codes that never collide.

100000000New URLs shortened per day
1160Sustained shorten rate
11600Sustained redirect rate at 10 to 1
365Ten year storage at 100 bytes per URL

Key points

Outcomes

01Derive the load from 100M per day: 1,160 writes per second, 11,600 reads per second, 365B rows and 365 TB over ten years.
02Choose 301 vs 302 by stating who pays: server load against click analytics.
03Explain why base 62 conversion beats hash plus collision resolution, and prove 7 characters hold 365B URLs.
01

Count the load before drawing a box


The interview opens with four questions and the answers set every later decision. Example mapping: a long systeminterview.com URL with query parameters becomes https://tinyurl.com/y7keocwj, and clicking the alias redirects to the original. Traffic is 100 million new URLs per day. Codes must be as short as possible over 62 characters (digits plus both cases). Shortened URLs are never deleted or updated, which removes a whole class of invalidation logic. Two use cases remain: shorten a long URL, and redirect a short URL.

Run the envelope live: 100M divided by 86,400 seconds is about 1,160 writes per second. Reads outnumber writes 10 to 1, so plan for 11,600 reads per second. Ten years of writes is 100M times 365 times 10, or 365 billion rows. At 100 bytes per URL that is 36.5 terabytes per year, 365 TB for the decade. Reads dominate, so the design that wins is the one that serves redirects cheapest.

Shorten writes
1.2K/s
Redirect reads
11.6K/s
Ten year row count
365B rows
πŸ“ Log scale β€” each step right is ~10Γ—. Bar lengths show order of magnitude, not raw proportion.
Headroom on a log scale: reads dominate writes ten to one.
02

Redirect first: 301 sheds load, 302 counts clicks


Two endpoints carry the whole API, REST style. POST api/v1/data/shorten takes the long URL and returns the short URL. GET api/v1/shortUrl returns the long URL as an HTTP redirect. The redirect has two flavors and the choice is the first real tradeoff of the lesson.

Option A

301: cache it, skip us next time

  • Semantics: the short URL has permanently moved
  • Browsers cache it, so only the first request per client hits your servers
  • Server load drops, which matters at 11,600 reads per second
Option B

302: count every click

  • Semantics: the short URL has temporarily moved
  • Every repeat request hits your servers first, then redirects
  • Click rate and click source stay measurable, which analytics needs

Rule of thumb: pick 301 when the priority is shedding server load, 302 when the priority is knowing who clicked. Most interviewers follow up by asking what changes when analytics is added later, and the answer is that flipping 301 to 302 replays load onto the web tier, so the cache and database must already be sized for it.

03

The redirect path: cache first, database second


The naive redirect store is a hash table of short URL to long URL pairs. It fits the whiteboard and fails reality: memory is limited and expensive, so the mapping lives in a relational table with three columns (id auto increment, shortURL, longURL). Reads outnumber writes ten to one, so a cache sits in front of the database and the redirect path checks it first. The load balancer sprays clicks across stateless web servers, each of which reads through the same cache.

How to read: Follow the dots: the click fans out across web servers, takes the upper path on a cache hit or drops to the database on a miss, and lands on the long URL.

You clickLoad balancerWeb serversCache hitDatabase
  1. You click the short URL link.

  2. The load balancer forwards to one web server.

  3. Cache hit: return the long URL directly.

  4. Cache miss: fetch from the database, then return.

  5. Missing in both: the short URL is invalid.

  6. The browser follows the redirect to the long URL.

One click, two paths: cache hit returns immediately, cache miss falls through to the database.
GotchaGotcha: a miss in both cache and database almost always means a typed or forged short URL, not a system fault. Return an error fast instead of retrying a lookup that will never succeed.
04

Minting codes: hash and pray, or convert and know


The short URL is the domain plus a code, so the design problem reduces to a function from long URL to code that is reversible: every long URL maps to one code, and every code maps back. The alphabet holds 62 characters, and the smallest length covering 365 billion URLs comes from 62 to the 7, which is about 3.5 trillion. Length 6 holds only 56.8 billion, so 7 is the floor, not a guess.

62 to the 5
916M
62 to the 6: too small
56.8B
Decade requirement
365B
62 to the 7: picked
3.5T
πŸ“ Log scale β€” each step right is ~10Γ—. Bar lengths show order of magnitude, not raw proportion.
Capacity by length: 7 is the first length that clears the decade requirement.
Option A

Hash plus collision: simple, priced in

  • Hash with CRC32, MD5, or SHA-1, keep 7 chars, append a string and rehash on collision
  • Needs a database existence check per write, plus bloom filters to stay fast
  • Collisions are certain at this scale, so the hot path pays for them forever
Option B

Base 62 conversion: free uniqueness

  • Take the row unique ID and convert it to base 62, as in 11157 becomes 2TX
  • No existence check: IDs are unique by construction, so codes are too
  • Reuses the Chapter 7 unique ID generator for distributed minting
Interview tipInterview line: when asked to compare, say hash plus collision is stateless but pays a read per write, while base 62 needs a coordinated ID generator and then never collides. The book picks base 62.
05

The shorten path: dedup first, mint second


The shortening flow is six steps and the database check up front is what makes repeat submissions free. Given a long URL, look it up: if present, return the stored code untouched. If new, ask the unique ID generator for an ID, convert it to base 62, and insert the (ID, code, long URL) row. Concrete trace from the chapter: the Wikipedia systems design URL arrives, the generator returns 2009215674938, conversion yields zn9edcu, and that triple is stored.

How to read: Follow the dots: the submission checks the database first, takes the upper path for a known URL or mints and converts on the lower path, and a row lands in storage.

You submitWeb serverKnown: return codeNew: mint IDURL row
  1. Submit the long URL to the web server.

  2. Long URL already stored: return its code.

  3. New URL: request a unique ID from the generator.

  4. Convert the ID to base 62 for the code.

  5. Insert the ID, code, and long URL row.

  6. Return the short URL to the client.

One submission, two outcomes: known URLs return instantly, new URLs mint an ID and convert it.

The distributed ID generator is doing quiet heavy lifting here: globally unique IDs without coordination per request, which is exactly what Chapter 7 built the snowflake for. If the generator stalls, shortening stalls, so it inherits the mission-critical availability requirement.

06

Code it: base 62 in twenty lines


The lines that matter are the digit map and the divide loop. Everything else is argument plumbing. Run it: 11157 encodes to 2TX, and decoding reverses the walk.

python
ALPHABET = '0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ'
BASE = len(ALPHABET)  # 62

def encode(num):
    assert num >= 0
    if num == 0:
        return ALPHABET[0]
    out = ''
    while num > 0:  # repeated divmod, least significant digit first
        num, rem = divmod(num, BASE)
        out = ALPHABET[rem] + out
    return out

def decode(s):
    num = 0
    for ch in s:  # Horner walk: shift left by one digit, add value
        num = num * BASE + ALPHABET.index(ch)
    return num

print(encode(11157))           # 2TX, the chapter trace
print(decode('2TX'))           # 11157, round trip holds
print(encode(2009215674938))  # chapter-scale ID to 7-char code
print(len(encode(62**7 - 1)))  # 7: the full range fits the picked length

Wrap-up talking points when time remains: a rate limiter in front of shortening (malicious clients can flood minting, see Chapter 4), stateless web tier scaling by adding servers, database replication plus sharding, an analytics pipeline for click counts, and the availability versus consistency stance inherited from Chapter 1. The interview closes the same arc it opened: scope with numbers, high-level flows, deep dive on the code mint, then honest extras.

Q&A

Check yourself


Q1Marketing wants per-link click counts by hour and referrer. Which redirect do you serve?
  • Return 301 so browsers cache and skip your servers
  • Return 302 so each click is counted at your servers
  • Return 301 for the first click and 302 after
βœ“ Return 302 so each click is counted at your servers β€” 302 forces every repeat click through your servers, which is exactly the hit stream analytics needs.
Q2You hash with MD5 and keep the first 7 characters, resolving collisions by retry. What breaks first at 1,160 writes per second?
  • Every write pays a database existence check, often more than one
  • Reads slow down because the cache cannot hold colliding keys
  • IDs stop sorting by time and the feed breaks
βœ“ Every write pays a database existence check, often more than one β€” Truncating any hash to 7 chars guarantees collisions at this scale, so every write would pay a read-check plus possible retries.
Q3The alphabet is 0-9, a-z, A-Z and the decade needs 365B codes. What length do you pick?
  • 5 characters: 916M codes are plenty with rate limiting
  • 6 characters: 56.8B codes cover a decade of growth
  • 7 characters: the smallest n with 62 to the n above 365B
βœ“ 7 characters: the smallest n with 62 to the n above 365B β€” 62 to the 6 holds only 56.8B, below the 365B requirement, while 62 to the 7 holds 3.5T with headroom to spare.
Sources: System Design Interview Vol 1: Ch. 8, Design a URL shortener (pp. 119-131)