When 500,000 people hit your site at 10:00:00 AM, your backend doesn’t scale to 500,000 concurrent sessions in milliseconds. A virtual waiting room absorbs the spike: users enter a queue, receive a position, and are admitted to the actual purchase flow in controlled batches. Without it, the site goes down for everyone. With it, users wait but the experience is orderly.

The Queue Token Pattern#

When a user arrives during high load, issue them a signed queue token containing their position and timestamp. The user’s browser polls a lightweight status endpoint with this token every few seconds: “am I admitted yet?” This endpoint does one thing: check if the user’s position is below the current admission threshold.

The admission threshold advances as users complete or abandon purchases. If 1,000 users are admitted at a time and each has a 10-minute purchase window, you advance the threshold by roughly 1,000 every 10 minutes. The math is: admitted users per batch = (peak backend capacity) / (average session duration in batches).

The token must be signed to prevent forgery. A user shouldn’t be able to change their position from 50,000 to 1. Use an HMAC with a server secret: token = base64(position + timestamp + HMAC(position + timestamp, secret)).

graph TD A[User arrives at 10:00 AM] --> B{Load above threshold?} B --> |No| C[Admit directly to purchase flow] B --> |Yes| D[Issue signed queue token: position 47382] D --> E[User polls status endpoint every 5 seconds] E --> F{Position below admission threshold?} F --> |No| E F --> |Yes| G[Issue time-limited admission token: valid 10 min] G --> H[User enters purchase flow] style A fill:#000000,stroke:#00ff00,stroke-width:2px,color:#fff style B fill:#000000,stroke:#00ff00,stroke-width:2px,color:#fff style C fill:#000000,stroke:#00ff00,stroke-width:2px,color:#fff style D fill:#000000,stroke:#00ff00,stroke-width:2px,color:#fff style E fill:#000000,stroke:#00ff00,stroke-width:2px,color:#fff style F fill:#000000,stroke:#00ff00,stroke-width:2px,color:#fff style G fill:#000000,stroke:#00ff00,stroke-width:2px,color:#fff style H fill:#000000,stroke:#00ff00,stroke-width:2px,color:#fff

Admission Control#

The admission token (separate from the queue token) is what actually grants access to the purchase flow. It’s time-limited: if you don’t start a purchase within 10 minutes of being admitted, your token expires and you go back to the queue. This prevents admitted users from holding spots indefinitely.

The admission service is a simple Redis counter: INCR admitted_count. If admitted_count < capacity_limit, admit. Expired sessions decrement the counter via a background job that checks for sessions older than the admission window.

Backpressure is the underlying principle: the waiting room is the mechanism that applies backpressure from the saturated purchase backend to the incoming user stream.

Fairness vs Speed#

A pure FIFO queue is fair but slow: users at position 1 who are slow to complete a purchase block everyone behind them. Alternatives: admit in batches (everyone below position N admitted simultaneously, creating a rush within the batch), or use randomized admission within a position window (anyone between position 40,000 and 50,000 has equal probability of being next). Ticketmaster uses a randomized approach within windows to prevent bots from gaming exact positions.

At Salesforce#

We had a version of this for scheduled maintenance windows where all customers could re-enable their orgs after downtime. Every org admin hit the “re-enable” button at the same time. Without rate control, the database was saturated within seconds. We added a token bucket at the admission layer: 500 re-enable operations per second, excess requests queued with a polling response. Queue depth peaked at 8,000 orgs. Everyone was re-enabled within 16 minutes instead of the site going down.

What I’m Learning#

Virtual waiting rooms solve a specific problem: absorbing instantaneous traffic spikes that exceed backend capacity. The key insight is separating the “accept the user into the system” layer from the “let the user do the expensive thing” layer. The waiting room handles the spike; the backend only sees controlled load.

Have you built or used virtual waiting rooms, and what was the hardest part to get right?