I spent two days debugging why users were reporting “lost data” in a distributed KV store I was building. Turns out, nothing was lost. The data was there. Just… not where the user expected it.

The problem: user writes “value2”, immediately reads back “value1”. From their perspective, the database ate their write. From the database’s perspective, everything’s working perfectly - the write went to the leader, the read hit a follower that hadn’t caught up yet.

This is the gap session guarantees fill.

What Session Guarantees Actually Mean#

Read-your-writes: After you write something, you always see it (or something newer). Sounds obvious, right? But in a distributed system with async replication and reads from followers, it’s not automatic.

Monotonic reads: Once you’ve seen data at version X, you never see anything older than X. Time doesn’t go backwards for you.

These aren’t the same thing. I confused them for weeks.

The Difference That Clicked for Me#

Think of it like a time machine with two rules:

Read-your-writes = “You remember what YOU did” You planted a tree yesterday. Every time you visit the garden, that tree must be there. Even if someone else planted flowers last week, you must see YOUR tree.

Monotonic reads = “Time only moves forward” You visited the garden in 2024 and saw robots. You can never visit the garden in 2023 again, even if it’s a different garden. Once you’ve experienced 2024, you can’t unsee it.

How I Implemented It (The Hard Way)#

First attempt: track timestamps per key. Memory exploded. For 10,000 unique keys read in a session, that’s 640 KB just for tracking. For a batch job reading millions of keys? Gigabytes.

Second attempt: global session timestamp. Track one number - the highest timestamp you’ve ever seen across ALL keys.

long lastReadTimestamp = 0;   // Highest you've read
long lastWriteTimestamp = 0;  // Highest you've written

// When reading with VERSION_CHECK mode
long minVersion = Math.max(lastReadTimestamp, lastWriteTimestamp);
// Server rejects if it has older data

Memory: 16 bytes. Doesn’t matter if you read 1 key or 1 million keys.

%%{init: {'theme':'base', 'themeVariables': { 'primaryColor':'#000000','primaryTextColor':'#00ff00','primaryBorderColor':'#00ff00','lineColor':'#00ff00','secondaryColor':'#000000','tertiaryColor':'#000000','noteBkgColor':'#000000','noteBorderColor':'#00ff00','noteTextColor':'#00ff00'}}}%% sequenceDiagram autonumber participant C as Client participant F as Follower participant L as Leader Note over C: lastWriteTimestamp = 2000 C->>C: Write key=value2 at t=2000 C->>F: Read key?minVersion=2000 Note over F: Has timestamp=1500
Too stale! F-->>C: 409 Conflict Note over C: Auto-retry with leader C->>L: Read key?minVersion=2000 L-->>C: value2, timestamp=2000 ✓ Note over C: User sees their write

Server-side validation prevents stale data from ever reaching the client.

The Trade-off I Didn’t Expect#

Global tracking is stricter than per-key. If you read key1 at time 1000, then try to read key2 at time 500, it gets rejected even though they’re different keys.

This is actually causal consistency - stronger than monotonic reads. MongoDB does this with “operation time” for sessions. Once your session has seen time T, it never goes backwards for ANY operation.

Is it too strict? Maybe. But it’s simpler, uses constant memory, and users prefer predictable behavior over surprising edge cases.

When This Actually Matters#

Building a social feed: User posts an update, refreshes, doesn’t see it. Without read-your-writes, this happens randomly with async replication. With it, you either route them to the leader or track the write timestamp and reject stale followers.

Building a dashboard: User views metrics, refreshes, sees older metrics. Without monotonic reads, your dashboard looks buggy. Numbers jumping around. With it, time only moves forward.

Building a batch processor: Processing millions of records. Do you really need session guarantees per key? Probably not. Global tracking keeps memory bounded without exploding.

The Code Pattern#

// After every write
PutResponse response = httpClient.send(putRequest, ...);
lastWriteTimestamp = response.getTimestamp();

// Before every read with VERSION_CHECK
long minVersion = Math.max(lastReadTimestamp, lastWriteTimestamp);
String url = server + "/get/" + key + "?minVersion=" + minVersion;

// Server validates
if (storageValue.getTimestamp() < minVersion) {
    return 409;  // Too stale, client retries with leader
}

// After successful read, update session horizon
if (response.getTimestamp() > lastReadTimestamp) {
    lastReadTimestamp = response.getTimestamp();
}

The client sends what it needs. The server validates before serving. If a follower is too stale, it says so, and the client automatically retries with the leader. User never sees stale data.

What I’d Do Differently#

For a real production system with long-lived sessions (hours, not minutes), I might use an LRU cache with bounded size for per-key tracking. Top 10,000 recently accessed keys get precise guarantees, rest fall back to global. Best of both worlds.

But for learning? For most use cases? Global tracking is simpler and good enough.

Session guarantees aren’t magic. They’re just timestamps and validation. But they’re the difference between a database that feels broken and one that makes sense to users.

Have you dealt with read-your-writes violations in production? How’d you fix it?