Posts for: #Databases

Columnar Storage: Predicate Pushdown

A row-oriented database stores each row contiguously: all columns for row 1, then all columns for row 2. This is optimal for OLTP workloads that read or write one row at a time. Analytics queries read one or two columns across millions of rows: “sum of revenue where region = ‘us-east’.” Row storage forces you to read every column of every row to compute this, even though you only need two.
[Read more]

Trace Storage and Querying

A distributed trace is a tree of spans. Each span records one operation: service name, operation name, start time, duration, tags (key-value pairs), and parent span ID. Storing traces so that you can find “all traces for user 12345 that had errors in the payment service last hour” requires a storage design that supports multiple query patterns simultaneously on append-heavy write load. The Write Pattern Spans arrive as a stream: millions per second in a large system (after sampling).
[Read more]

Comment Pagination at Scale

A viral post on Reddit has 50,000 comments. You can’t load them all at once. You need pagination — but comment pagination is harder than standard cursor-based pagination because the data is a tree, not a list. Loading “the next 20 comments” in a threaded comment system requires deciding what “next” means for hierarchical data. Top-Level Pagination The simplest strategy: paginate only top-level comments. Load the first 25 top-level comments. Each top-level comment shows its top 3 replies inline.
[Read more]

Nested Comment Trees

Reddit comments nest arbitrarily deep. A top-level comment has replies. Each reply has replies. The thread can be 15 levels deep. Storing and querying hierarchical data in a relational database has multiple approaches, and the right one depends on how the data is read: do you fetch the whole tree at once, or do you load levels lazily? Adjacency List The simplest model: each comment stores its parent’s ID. comments(id, post_id, parent_id, author, body, created_at) A top-level comment has parent_id = NULL.
[Read more]

Multi-Location Inventory

Amazon has thousands of warehouses. When you order a product, the system checks which warehouses have it in stock, selects the optimal fulfillment location, and reserves that unit. Multi-location inventory is different from seat inventory consistency: seats are identical and interchangeable, but warehouse location matters for delivery time and shipping cost. The same product at a warehouse 2,000 miles away is not the same as one 50 miles away. The Data Model Inventory is per-SKU per-warehouse: (sku_id, warehouse_id, quantity_available, quantity_reserved).
[Read more]

Variable-Attribute Product Catalog

Amazon sells 350 million products. A shoe has size, color, and material. A TV has screen size, resolution, refresh rate, and HDR type. A book has ISBN, author, and page count. No two product categories share the same attributes. A relational table with a column per attribute would have thousands of columns, almost all null for any given product. Variable-attribute product catalog is the data modeling problem of storing structured but heterogeneous data efficiently.
[Read more]

Swipe Storage at Scale

Tinder processes 1.6 billion swipes per day. Each swipe is a binary decision: left (pass) or right (like). This data drives match detection, recommendation quality, and abuse prevention. Swipe storage is a write-heavy, eventually consistent problem with specific read patterns: “has user A already swiped on user B?” and “who has liked user A?” Write Path At 1.6 billion swipes per day, that’s 18,500 writes per second on average, with heavy peaks during evenings.
[Read more]

Ledger Architecture

A ledger that works for 1,000 transactions per day and a ledger that works for 1 million transactions per second are built differently. The double-entry model tells you what to store. Ledger architecture tells you how to store it at scale while keeping balance queries fast and consistency guarantees intact. The Balance Computation Problem Deriving balance as SELECT SUM(amount) FROM ledger WHERE account_id = X is correct but slow when the ledger has billions of rows.
[Read more]

Double-Entry Bookkeeping in Distributed Systems

Every financial system eventually rediscovers double-entry bookkeeping. Not because engineers read accounting textbooks, but because the alternative — tracking balances as a single number — produces bugs that are impossible to audit and reconcile. Double-entry is a 700-year-old invention that turns out to be exactly the right data model for distributed financial systems. The Core Principle Every transaction touches exactly two accounts: a debit on one, a credit on the other of equal amount.
[Read more]

Ephemeral Location Storage

A Nearby Friends feature needs to know where each of your friends is right now. Not where they were an hour ago. The data has a very short useful life: a location update older than 30 seconds is essentially stale. This is ephemeral location storage: high-frequency writes, short TTL, and no need for historical persistence. The Write Pattern Each mobile client sends location updates every 30 seconds while the app is active.
[Read more]

Seat Inventory Consistency

Selling seats is harder than it looks. At 10 AM when Taylor Swift tickets go on sale, you have 50,000 seats and 500,000 concurrent buyers. You need to sell each seat exactly once. Overselling is a business disaster. Underselling (seats going unsold because they were locked but not purchased) is revenue loss. The core problem: how do you keep seat inventory consistent under massive concurrent load? Why This Is Hard A seat has three states: available, held, sold.
[Read more]