A row-oriented database stores each row contiguously: all columns for row 1, then all columns for row 2. This is optimal for OLTP workloads that read or write one row at a time. Analytics queries read one or two columns across millions of rows: “sum of revenue where region = ‘us-east’.” Row storage forces you to read every column of every row to compute this, even though you only need two. Columnar storage solves this by storing each column’s values contiguously instead.

How Parquet Works#

Parquet organizes a file into row groups (default 128MB each). Within each row group, data is stored by column. To read the revenue column from a 10TB table, you read only the revenue column chunks from each row group, skipping all other columns.

Each column chunk is compressed independently. Compression is extremely effective on columnar data because values in the same column are often similar: a region column with 5 distinct values compresses with dictionary encoding to near-zero overhead. A timestamp column with monotonically increasing values compresses well with delta encoding.

Compression ratios of 5-10x over raw row storage are common. Smaller files mean less I/O, which in object storage (S3) translates directly to query cost.

Predicate Pushdown#

Parquet stores statistics for each column chunk in each row group: min value, max value, null count. These are stored in the file footer, readable without scanning the data.

Query: WHERE event_date = '2026-08-01'. Before reading any data, the query engine reads the footer and checks: which row groups have an event_date min/max range that includes ‘2026-08-01’? Row groups that cannot contain matching rows are skipped entirely.

graph TD A[Query: SELECT SUM revenue WHERE region = us-east AND date = 2026-08-01] --> B[Read Parquet file footer: row group statistics] B --> C{Row group min/max date includes 2026-08-01?} C --> |No| D[Skip entire row group: zero I/O] C --> |Yes| E[Read only revenue and region column chunks for this row group] E --> F[Apply region filter in memory] F --> G[Sum revenue values that pass filter] D --> H[Aggregate results across qualifying row groups] G --> H H --> I[Return: total revenue for us-east on 2026-08-01] style A fill:#000000,stroke:#00ff00,stroke-width:2px,color:#fff style B fill:#000000,stroke:#00ff00,stroke-width:2px,color:#fff style C fill:#000000,stroke:#00ff00,stroke-width:2px,color:#fff style D fill:#000000,stroke:#00ff00,stroke-width:2px,color:#fff style E fill:#000000,stroke:#00ff00,stroke-width:2px,color:#fff style F fill:#000000,stroke:#00ff00,stroke-width:2px,color:#fff style G fill:#000000,stroke:#00ff00,stroke-width:2px,color:#fff style H fill:#000000,stroke:#00ff00,stroke-width:2px,color:#fff style I fill:#000000,stroke:#00ff00,stroke-width:2px,color:#fff

Column Encoding#

Dictionary encoding: for a status column with values (pending, active, cancelled), store a dictionary mapping each string to an integer (0, 1, 2) and store only integers in the column data. Strings that repeat thousands of times take 2 bytes instead of 7-9.

Run-length encoding (RLE): for a region column sorted by region, consecutive identical values compress to (value, count) pairs. 10,000 consecutive us-east entries store as (us-east, 10000).

Bit packing: integer columns with small ranges (status codes 0-5) use 3 bits per value instead of 32.

These encodings stack: a dictionary-encoded column further compresses with RLE. Data sorted on the partition key before writing maximizes RLE effectiveness.

Partitioning and File Layout#

Write Parquet files partitioned by date and region: data/date=2026-08-01/region=us-east/part-0001.parquet. A query filtered to one date and region reads only the files in that partition. The file system structure acts as a coarse partition filter before Parquet’s row group statistics apply.

At Oracle#

Oracle Analytics Cloud’s data layer used Parquet on object storage for event data from production systems. A daily revenue reconciliation query over 18 months of data (120TB raw) ran in 4 hours against row-oriented CSV exports. After converting to Parquet with date partitioning and sorting by customer_id within each file, the same query ran in 11 minutes. Column pruning (reading 3 of 47 columns) plus row group skipping (date predicate eliminated 97% of row groups) accounted for the speedup.

What I’m Learning#

Columnar storage is the right default for any write-once, read-many analytics workload. The gains come from three orthogonal sources: column pruning (read fewer bytes), predicate pushdown (skip row groups), and compression (store fewer bytes). Getting all three requires partition layout and sort order to align with actual query patterns.

Have you tuned Parquet file layout for a specific query pattern, and how much did sort order within files change your query performance?