Every phone running Google Maps is a sensor. It sends GPS coordinates every few seconds. Aggregate enough of those pings from enough phones on the same road, and you can calculate actual vehicle speeds. That’s where the red and yellow lines on the map come from.

The Pipeline#

Each GPS update includes a location, timestamp, and device ID. The system needs to:

Map the coordinate to a road segment. A phone at (37.7749, -122.4194) is on a specific 100-meter stretch of Market Street. This is a map-matching step, using the road graph and nearby segment candidates.

Group pings by segment and compute speed. If a device was at position A 10 seconds ago and position B now, and the road segment between A and B is 200 meters long, the speed was 72 km/h.

Aggregate across all devices on that segment to get a representative speed. A sliding window of the last few minutes smooths out noise from single vehicles.

graph TD A[GPS Ping: lat, lng, timestamp, device_id] --> B[Map Matching: lat/lng to segment_id] B --> C[Speed Calculation per Device] C --> D[Sliding Window Aggregation per Segment] D --> E[Segment Speed: 45 km/h] E --> F[Update Road Graph Weights] F --> G[Routing Queries Use Updated Weights] style A fill:#000000,stroke:#00ff00,stroke-width:2px,color:#fff style B fill:#000000,stroke:#00ff00,stroke-width:2px,color:#fff style C fill:#000000,stroke:#00ff00,stroke-width:2px,color:#fff style D fill:#000000,stroke:#00ff00,stroke-width:2px,color:#fff style E fill:#000000,stroke:#00ff00,stroke-width:2px,color:#fff style F fill:#000000,stroke:#00ff00,stroke-width:2px,color:#fff style G fill:#000000,stroke:#00ff00,stroke-width:2px,color:#fff

Why This Is a Stream Processing Problem#

GPS pings arrive continuously. Aggregation has to be continuous too. A batch job that runs every 5 minutes would produce traffic data that’s already stale by the time routing queries use it.

Stream processing windows are the right abstraction here: sliding windows per segment, computing average speed over the last N minutes, updating as new pings arrive and old ones expire.

Handling Sparse Segments#

Not every road has enough traffic to generate real-time speed data. Side streets at 3am have no GPS pings. For these, you fall back to historical speed data: the typical speed for that segment at that time of day, learned from months of past data.

The system blends live data (when available) with historical baseline (when live is sparse). Confidence degrades gracefully rather than breaking.

At Oracle#

We had network KPIs, metrics aggregated per 15-minute intervals across thousands of network elements. Late-arriving data was a constant problem: a node would report metrics 10 minutes after the window closed. We had to decide whether to reopen closed windows or discard late data. We settled on a short allowed-lateness window (20 minutes) with a final correction pass. Watermarks solve exactly this in modern stream systems.

What I’m Learning#

Traffic aggregation is fundamentally a sliding window problem with a map-matching preprocessing step. The volume is high, but each computation is simple. The hard part is handling edge cases: sparse segments, GPS noise, tunnels where signal drops and then resumes.

Have you built real-time aggregation pipelines that had to handle sparse or noisy input data?