Video conferencing runs over UDP, not TCP. TCP retransmits lost packets, which adds latency: waiting for a retransmit before playing the next frame makes real-time audio and video stutter. UDP drops lost packets. The application must handle loss itself — and must do so in under 20ms to stay imperceptible. Packet loss concealment is the set of techniques for making packet loss invisible or inaudible to users.

Why UDP#

TCP’s retransmission is fine for file transfer: it doesn’t matter if a packet arrives 200ms late as long as it arrives. For audio, a 200ms gap is obvious. For video, a missing frame that arrives after the next frame has already been rendered is useless. Real-time media accepts loss over latency.

At 1-2% packet loss (typical on the internet), a 30fps video stream loses one frame every 1.5-3 seconds. Without concealment, these appear as visible freezes or audio glitches.

Forward Error Correction#

FEC adds redundancy: alongside every N packets, send a recovery packet that can reconstruct any one of the N original packets. If one packet from the group is lost, the receiver uses the FEC packet to rebuild it without a retransmit.

WebRTC uses RED (Redundant Encoding) for audio: each audio packet also contains the previous packet’s audio at lower quality. If the current packet is lost, the receiver uses the copy from the next packet. Doubles the audio bandwidth but recovers from single-packet loss without any round-trip delay.

For video, FEC uses XOR recovery packets: send packets 1-5 plus packet 1 XOR 2 XOR 3 XOR 4 XOR 5. Any single loss from the group can be recovered.

graph TD A[Sender: video frames 1-5] --> B[FEC encoder: compute XOR recovery packet] B --> C[Send: packets 1, 2, 3, 4, 5, FEC] C --> D{Packet 3 lost in transit} D --> E[Receiver has: 1, 2, 4, 5, FEC] E --> F[Recover packet 3: XOR of 1 XOR 2 XOR 4 XOR 5 XOR FEC] F --> G[Decode video without visible artifact] style A fill:#000000,stroke:#00ff00,stroke-width:2px,color:#fff style B fill:#000000,stroke:#00ff00,stroke-width:2px,color:#fff style C fill:#000000,stroke:#00ff00,stroke-width:2px,color:#fff style D fill:#000000,stroke:#00ff00,stroke-width:2px,color:#fff style E fill:#000000,stroke:#00ff00,stroke-width:2px,color:#fff style F fill:#000000,stroke:#00ff00,stroke-width:2px,color:#fff style G fill:#000000,stroke:#00ff00,stroke-width:2px,color:#fff

Jitter Buffer#

Packets arrive out of order due to varying network paths. A jitter buffer reorders them before playback. The buffer adds a fixed delay (50-150ms) to allow out-of-order packets to arrive and be sorted. Too small a buffer: frequent out-of-order playback. Too large: perceptible latency in conversation.

Adaptive jitter buffers adjust depth dynamically: when network jitter increases (more out-of-order arrivals), depth increases. When the network stabilizes, depth shrinks. WebRTC implements adaptive jitter buffering in the browser.

NACK and PLI#

For high-importance video frames (keyframes), a retransmit is worth the latency. NACK (Negative Acknowledgment): receiver sends a NACK for a lost packet, sender retransmits it. The round-trip adds 20-100ms but recovers critical frames.

PLI (Picture Loss Indication): receiver requests a full keyframe when too many packets are lost to reconstruct the current frame. The sender immediately generates a keyframe, resetting the video stream to a decodable state. PLIs are expensive (keyframes are large) but necessary after severe loss events.

At Oracle#

Oracle’s remote desktop product over WebRTC used adaptive FEC: FEC overhead started at 20% redundancy and scaled up to 50% when measured packet loss exceeded 3%. Under normal network conditions, FEC overhead was minimal. During poor network periods, the higher redundancy maintained acceptable video quality without requiring retransmits. Measured video freeze rate dropped 70% after implementing adaptive FEC.

What I’m Learning#

Packet loss concealment is a layered approach: FEC handles predictable loss without retransmits, jitter buffers handle reordering, NACKs handle important missing packets, and PLIs reset the video when all else fails. Each layer adds overhead but reduces the visible impact of an imperfect network.

Have you tuned jitter buffer or FEC parameters in production, and what metrics did you optimize for?