A user uploads a 500MB video. 90% through, their mobile connection drops. They reconnect and have to start over. That’s the worst case for user experience and a waste of everyone’s resources. Resumable uploads solve this by making partial progress durable.

Why Standard HTTP Uploads Break#

A standard HTTP POST sends the entire file as one request body. The server processes it when the full body is received. If the connection drops at byte 490MB of a 500MB upload, the server discards everything received so far. The client has to start from byte 0.

For small files this is acceptable. For large files (video, raw images, database dumps), it’s not.

Chunked Upload Protocol#

The solution: split the file into chunks on the client side, upload each chunk as a separate request, and track which chunks the server has received.

Initialization: the client sends a request to start an upload session, receiving back an upload ID and the expected chunk size.

Chunk upload: the client sends chunks sequentially or in parallel, each with the upload ID, chunk index, and byte range.

Completion: after all chunks are uploaded, the client sends a completion request. The server assembles the chunks into the final file.

On resume after a connection drop: the client queries “which chunks have you received for upload ID X?” and uploads only the missing ones.

POST /uploads/initiate
  → { "upload_id": "abc123", "chunk_size_bytes": 5242880 }

PUT /uploads/abc123/chunk/0
  Content-Range: bytes 0-5242879/524288000
  Body: [5MB of file data]

PUT /uploads/abc123/chunk/1
  Content-Range: bytes 5242880-10485759/524288000
  Body: [5MB of file data]

POST /uploads/abc123/complete
graph TD A[Client: 500MB file] --> B[Split into 100 chunks of 5MB] B --> C[Upload chunk 0 - success] B --> D[Upload chunk 1 - success] B --> E[Upload chunk 2 - connection drops] E --> F[Client reconnects] F --> G[Query: which chunks received for upload_id?] G --> H[Server: chunks 0 and 1 received] H --> I[Client resumes from chunk 2] I --> J[All chunks uploaded, send complete] style A fill:#000000,stroke:#00ff00,stroke-width:2px,color:#fff style B fill:#000000,stroke:#00ff00,stroke-width:2px,color:#fff style C fill:#000000,stroke:#00ff00,stroke-width:2px,color:#fff style D fill:#000000,stroke:#00ff00,stroke-width:2px,color:#fff style E fill:#000000,stroke:#ff0000,stroke-width:2px,color:#fff style F fill:#000000,stroke:#00ff00,stroke-width:2px,color:#fff style G fill:#000000,stroke:#00ff00,stroke-width:2px,color:#fff style H fill:#000000,stroke:#00ff00,stroke-width:2px,color:#fff style I fill:#000000,stroke:#00ff00,stroke-width:2px,color:#fff style J fill:#000000,stroke:#00ff00,stroke-width:2px,color:#fff

Server-Side Storage During Upload#

Chunks need to be stored durably as they arrive so they survive server restarts. Each chunk is written to object storage (S3 or equivalent) immediately, not held in memory. The completion step assembles them using the object storage’s multipart copy API, which avoids downloading and re-uploading the data.

AWS S3 multipart uploads implement exactly this pattern: initiate, upload parts, complete. The completion step runs entirely server-side, assembling parts into the final object without moving data back to the client.

At Salesforce#

We processed large data imports: customers uploading millions of rows of CSV data to be imported into their org. Files could be gigabytes. We had a 50MB limit initially, which frustrated enterprise customers with large datasets. When we increased the limit and added chunked uploads, the new challenge was partially-completed imports: if an import job failed after processing 60% of the rows, should we roll back the 60% or let the customer resume from row 600001? We chose idempotent chunk processing with a progress marker, similar to checkpointing. Resume from the last successfully processed chunk, not from zero.

What I’m Learning#

Resumable uploads are checkpointing applied to the upload process. The principle is identical: make progress durable so failures don’t require starting over. The implementation detail is that the checkpoint lives at the server, not the client.

Have you implemented resumable uploads or processing in your systems, and what was the hardest part to get right?