Slice the file into chunks, upload them straight to storage with presigned URLs, and let the server track which parts arrived so the upload can resume.
Requirements:
- drag and drop, or pick files
- multiple uploads
- progress per file
- pause, resume and cancel
- survive network drops and page reloads
Non-functional:
- Do not freeze the UI.
- Do not load whole files into memory.
- Keep upload traffic off the app server.
Architecture: an Uploader UI sits on top of an upload manager (a class or store). The manager owns a queue and a concurrency limit (how many uploads run at once). It also holds a state machine per file, with these states: queued, uploading, paused, failed, done.
API:
POST /uploadscreates an upload session.- It returns an id, plus presigned URLs for the parts (S3 multipart style) or a resumable endpoint (the tus protocol). A presigned URL lets the browser upload straight to storage for a limited time.
- The client slices the file with
Blob.slice()into chunks of, say, 5 to 10 MB. - It uploads several chunks in parallel.
- Finally it calls
completewith the checksums of the parts.
Resume: after a network drop or a reload, ask the server which parts it already has. Upload only the missing ones. Upload session ids are stored in IndexedDB, keyed by file name, size and last-modified time.
Optimisations:
- Retry each chunk with backoff (longer waits between tries).
- Compute checksums in a Web Worker, so the UI does not freeze.
- Track progress from upload events.
- Warn with
beforeunloadwhile uploads are running.
Trade-offs: larger chunks mean fewer requests, but more work is lost when one fails.
- Split the file with
Blob.slice()and upload the parts in parallel. - Upload directly to storage with presigned URLs.
- The server tracks which parts arrived, so uploads can resume.
- Use a queue with a concurrency limit and a state machine per file.
- Retry each chunk with backoff, and hash files in a Web Worker.
- MDN: Blob.slice() (https://developer.mozilla.org/en-US/docs/Web/API/Blob/slice)developer.mozilla.org