Uploading: chunks, retries and your server

A whole file#

By default each file is one POST of multipart/form-data — what every server already accepts:

POST /upload
Content-Type: multipart/form-data; boundary=…

  file: the file, under `fieldName`
  …any `data` fields

// Laravel
public function store(Request $request)
{
    $request->validate(['file' => ['required', 'file', 'max:5120']]);
    $path = $request->file('file')->store('uploads');

    return response()->json(['path' => $path]);   // whatever you return reaches onSuccess
}

Whatever the server answers with reaches onSuccess(file, response) as text, and stays on the file as file.response. getResponses() collects them all — which is how you post a form of ids after the files have gone up.

In pieces#

createFileUploader(input, { url: '/upload', chunkSize: 5_000_000 });

Each piece is a plain POST — no envelope, no base64, no tus — so the bytes on the wire are the bytes of the file:

The body is the raw slice, as application/octet-stream. The last piece's response is the response — the server answers once, at the end, exactly as it would for a whole file.

Resuming#

Before the first piece, the client sends HEAD to the same URL with X-Upload-Id and reads:

X-Uploaded-Bytes: 10485760

and starts from there. A server that does not answer, or answers 0, simply gets the whole file. Turn the probe off with resume: false.

A receiver, in full#

Route::match(['head', 'post'], '/upload', function (Request $request) {
    $id = preg_replace('/[^\w.-]/', '', $request->header('X-Upload-Id', ''));
    abort_if($id === '', 400);
    $path = storage_path("app/chunks/{$id}.part");

    if ($request->isMethod('head')) {
        return response('', 200, ['X-Uploaded-Bytes' => is_file($path) ? filesize($path) : 0]);
    }

    // Append this piece where it says it goes, so a repeat is harmless.
    File::ensureDirectoryExists(dirname($path));
    $handle = fopen($path, 'c');
    fseek($handle, (int) $request->header('X-Chunk-Offset', 0));
    fwrite($handle, $request->getContent());
    fclose($handle);

    $last = (int) $request->header('X-Chunk-Index') === (int) $request->header('X-Chunk-Count') - 1;
    if (! $last) {
        return response()->noContent();
    }

    $name = urldecode($request->header('X-File-Name', 'upload'));
    $final = 'uploads/'.Str::random(40).'.'.pathinfo($name, PATHINFO_EXTENSION);
    Storage::put($final, fopen($path, 'r'));
    unlink($path);

    return response()->json(['path' => $final]);
});

Nothing in that is specific to this package: it is a file, an offset, and a write. The browser test suite ships an equivalent in twenty lines of Node, and the tests prove resume works against it.

In production, check the size against X-File-Size before you accept the first piece, cap what one id may write, and sweep old .part files on a schedule.

When something goes wrong#

A failure is retried only when retrying could help:

retries is how many extra tries (default 2), each waiting twice as long as the last. After that the row shows a Try again button.

createFileUploader(input, {
  url: '/upload',
  retries: 3,
  onError: (file, message) => toast(`${file.file.name}: ${message}`),
});

Changing a file before it is sent#

createFileUploader(input, {
  url: '/upload',
  transform: async (file) => {
    if (!file.type.startsWith('image/')) return file;
    const bitmap = await createImageBitmap(file);
    const scale = Math.min(1, 1600 / Math.max(bitmap.width, bitmap.height));
    const canvas = new OffscreenCanvas(bitmap.width * scale, bitmap.height * scale);
    canvas.getContext('2d').drawImage(bitmap, 0, 0, canvas.width, canvas.height);
    return canvas.convertToBlob({ type: 'image/webp', quality: 0.82 });
  },
});

Half the bandwidth of a phone-camera upload, in twelve lines and no dependency. The widget does not ship an image codec of its own: the browser already has several.

A token that must stay fresh#

createFileUploader(input, {
  url: '/upload',
  headers: () => ({ Authorization: `Bearer ${auth.token()}` }),
});

headers and data are read again for every request, so a long upload does not carry the token it started with.