Skip to main content

Chapter 6: Persistence & Offline

WIP

Survive reloads and crashes: persist upload state, restore it, and rebind files to resume.

Goal

A user drops 20 photos, the tab crashes, they reopen — the uploads should still be there, paused with progress intact. Add persistence so in-flight uploads survive reloads, and wire up rebind so users can re-select files and resume.

Loading diagram...

Choose an adapter

Pick an adapter

All adapters ship from @gentleduck/upload/core (there are no /persistence/* subpaths):

src/upload.ts
import { IndexedDBAdapter, LocalStorageAdapter, MemoryAdapter } from '@gentleduck/upload/core'
AdapterStorageSync?Best for
IndexedDBAdapterIndexedDBasyncProduction, large/many uploads
LocalStorageAdapterlocalStoragesyncSmall apps (~5 MB cap)
MemoryAdapterRAMsyncTests, SSR

Each implements UploadPersistence.Adapter:

type Adapter = {
  load(key: string): unknown | null | Promise<unknown | null>
  save(key: string, snapshot: unknown): void | Promise<void>
  clear(key: string): void | Promise<void>
}

Configure persistence

The default deserializer reads untrusted storage, so it needs isPurpose and isIntent guards:

src/upload.ts
import { IndexedDBAdapter } from '@gentleduck/upload/core'

export const store = createUploadStore<PhotoIntents, PhotoCursors, PhotoPurpose, PhotoResult>({
  api,
  strategies,
  persistence: {
    key: 'photoduck-uploads',
    version: 1,
    adapter: IndexedDBAdapter,
    debounceMs: 200,
    isPurpose: (v): v is PhotoPurpose => v === 'photo',
    isIntent: (v): v is PhotoIntents[keyof PhotoIntents] =>
      typeof v === 'object' && v !== null && 'strategy' in v && 'fileId' in v,
  },
})
OptionRequiredDescription
keyYesStorage key
versionYesSchema version (bump on shape changes)
adapterYesThe UploadPersistence.Adapter
debounceMsNoDelay before writing (batches progress bursts)
isPurpose / isIntentWith default deserializerGuards for untrusted data
serialize / deserializeNoCustom mappers
</Step>

What restores

On load, the store hydrates from the snapshot. Restored items come back paused with no File:

{
  phase: 'paused',
  localId: 'abc-123',
  purpose: 'photo',
  fingerprint: { name: 'sunset.jpg', size: 5_242_880, type: 'image/jpeg', lastModified: 1710000000000 },
  intent: { strategy: 'multipart', fileId: 'file-456', uploadId: 'upl-789', partSize: 5_242_880, partCount: 2 },
  // The cursor surfaces wrapped: { strategy, value: <raw cursor> }
  cursor: { strategy: 'multipart', value: { done: [{ partNumber: 1, etag: '"abc"', size: 5_242_880 }] } },
  progress: { uploadedBytes: 5_242_880, totalBytes: 10_485_760, pct: 50 },
  pausedAt: 1710000000000,
  createdAt: 1709999000000,
  file: undefined, // File objects can't be serialized
}

Rebind to resume

File is missing after restore, so the user re-selects it. rebind computes the new file's fingerprint and compares name/size/lastModified against the stored one (not type). A mismatch is a silent no-op — the item stays paused without a file.

src/RebindPrompt.tsx
import { useUploaderActions } from '@gentleduck/upload/react'
import type { Engine } from '@gentleduck/upload/core'

type Item = Engine.Item<PhotoIntents, PhotoCursors, PhotoPurpose, PhotoResult>

function RebindPrompt({ item }: { item: Item }) {
  const { dispatch } = useUploaderActions<PhotoIntents, PhotoCursors, PhotoPurpose, PhotoResult>()
  if (item.phase !== 'paused' || item.file) return null

  return (
    <div>
      <strong>{item.fingerprint.name}</strong> was interrupted at{' '}
      {Math.round(item.progress.pct)}%. Re-select it to continue.
      <input
        type="file"
        onChange={(e) => {
          const file = e.target.files?.[0]
          if (file) dispatch({ type: 'rebind', localId: item.localId, file })
        }}
      />
    </div>
  )
}

rebind sets the file synchronously in the reducer, so to auto-resume, check the snapshot right after dispatching (resume is a no-op if the file wasn't attached):

dispatch({ type: 'rebind', localId, file })
if (store.getSnapshot().items.get(localId)?.file) {
  dispatch({ type: 'resume', localId })
}

Prune stale items

The serializer only persists items that have an intent and are non-terminal. completed, canceled, and error items are excluded. Tune in-memory retention with config:

src/upload.ts
config: {
  maxItems: 100,
  completedItemTTL: 60_000, // evict completed items after 60s
}

Clear everything manually via the adapter:

await IndexedDBAdapter.clear('photoduck-uploads')

What is (and isn't) persisted

The serializer produces a UploadPersistence.Snapshot of UploadPersistence.PersistedItems:

type Snapshot<M, C, P> = {
  version: number
  createdAt: number
  items: Record<string, PersistedItem<M, C, P>>
}

type PersistedItem<M, C, P> = {
  id: string        // localId
  purpose: P
  status: string    // phase at save time
  file: { name: string; size: number; type: string; lastModified: number; checksum?: string }
  intent: Contracts.Intent.Any<M>
  cursor?: Contracts.Cursor.Any<C>
  progress?: { uploadedBytes: number; totalBytes: number; pct?: number }
}

Persisted: localId, purpose, phase, fingerprint, intent, cursor, progress. Not persisted: the File, in-flight network state, live timestamps (pausedAt is reset on restore).

Why File objects can't be serialized

A File is backed by an OS handle — it JSON-serializes to {}. The engine stores the fingerprint so it can confirm the correct file on rebind.

Deserialization checks

For each stored item the default deserializer: validates the snapshot shape, checks isPurpose, checks isIntent, verifies the intent's strategy is registered, and verifies the cursor's strategy matches. Anything that fails is silently skipped — safe for snapshots from older app versions.

Checkpoint

src/RestoredUploads.tsx
import { useUploader } from '@gentleduck/upload/react'

export function RestoredUploads() {
  const { items, dispatch } = useUploader<PhotoIntents, PhotoCursors, PhotoPurpose, PhotoResult>()
  const needsRebind = items.filter((i) => i.phase === 'paused' && !i.file)
  const canResume = items.filter((i) => i.phase === 'paused' && i.file)

  return (
    <div>
      {needsRebind.length > 0 && (
        <div>
          <h3>{needsRebind.length} upload(s) need re-selection</h3>
          <input
            type="file"
            multiple
            onChange={(e) => {
              const files = Array.from(e.target.files ?? [])
              // rebind checks each file's fingerprint; only matches are accepted
              for (const item of needsRebind) {
                for (const file of files) dispatch({ type: 'rebind', localId: item.localId, file })
              }
            }}
          />
        </div>
      )}
      {canResume.length > 0 && (
        <button onClick={() => dispatch({ type: 'startAll' })}>Resume all ({canResume.length})</button>
      )}
      <ul>
        {items.map((item) => (
          <li key={item.localId}>
            {item.fingerprint.name}{item.phase}
            {'progress' in item && item.progress && ` ${Math.round(item.progress.pct)}%`}
            {item.phase === 'paused' && !item.file && ' · needs file'}
          </li>
        ))}
      </ul>
    </div>
  )
}

Chapter 6 FAQ


Next: Chapter 7: Validation & Plugins