Skip to main content

Chapter 8: Production Patterns

WIP

Typed results, checksum deduplication, multiple purposes, and a testing strategy.

Goal

Ship PhotoDuck. Add typed results, checksum deduplication, multiple purposes with distinct configs, and a testing approach — tying Chapters 1–7 into a production setup.

Loading diagram...

Typed results

Extend Contracts.Result.Base

src/types/upload.ts
import type { Contracts } from '@gentleduck/upload/core'

// Contracts.Result.Base = { fileId: string; key: string }
export type PhotoDuckResult = Contracts.Result.Base & {
  url: string
  thumbnailUrl: string
  width: number
  height: number
}

R flows through the whole store. With createUploadStore<M, C, P, PhotoDuckResult>, every completed item exposes item.result: PhotoDuckResult — no casts.

Type the intent and cursor maps

Build intents on Contracts.Intent.Base; the cursor map holds raw cursor types.

src/types/upload.ts
import type { Contracts } from '@gentleduck/upload/core'
import type { PostStrategy, MultipartStrategy } from '@gentleduck/upload/strategies'

export type PhotoDuckIntents = {
  post: PostStrategy.Intent
  multipart: MultipartStrategy.Intent
}
export type PhotoDuckCursors = {
  post?: PostStrategy.Cursor
  multipart?: MultipartStrategy.Cursor
}
export type PhotoDuckPurpose = 'photo' | 'avatar' | 'document'

These guarantee: createIntent returns a valid intent shape, a paused item's cursor matches its strategy, addFiles({ purpose: 'invalid' }) is a compile error, and item.result is PhotoDuckResult.

Implement the typed contract

Every method takes args and ctx; use ctx.signal to stay cancelable.

src/lib/api.ts
import type { Contracts } from '@gentleduck/upload/core'
import type { PhotoDuckIntents, PhotoDuckPurpose, PhotoDuckResult } from '../types/upload'

export const photoDuckApi: Contracts.Api.Me<PhotoDuckIntents, PhotoDuckPurpose, PhotoDuckResult> = {
  async createIntent({ purpose, contentType, size, filename, checksum }, ctx) {
    const res = await fetch('/api/uploads/intent', {
      method: 'POST',
      headers: { 'content-type': 'application/json' },
      body: JSON.stringify({ purpose, contentType, size, filename, checksum }),
      signal: ctx.signal,
    })
    if (!res.ok) throw new Error(`intent failed: ${res.status}`)
    return res.json() // PostStrategy.Intent | MultipartStrategy.Intent
  },
  async complete({ fileId, filename, contentType, size }, ctx) {
    const res = await fetch(`/api/uploads/${fileId}/complete`, {
      method: 'POST',
      headers: { 'content-type': 'application/json' },
      body: JSON.stringify({ filename, contentType, size }),
      signal: ctx.signal,
    })
    if (!res.ok) throw new Error(`complete failed: ${res.status}`)
    return res.json() // PhotoDuckResult
  },
  async findByChecksum({ checksum, purpose }, ctx) {
    const res = await fetch(`/api/uploads/find?checksum=${checksum}&purpose=${purpose}`, {
      signal: ctx.signal,
    })
    if (!res.ok) return null
    return (await res.json()) as PhotoDuckResult | null
  },
  multipart: {
    async signPart({ fileId, uploadId, partNumber }, ctx) {
      const res = await fetch('/api/uploads/multipart/sign', {
        method: 'POST',
        headers: { 'content-type': 'application/json' },
        body: JSON.stringify({ fileId, uploadId, partNumber }),
        signal: ctx.signal,
      })
      return res.json()
    },
    async completeMultipart({ fileId, uploadId, parts }, ctx) {
      await fetch('/api/uploads/multipart/complete', {
        method: 'POST',
        headers: { 'content-type': 'application/json' },
        body: JSON.stringify({ fileId, uploadId, parts }),
        signal: ctx.signal,
      })
    },
  },
}

createIntent is the backend's decision point: it inspects size/purpose and returns the right intent. The engine never picks the strategy — the backend does.

Deduplicate by checksum

If the fingerprint carries a checksum and findByChecksum is implemented, the engine checks for an existing file during validation and, on a match, completes the item instantly with completedBy: 'dedupe' — no intent, no bytes.

The default fingerprint has no checksum, and the fingerprint option must be synchronous. So compute hashes before adding files, then look them up in fingerprint:

src/lib/add-with-checksum.ts
async function sha256Hex(file: File): Promise<string> {
  const buf = await crypto.subtle.digest('SHA-256', await file.arrayBuffer())
  return [...new Uint8Array(buf)].map((b) => b.toString(16).padStart(2, '0')).join('')
}

const checksums = new Map<File, string>()

export async function addWithChecksum(files: File[], purpose: PhotoDuckPurpose) {
  await Promise.all(files.map(async (f) => checksums.set(f, await sha256Hex(f))))
  store.dispatch({ type: 'addFiles', files, purpose })
}

// In the store:
// fingerprint: (file) => ({
//   name: file.name, size: file.size, type: file.type,
//   lastModified: file.lastModified, checksum: checksums.get(file),
// }),

Contracts.FingerprintFile is { name, size, type, lastModified, checksum? }. For very large files, respect config.checksumMaxSize (the built-in checksum step skips files above it).

Multiple purposes

Each purpose gets its own rules, and autoStart can target specific ones:

src/lib/upload.ts
createUploadStore<PhotoDuckIntents, PhotoDuckCursors, PhotoDuckPurpose, PhotoDuckResult>({
  api: photoDuckApi,
  strategies,
  config: {
    maxConcurrentUploads: 3,
    autoStart: (purpose) => purpose === 'avatar', // or ['avatar']
    validation: {
      photo:    { maxFiles: 50, maxSizeBytes: 10 * 1024 * 1024, allowedTypes: ['image/*'] },
      avatar:   { maxFiles: 1,  maxSizeBytes: 2 * 1024 * 1024,  allowedTypes: ['image/jpeg', 'image/png', 'image/webp'] },
      document: { maxFiles: 10, maxSizeBytes: 50 * 1024 * 1024, allowedTypes: ['application/pdf'], allowedExtensions: ['pdf'] },
    },
  },
})

Scope batch commands by purpose, and filter items by purpose in the UI:

dispatch({ type: 'startAll', purpose: 'photo' })
const photos = items.filter((i) => i.purpose === 'photo')

Testing

Mock at the Contracts.Api.Me level and use MemoryAdapter for persistence.

Mock the contract

src/test/mock-api.ts
import type { Contracts } from '@gentleduck/upload/core'
import type { PhotoDuckIntents, PhotoDuckPurpose, PhotoDuckResult } from '../types/upload'

export function createMockApi(
  overrides?: Partial<Contracts.Api.Me<PhotoDuckIntents, PhotoDuckPurpose, PhotoDuckResult>>,
): Contracts.Api.Me<PhotoDuckIntents, PhotoDuckPurpose, PhotoDuckResult> {
  return {
    createIntent: overrides?.createIntent ?? (async ({ filename }) => ({
      strategy: 'post',
      fileId: `file-${Date.now()}`,
      url: 'https://storage.example.com/upload',
      fields: { key: `uploads/${filename}` },
    })),
    complete: overrides?.complete ?? (async ({ fileId }) => ({
      fileId,
      key: `uploads/${fileId}`,
      url: `https://cdn.example.com/${fileId}`,
      thumbnailUrl: `https://cdn.example.com/${fileId}/thumb`,
      width: 1920,
      height: 1080,
    })),
    findByChecksum: overrides?.findByChecksum ?? (async () => null),
  }
}

Test with MemoryAdapter and waitFor

src/test/upload.test.ts
import { describe, it, expect } from 'vitest'
import { createUploadStore } from '@gentleduck/upload/core'
import { PostStrategy, createStrategyRegistry } from '@gentleduck/upload/strategies'
import { createMockApi } from './mock-api'

const strategies = createStrategyRegistry<PhotoDuckIntents, PhotoDuckCursors, PhotoDuckPurpose, PhotoDuckResult>()
strategies.set(PostStrategy())

const file = (name: string, size: number, type: string) =>
  new File([new ArrayBuffer(size)], name, { type, lastModified: Date.now() })

describe('PhotoDuck uploads', () => {
  it('rejects oversized files before entering state', () => {
    const store = createUploadStore({
      api: createMockApi(),
      strategies,
      config: { validation: { photo: { maxSizeBytes: 1024 } } },
    })
    store.dispatch({ type: 'addFiles', files: [file('big.jpg', 2048, 'image/jpeg')], purpose: 'photo' })
    // Built-in size failures emit file.rejected and never enter state.
    expect(store.getSnapshot().items.size).toBe(0)
  })

  it('deduplicates by checksum', async () => {
    const existing: PhotoDuckResult = {
      fileId: 'existing', key: 'uploads/x', url: 'https://cdn/x',
      thumbnailUrl: 'https://cdn/x/t', width: 100, height: 100,
    }
    const store = createUploadStore({
      api: createMockApi({ findByChecksum: async () => existing }),
      strategies,
      config: { autoStart: ['photo'] },
      fingerprint: (f) => ({
        name: f.name, size: f.size, type: f.type, lastModified: f.lastModified, checksum: 'abc123',
      }),
    })
    store.dispatch({ type: 'addFiles', files: [file('sunset.jpg', 100, 'image/jpeg')], purpose: 'photo' })

    const id = [...store.getSnapshot().items.keys()][0]!
    const [outcome] = await store.waitFor([id])
    expect(outcome.status).toBe('completed')
    if (outcome.status === 'completed') {
      expect(outcome.completedBy).toBe('dedupe')
      expect(outcome.result.fileId).toBe('existing')
    }
  })
})

store.waitFor(ids) resolves with Engine.Outcome values once items reach a terminal phase:

// Engine.Outcome<R> =
//   | { localId; status: 'completed'; completedBy: 'upload' | 'dedupe'; result: R }
//   | { localId; status: 'error'; error: UploadError }
//   | { localId; status: 'canceled' }
//   | { localId; status: 'missing'; reason: 'removed' | 'evicted' | 'never-existed' | 'destroyed' }

How the generics flow

The four parameters thread through every layer:

createUploadStore<M, C, P, R>
  ├─ Contracts.Api.Me<M, P, R>
  │    createIntent -> M[keyof M]      complete -> R      findByChecksum -> R | null
  ├─ Contracts.Strategy.Registry<M, C, P, R>
  │    start(ctx) with intent M[K], cursor C[K]
  ├─ Store.UploadStore<M, C, P, R>
  │    getSnapshot().items -> Map<string, Engine.Item<M, C, P, R>>
  │      item.intent M[keyof M] · item.cursor · item.result R · item.purpose P
  ├─ Engine.Plugin<M, C, P, R>
  │    on('upload.completed', ({ result }) => …)  // result: R
  └─ Engine.EventMap<M, C, P, R>
       'upload.completed' -> { result: R; completedBy }
       'intent.created'   -> { intent: M[keyof M] }

So item.result.width is typed all the way from the R you declared.


Chapter 8 FAQ


You built PhotoDuck from an empty folder to a production upload system: pause/resume, persistence, validation, plugins, typed results, and deduplication. The engine carries the complexity so your product code stays focused.