Skip to main content

Engine

WIP

The phase state machine, scheduler, event layer, and retry model.

The engine is the store's brain: a strict phase state machine, a scheduler that keeps work flowing within concurrency limits, a single event-emission layer, and a centralized retry policy. All of it lives under the Engine namespace and runs through one reducer.

Phases

Every item carries a phase discriminant. Engine.Phases is the const map; Engine.Phase is its union.

PhaseMeaning
validatingRunning validation rules
creating_intentCalling api.createIntent()
readyIntent received; waiting for start/autoStart
queuedWants to upload; waiting for a concurrency slot
uploadingTransferring bytes
pausedPaused; resumable cursor preserved
completingCalling api.complete()
completedDone and finalized (completedBy: 'upload' | 'dedupe')
errorTerminal-for-now failure (may be retryable)
canceledCanceled by the user

Engine.Item is a discriminated union over phase, so checking item.phase === 'uploading' narrows the type and gives you item.progress, item.intent, and friends.

State machine

Loading diagram...

Scheduling

After every state change the scheduler looks for work it can start without exceeding limits:

  • Intent creation for creating_intent items.
  • Upload slots for queued items, up to maxConcurrentUploads.
  • Finalization for completing items.

When more items are ready than there are slots, the extras sit in queued until a slot frees.

The event layer

State-derived public events emit from one place after transitions, never from inside individual handlers. This guarantees no duplicate events and consistent ordering. Internal reducer events map to the public Engine.EventMap names you subscribe to with store.on(...):

Internal eventPublic event(s)
validation.okvalidation.ok, then intent.creating
intent.okintent.created
upload.beginupload.started
upload.progressupload.progress
cursor.updatedupload.cursor
upload.okupload.completing
dedupe.ok / complete.okupload.completed

file.rejected emits during addFiles, because rejected files never enter the state machine.

Public events

The events actually emitted are: file.added, file.rejected, validation.ok/validation.failed, intent.creating/intent.created/intent.failed, upload.queued, upload.resumed, upload.started, upload.progress, upload.cursor, upload.paused, upload.canceled, upload.completing, upload.completed, and upload.error.

Engine.EventMap also declares upload.removed, rebind.ok, and rebind.failed, but the engine does not currently emit them. Don't rely on subscribing to those. After a rebind, read the snapshot to see whether item.file was attached.

Retry model

Retry decisions are centralized. When a phase fails, the engine consults config.retryPolicy (or the default exponential backoff) for an Engine.RetryDecision, and maxAttempts caps the total. Handlers never hardcode retry logic; they ask for a decision. Where a retry re-enters depends on how far the item got:

  • No intent yet → back to creating_intent.
  • Has an intent, bytes still pending → back to ready (re-queued).
  • Failed during finalization → back to completing (retries only the finalize call).

This means a retry never re-uploads bytes that were already confirmed.

Reading state

store.getSnapshot() returns an immutable Engine.State whose items is a Map<string, Engine.Item>:

for (const item of store.getSnapshot().items.values()) {
  if (item.phase === 'uploading') console.log(item.progress.pct)
  if (item.phase === 'error') console.log(item.error.message, item.retryable)
}

Awaiting outcomes

store.waitFor(localIds) resolves with Engine.Outcome values once each item reaches a terminal phase: completed, error, canceled, or missing.