Skip to main content

Troubleshooting

WIP

Real error messages from the source paired with the cause and the fix, plus how to diagnose a wrong decision that throws nothing

Most console.warn/console.error lines below are literal strings logged by @gentleduck/iam - search for the text between the brackets and the tag tells you which subsystem produced it. Thrown errors are IamErrors: search for the CODE shown instead, and read the detail off err.meta, since err.message is the bare code.

Start here

Pick your branch from the tag in the message, or from the fact that nothing was thrown at all.

Loading diagram...

The two branches under "Did something throw?" behave differently. A throw is a configuration or input problem and is always reproducible from a unit test; the tag in the message routes you to the right table below. A wrong decision with no throw is a policy problem, and the only reliable instrument is explain() - which is why the "mode development?" node exists: production mode never builds a trace, so a production-only bug has to be reproduced on a dev-mode replica before you can see targetMatch, conditionsMet, or the conditions array.

Diagnostic toolbox

ToolSignatureWhat it tells you
engine.explain()(subjectId, action, resource, environment?, scope?)Per-policy and per-rule trace: targetMatch, actionMatch, resourceMatch, conditionsMet, decidingRuleId. Development mode only.
access.validateRoles()(roles)IamValidate.IResult - { valid, issues } with closed-set codes for the whole role catalog, including cycles and dangling parents.
access.validatePolicy()(policy)Same shape, for one policy: unreachable targets, unresolvable fields, bad operators, limit breaches.
engine.stats.get()(){ policies, roles, rbacPolicy, mergedPolicies, subjects }, each { hits, misses, size }. Cache health, nothing else.
engine.healthCheck()(){ ok, adapter, cacheHitRate, adapterLatencyMs, lastError?, compiledTable?, invalidator? }. Adapter reachability, the last error the engine swallowed, and - only when the table could not be built - why. ok never reflects invalidator health, but it does read the invalidator's own status(): when the invalidator reports it is not subscribed, the result gains invalidator: { subscribed: false } alongside an ok: true.
engine.preload()({ validator?: boolean })Warms the caches at boot and surfaces adapter failures before the first request. Pass { validator: true } to run the validators over what the adapter returned.
engine.getEffectiveRoles()(subjectId, scope?)The flattened role list after inheritance and scope merging - the fastest way to prove a role is or is not reaching the subject.

Start with explain(). Nine times out of ten the trace names the rule and the failed condition outright.

Engine configuration and startup

SymptomCauseFix
IAM_ENGINE_FAIL_OPEN_NOT_CONFIRMED (no meta)new IamEngine({ defaultEffect: 'allow' }) without the companion flag. The constructor refuses to build.This is a guard, not a default - defaultEffect is 'deny' and should stay there. If you genuinely want an allow-by-default engine, add allowFailOpen: true and accept the startup warning. See fail-closed defaults.
[@gentleduck/iam:engine] engine configured with defaultEffect: 'allow' (fail-open). Every request with no applicable policy will be allowed. (console warning)The opt-in above succeeded. The warning fires on every boot by design so the configuration is greppable in logs.Nothing to fix if intentional. If it is not intentional, remove both defaultEffect: 'allow' and allowFailOpen.
IAM_ENGINE_POLICY_COMBINE_INCOMPATIBLE (meta: { mode, policyCombine })policyCombine: 'first-applicable' with mode: 'production'.Use 'and' or 'allow-overrides' in production, or keep the deployment on mode: 'development'.
IAM_ENGINE_INVALID_CONFIG (meta: { field: 'maxPolicies', got, constraint: 'finite number >= 1' })The value was NaN, Infinity, 0, or negative - usually Number(process.env.IAM_MAX_POLICIES) on an unset variable.Parse and default explicitly: Number(process.env.IAM_MAX_POLICIES ?? 10000). The same guard (same code, different meta.field) exists for maxRoles, adapterTimeoutMs, hookTimeoutMs, and maxConcurrentSubjectLoads.
IAM_ENGINE_INVALID_CONFIG (meta: { field: 'maxConcurrentSubjectLoads', got, constraint: '0 (unbounded) or finite number >= 1' })Same NaN path, or someone passed -1 meaning "off".0 is the "unbounded" sentinel, not -1.
IAM_REDIS_INVALIDATOR_TENANT_ID_INVALID (meta: { got })A tenant slug with a :, *, or ? in it. The shape check exists because those characters would inject into the pub/sub channel pattern.Slugify the tenant id before passing it.
IAM_METRICS_SAMPLE_SIZE_INVALID (meta: { got })iamCreateMetricsAggregator({ sampleSize }) with a float, zero, or NaN.Pass an integer; the default is fine for most deployments.
IAM_HTTP_BASE_URL_INVALID (meta.reason: 'bad-scheme' / 'has-query-or-fragment' / 'unparseable')Malformed policy-service URL.Give a bare origin plus path, no query.
IAM_HTTP_BASE_URL_INVALID (meta.reason: 'private-host', meta.host)SSRF guard. Pointing the HTTP adapter at localhost or an RFC1918 address is blocked unless you say so.For local development set allowPrivateHosts: true. In production, fix the URL.
[@gentleduck/iam:http] `allowedHosts` not set - any host accepted. Pass `init.allowedHosts` for SSRF defense in depth. (console warning)The HTTP adapter was constructed without a host allowlist.Pass allowedHosts: ['policies.internal'].
[@gentleduck/iam:invalidator:redis] `secret` not set on channel "<channel>" - accepting unsigned pub/sub. Anyone with PUBLISH rights on the channel can wipe caches. Pass `secret` to require HMAC-SHA256. (console warning)The Redis invalidator is running unsigned on that channel. The warning latches per channel, so one process running one invalidator per tenant reports each unsigned channel.Set secret in production. Without it, any client with PUBLISH on the channel can force every replica to flush and re-read - a cheap denial-of-service and a cache-poisoning primitive. See multi-instance invalidation.
[@gentleduck/iam:file] IamFileAdapter constructed without rootDir. Any caller deriving the path from request data should set rootDir for defence in depth. (console warning)No rootDir containment on the file adapter.Pass rootDir; the adapter then rejects any path that escapes it.

Roles and policies rejected at build time

SymptomCauseFix
IAM_VALIDATION_FAILED (meta: { kind: 'policy', issues: string[] })build() runs validatePolicy() and refuses to return an invalid policy. Each issues entry is `<MathMl mathml="<span class=&quot;katex&quot;><span class=&quot;katex-mathml&quot;><math xmlns=&quot;http://www.w3.org/1998/Math/MathML&quot;><semantics><mrow><mi>c</mi><mi>o</mi><mi>d</mi><mi>e</mi></mrow><annotation encoding=&quot;application/x-tex&quot;>{code}</annotation></semantics></math></span><span class=&quot;katex-html&quot; aria-hidden=&quot;true&quot;><span class=&quot;base&quot;><span class=&quot;strut&quot; style=&quot;height:0.6944em;&quot;></span><span class=&quot;mord&quot;><span class=&quot;mord mathnormal&quot;>co</span><span class=&quot;mord mathnormal&quot;>d</span><span class=&quot;mord mathnormal&quot;>e</span></span></span></span></span>"/>{where}: ${message}`, the real diagnosis.Read the codes in the tables below. Call access.validatePolicy(draft) in a test to get the structured issues instead of a concatenated string.
IAM_VALIDATION_FAILED (meta: { kind: 'role', issues: string[] })Same for roles - RoleBuilder.build() and PolicyBuilder.build() now share the identical throw and format.Same.
UNREACHABLE_TARGET - Target admits "X" on "Y" but no allow rule covers it, so every request matching it is denied by this policy. Add a rule that allows it, or narrow the target.The policy target is wider than its rules. Because the policy is applicable but has no allow, it falls to defaultEffect and vetoes the request under policyCombine: 'and'.This is an error, not a warning - it fails the build. Either add the missing allow rule or shrink the target. The single most common cause of "my role grants it but the check still denies".
DANGLING_INHERIT - Role "X" inherits from "Y" which does not exist.inherits('viewer') where no role with id viewer is in the set being validated. Inheritance takes ids, not role objects.Validate the whole catalog together: access.validateRoles(allRoles), not role by role.
CIRCULAR_INHERIT - Circular inheritance detected involving role "X" (cycle includes "Y")An inheritance cycle.Break the edge named in the message.
INHERITANCE_TOO_DEEP - Role "X" has an inheritance chain N deep; the runtime caps at 32 and silently drops anything past itThe chain exceeds MAX_INHERITANCE_DEPTH (32, not 16).Flatten the hierarchy. Anything past the cap is dropped at runtime without an error, which is why the validator is loud about it.
UNRESOLVABLE_FIELD - Condition field "X" has no resolvable root (expected subject/resource/environment, or shorthand action/scope)A dot-path rooted at something that does not exist, such as user.id or ctx.tenant. Only subject, resource, and environment are resolvable roots.Rewrite as subject.id, resource.attributes.ownerId, environment.ip. Use the when() helpers (attr, resourceAttr, env) so the prefix is added for you.
INVALID_RULE - Rule "priority" must be a finite number (NaN/Infinity break highest-priority ranking)A computed priority that came out NaN.Priorities must be finite. Only highest-priority reads them, but the validator checks always.
INVALID_RULE - Rule must have a "conditions" object (use `{ all: [] }` for an unconditional rule)A hand-written rule object with no conditions key.Builders always emit one; if you are constructing rules literally, add conditions: { all: [] }.
LIMIT_EXCEEDEDA policy blew POLICY_LIMITS: 1,000 rules per policy, 100 actions per rule, 100 resources per rule, 1,000 action-x-resource pairs per rule.The Cartesian cap usually bites first: .on(...10 actions).of(...110 resources) is 1,100 pairs. Split the policy.
INVALID_OPERATOR / INVALID_EFFECT / INVALID_ALGORITHM / INVALID_TYPEA string that is not in the closed set - typically from JSON loaded off an adapter rather than built with the builders.Validate adapter rows at boot with engine.preload({ validator: true }).
ERR_REGEX_CATASTROPHICA matches pattern with more than 4 unbounded quantifiers, or longer than 128 characters.Anchor the pattern and bound the quantifiers. Regexes are also cached, capped at 256 compiled entries.
BROAD_ALLOW / EMPTY_ROLEWarnings, not errors. valid stays true.Review, then ignore or fix. BROAD_ALLOW on an intentional superuser policy is expected.

Runtime denials and throws

SymptomCauseFix
IAM_ENGINE_EXPLAIN_UNAVAILABLEengine.explain() on an engine built with mode: 'production'. The compiled table has no trace to give.Reproduce on a development-mode engine over the same adapter, or keep one dev-mode replica out of the load balancer for diagnosis.
IAM_ROLE_LIMIT_EXCEEDED (meta: { roleCount, limit: 32 })Production mode packs role grants into a 32-bit mask. Beyond 32 roles the bit indices alias.Either drop to 32 roles or fewer, or run mode: 'development'. The throw happens inside authorize(), so it surfaces through onError and the request fails closed - you get denials, not silent aliasing. Watch for it in the first minutes after adding a role.
IAM_ENGINE_BATCH_TOO_LARGE (meta: { count, limit: 1024 })A permission map built from an unbounded list.Chunk the checks, or narrow what the page actually needs.
IAM_ENGINE_PARAM_INVALID (meta: { name: 'permissions(): subjectId' | 'explain(): subjectId', reason: 'empty' | 'too-long', got?, length? })An empty string, undefined coerced, or an oversized opaque id, passed to explain() or permissions().Resolve the identity before calling; do not pass a raw header value.
A malformed action, resource, or scope on can() / check()These do not throw. An invalid subjectId (empty, non-string, or over 1024 chars) is folded into the request instead and evaluates to a deny - the same fail-closed outcome as a throw, but silent.If a check denies for no policy reason you can find in explain(), verify the caller actually resolved a real subject id first.
Admin operations (savePolicy, assignRole, ...) called with an empty subjectId / roleId / idIAM_ENGINE_PARAM_INVALID (meta: { name, reason: 'empty', got }), unlike the request path above - these do throw.Validate the id before calling into engine.admin.
IAM_ENGINE_ATTRIBUTES_PARAM_INVALID (meta.reason: 'not-object' / 'too-many-keys' / 'too-deep')A whole ORM row or a deeply nested document was passed to engine.admin.setAttributes(): not a plain object, over 256 own keys, or nesting past depth 16.Project down to the fields your conditions actually read. This also makes the check faster.
`[@gentleduck/iam:engine] <MathMl mathml="<span class=&quot;katex&quot;><span class=&quot;katex-mathml&quot;><math xmlns=&quot;http://www.w3.org/1998/Math/MathML&quot;><semantics><mrow><mrow><mi>l</mi><mi>a</mi><mi>b</mi><mi>e</mi><mi>l</mi></mrow><mi>t</mi><mi>i</mi><mi>m</mi><mi>e</mi><mi>d</mi><mi>o</mi><mi>u</mi><mi>t</mi><mi>a</mi><mi>f</mi><mi>t</mi><mi>e</mi><mi>r</mi></mrow><annotation encoding=&quot;application/x-tex&quot;>{label} timed out after </annotation></semantics></math></span><span class=&quot;katex-html&quot; aria-hidden=&quot;true&quot;><span class=&quot;base&quot;><span class=&quot;strut&quot; style=&quot;height:0.8889em;vertical-align:-0.1944em;&quot;></span><span class=&quot;mord&quot;><span class=&quot;mord mathnormal&quot; style=&quot;margin-right:0.01968em;&quot;>l</span><span class=&quot;mord mathnormal&quot;>ab</span><span class=&quot;mord mathnormal&quot;>e</span><span class=&quot;mord mathnormal&quot; style=&quot;margin-right:0.01968em;&quot;>l</span></span><span class=&quot;mord mathnormal&quot;>t</span><span class=&quot;mord mathnormal&quot;>im</span><span class=&quot;mord mathnormal&quot;>e</span><span class=&quot;mord mathnormal&quot;>d</span><span class=&quot;mord mathnormal&quot;>o</span><span class=&quot;mord mathnormal&quot;>u</span><span class=&quot;mord mathnormal&quot;>t</span><span class=&quot;mord mathnormal&quot;>a</span><span class=&quot;mord mathnormal&quot; style=&quot;margin-right:0.10764em;&quot;>f</span><span class=&quot;mord mathnormal&quot;>t</span><span class=&quot;mord mathnormal&quot; style=&quot;margin-right:0.02778em;&quot;>er</span></span></span></span>"/>{adapterTimeoutMs}ms` (still a plain Error, not a coded IamError)An adapter call exceeded adapterTimeoutMs.The request denies (fail-closed). Fix the adapter or raise the timeout, and alert on the rate - a slow policy store turns into a site-wide outage of allows.
IAM_ENGINE_ROW_CAP_EXCEEDED (meta: { noun: 'policies' | 'roles', count, cap, capField })The policy store grew past the cap, or a query lost its tenant filter and returned every tenant's rows.Check the second cause first. Only raise maxPolicies/maxRoles once you have confirmed the count is legitimate.
IAM_ENGINE_SUBJECT_LOAD_SHED (meta: { subjectId, inFlight, cap })maxConcurrentSubjectLoads back-pressure. A cold cache plus a traffic spike stampedes the adapter.The shed request denies. Raise the cap, warm with preload(), raise cacheTTL, or fix whatever caused the mass invalidation.
IAM_CONDITION_REGEX_INPUT_TOO_LARGE (meta: { field, length })A matches condition was pointed at a large attribute (a body, a blob of JSON) past MAX_REGEX_INPUT_LENGTH (2048 bytes).The engine catches the throw and drops the policy as NotApplicable, not denied - under policyCombine: 'and' that means it stops voting entirely. Match against a short, bounded field.
IAM_ENGINE_SNAPSHOT_VERSION_UNSUPPORTED (meta: { got })admin.import() was handed a snapshot from a future version.Export and import with matching versions. ISnapshot.schemaVersion is 1.

Adapters and transports

SymptomCauseFix
IAM_HTTP_CIRCUIT_OPEN (meta: { state: 'open' })Five consecutive failures tripped the breaker; it refuses for circuitBreakerCooldownMs (30s default).The policy service is down. Every check in the window denies. Alert on this code.
IAM_HTTP_CIRCUIT_OPEN (meta: { state: 'half-open-busy' })Cooldown elapsed and one probe is testing the service; concurrent callers are refused rather than piling on.Transient by design. If it never clears, the service is still unhealthy.
IAM_HTTP_RESPONSE_ERROR (meta: { status: 401 | 403, body })The headers you passed the HTTP adapter are wrong or expired.Refresh the credential. Headers are static per instance - rebuild the adapter to rotate.
IAM_HTTP_RESPONSE_TOO_LARGE (meta: { capBytes })The policy endpoint returned more than the 4 MiB read cap.Paginate the endpoint or move to a database adapter.
IAM_HTTP_SUBJECT_ROLES_INVALID (meta: { subjectId, reason: 'not-array', got })The endpoint returned an object or a bare string instead of a JSON array.Return a JSON array of role ids.
IAM_REDIS_ASSIGNMENT_ENCODING_INVALID (meta: { field: 'role' | 'scope' })A NUL byte in a role or scope id - almost always a truncated buffer or a fuzzing input.Validate at the edge. Redis keys cannot represent it safely.
IAM_ATTRIBUTES_CORRUPT (meta: { adapter, subjectId, reason: 'parse-failed' | 'not-object' })Someone wrote to the attribute key outside the adapter. Same shared code fires with adapter: 'redis' | 'drizzle' | 'file' | 'prisma' | 'http'.Only write attributes through engine.admin.setAttributes().
`[@gentleduck/iam:redis] dropped malformed row "<id>": <message>` (console warning)One row failed shape validation on load. The rest of the catalog still loads.Fix the row. The engine deliberately does not fail the whole load for one bad row, so this warning is easy to miss - alert on it.
IAM_FILE_PATH_INVALID (meta.reason: 'escapes-rootdir' / 'dotdot-segment' / 'not-absolute' / 'not-resolvable-absolute' / 'rootdir-not-absolute')Path traversal containment on the file adapter.Pass an absolute path inside rootDir.
`[@gentleduck/iam:invalidator:redis] subscribe to "<channel>" failed (<message>) - this node receives no invalidations and serves stale allow decisions until a retry succeeds. ... Pass \`onSubscribeError\` to handle this.` (console warning)The pub/sub subscribe was rejected - NOAUTH, a bad ACL, an unreachable broker. The node receives nothing.Page on it. healthCheck() now surfaces this: ok stays true (the adapter is fine) but the response gains invalidator: { subscribed: false }. Until a later subscribe() succeeds - retried from publish() at a bounded interval, or from any further subscribe() call - this replica serves pre-revocation decisions for up to one cacheTTL.
`[@gentleduck/iam:invalidator:redis] dropping unverifiable message on channel "<channel>" (<reason>). <N> prior drops coalesced.` (console warning)An inbound invalidation was refused. The reason names which gate.Wire onMessageDropped and read the reason. See failure modes.

Framework and client integration

SymptomCauseFix
IAM_SERVER_ADMIN_AUTHORIZE_REQUIRED (meta: { framework, fn })iamAdminRouter(engine, {}) (Express). The same guard exists as iamBindAdminRouter (Hono), createIamAdminHandlers (Next), and createIamAdminOperations (Nest), each with its own meta.framework/meta.fn.Supply an authorize callback that checks the caller is an operator. Read-only admin endpoints leak your whole policy set; mutating ones let a caller grant themselves anything.
IAM_SERVER_GET_USER_ID_REQUIRED (meta: { framework: 'next' })withIamAccess() without getUserId.Derive the id from a verified session, never from x-user-id. See identity sourcing.
{ "error": "Forbidden (CSRF check failed)" } with HTTP 403 on an admin routeThe default Sec-Fetch-Site CSRF check rejected the request. Since 2.1.0 it is on by default.For a browser admin UI, send same-origin requests. For a bearer-token or mTLS API, pass csrfCheck: false. For a custom origin allowlist, pass a predicate.
`[@gentleduck/iam:generic] admin router: default CSRF check enabled - ... Pass \`csrfCheck: false\` for bearer-token/mTLS APIs, or supply a custom predicate. ...` (console notice, once per process)You did not pass csrfCheck at all, so the default is active.Make the choice explicitly and the notice stops.
[@gentleduck/iam:vue] useAccess() called without provideAccess(). Use provideAccess() in a parent component or install the plugin.The composable was called outside the provider tree.Wrap the app, or install the plugin.
IAM_CLIENT_PERMISSIONS_FETCH_FAILED (meta: { status }, vanilla client)The permissions endpoint returned a non-2xx.Check the endpoint URL and that the session cookie is being sent.
[@gentleduck/iam:engine] <hook> hook threw - swallowed to preserve decision (console error)One of your hooks threw. The decision is unaffected by design - a broken audit hook must not turn into a broken authorization system.Fix the hook. Alert on this string; a silently failing onDeny means a silently empty audit log.

The check is wrong but nothing threw

No error message means the engine did exactly what your policies said. Work the trace.

const trace = await engine.explain('user-bob', 'update', {
  type: 'post',
  id: 'post-1',
  attributes: { ownerId: 'user-bob' },
})

console.dir(trace, { depth: null })

Read trace.policies in order:

What you seeWhat it meansWhat to do
policies is emptyNo policy is applicable. The decision came from defaultEffect.Either the action/resource pair has no policy at all, or every policy's target excluded it.
targetMatch: falseThe policy's target did not admit this request.Widen the target - and re-check for UNREACHABLE_TARGET afterwards.
actionMatch: false or resourceMatch: false on every ruleThe policy is applicable by target but no rule covers the shape, so the policy returns applicable: false and abstains.Add the rule. Note the policy is not denying here - it is abstaining.
conditionsMet: false with the condition listedA condition evaluated false. The conditions array shows exactly which.The three usual causes are below.
decidingRuleId naming a deny you did not expectA rule matched under deny-overrides.Look at that rule's conditions, or reorder under a different algorithm.
The policy allowed but the request was still deniedAnother applicable policy denied. Under policyCombine: 'and' every applicable policy must allow.Scan the whole policies array, not just the one you were editing.

The three conditions that fail most often:

  • Strict equality. eq is === with no coercion. subject.id of '42' against resource.attributes.ownerId of 42 is false. So is 'Admin' against 'admin'.
  • Numeric-only comparison. gt, gte, lt, lte return false unless both operands are numbers. An ISO-8601 date compared with gte never matches. Use before / after, which coerce to epoch milliseconds.
  • A missing attribute. resolve() returns null for a path that does not exist, and only exists treats null specially. If the resource was passed as { type: 'post', attributes: {} }, every resource.attributes.* condition is comparing against null.

Checks are slower than expected

engine.stats.get() returns { hits, misses, size } for each of the five caches - policies, roles, rbacPolicy, mergedPolicies, subjects. There is no error counter; use healthCheck().lastError and the onError hook for that.

  • High subjects miss rate - only the subject cache is bounded by maxCacheSize (default 1000). If you have more active subjects than that, raise it.
  • High miss rate everywhere - cacheTTL (default 60 seconds, not milliseconds) is too short, or an invalidator is flushing more than it needs to.
  • size at zero after a while - something is calling engine.cache.invalidate() or publishing broad invalidation events. Check the invalidator channel.
  • Good hit rates but still slow - you are in mode: 'development', which allocates a full decision object per check. Production mode is roughly 7x faster. See modes and benchmarks.
  • A UI permission map dominating the profile - pass { telemetry: false } to engine.permissions() to skip per-check onMetrics.

Cold-start latency is a separate problem: call await engine.preload() at boot so the first real request does not pay the adapter round trip.

Getting help

See also