Diagnostic Runbook
Permissioned Retrieval
Locate where restricted evidence enters candidates, derivatives, caches, or traces before policy can contain it.
Start with the common causes
- 1
Policy runs after candidate generation
Visible titles are correct while counts or suggestions still reveal withheld records.
How to confirm → - 2
Cache reuse ignores authorization class
Another profile's search changes the current profile's result.
How to confirm → - 3
Secure filtering damages eligible recall
Approximate search returns too few useful records inside the allowed subset.
How to confirm →
Browse all symptoms
Candidate boundary
Caches and derivatives
Retrieval quality
Audit data
Deeper inspection and interventions
Boundary definition
This stage decides which records and fields may enter candidate generation for the current principal, action, and context. A correct-looking final list is insufficient evidence because restricted records can influence counts, facets, suggestions, ranking features, caches, prompts, and traces before display filtering runs.
First response
Capture the principal reference, action, policy version, scope reference, query, retrieval path, candidate IDs before ranking, visible result IDs, cache key dimensions, and trace field names. Avoid copying bearer credentials or restricted payloads into the incident record.
Reproduce with a known restricted record and a known eligible record. If the restricted record enters the pre-ranking candidate set, stop tuning ranking or generation and repair enforcement first.
Symptom catalog
late_filter_leakage: Policy runs after candidate generation
Signal. The visible list is correct while total hits, facets, suggestion terms, highlights, or trace identifiers differ when restricted records are present.
Confirm. Record observables before the display filter, then run the same query against a physically eligible-only fixture. Any difference attributable to restricted records confirms the leak.
Correct. Move policy into candidate generation. Recompute derived artifacts from authorized inputs and invalidate contaminated caches.
unauthorized_candidate_entry: Restricted records reach downstream stages
Signal. A restricted ID appears in the ranking window, reranker input, model context, or generated citations.
Confirm. Join candidate IDs to the decision log for the same principal, action, policy version, and request time.
Correct. Enforce the decision in every retrieval path. Treat an indeterminate decision as ineligible unless a documented policy says otherwise.
scope_blind_cache: Cache reuse ignores authorization class
Signal. Results or generated answers change after another profile searches the same normalized query.
Confirm. Inspect the cache key and stored authorization class. Reproduce a hit across two different scopes.
Correct. Partition by authorization class or include a stable scope fingerprint, policy version, tenant, schema, and other eligibility inputs in the key.
filtered_ann_recall_collapse: Secure filtering damages eligible recall
Signal. The vector path returns fewer than k eligible records or omits known relevant eligible neighbors.
Confirm. Compare the approximate result with exact vector search over the same eligible subset. Report eligible recall separately from boundary enforcement.
Correct. Tune filtered traversal, candidate depth, partitioning, or exact-search fallback for selective scopes. Do not widen the authorization filter to recover recall.
field_mask_after_embedding: A protected field already shaped a derivative
Signal. A hidden field still changes similarity, ranking features, cluster membership, or suggestions.
Confirm. Rebuild a controlled record without the protected field and compare its derivatives.
Correct. Generate governed derivatives only from fields authorized for that use, or place the derivative behind the source field’s access boundary.
restricted_trace_payload: Audit data copies protected inputs
Signal. Traces contain raw claims, access tokens, restricted passages, or complete candidate payloads.
Confirm. Inventory emitted fields at retrieval, ranking, generation, and policy spans. Check sampling and export paths too.
Correct. Retain references, categorical outcomes, reason codes, policy versions, and timings. Put sensitive payload capture behind separate access, retention, and redaction controls.
Diagnostic questions
- Does the authorization decision run before lexical, dense, and hybrid candidate IDs are returned?
- Do document, row, and field controls protect features, snippets, prompts, and generated evidence as well as titles?
- Do cache partitions or keys include every input that changes eligibility?
- Does exact search over the eligible subset recover records that filtered ANN misses?
- Can logs explain the decision through references and reason codes without storing protected claims or content?
Exit criteria
- Restricted records never enter ranking, generation, shared caches, or unrestricted traces.
- Lexical, dense, and hybrid paths enforce the same policy decision.
- Eligible recall is measured against an exact authorized baseline.
- Decision events remain attributable without copying credentials or protected content.