Skip to content
Search Engineering

Permissioned Retrieval

Make identity and policy part of candidate generation so restricted evidence cannot influence ranking, generation, caches, or traces.

Chapter 12 completed an inspectable agent trajectory. The path can now be evaluated for outcome, method, risk, and cost. That account still assumes one shared body of evidence.

Governed search removes that assumption. A query arrives with a principal, verified claims, a requested operation, and policy context. Those inputs determine which records may become candidates. Two people can issue the same query against the same index and receive different eligible sets without either result being an error.

The examples below use synthetic film certifications and household rules. They are teaching fixtures rather than claims about legal access or production corpus metadata. A table later in this section maps each fixture element to its enterprise equivalent.

Hold the query fixed

Keep the Family profile and Display filter selected, then open Observables. The visible titles follow the household rule, but restricted records have already changed the count, suggestions, cache, generated evidence, and trace. Switch enforcement to Candidate auth with a scoped cache. Those records now leave before ranking, and each derived observable inherits the same scope. The fixed query makes the enforcement point the only cause of the change.

  1. Interstellarrestricted
  2. The Terminatorrestricted
  3. Alienrestricted
  4. The Godfatherrestricted
  5. Ratatouilleeligible
  6. Festival cutrestricted
  7. Source pendingrestricted

Ratatouille

Household policy permits G and PG in this synthetic fixture.

The source explicitly supplied an unrated classification; policy decides whether it is eligible. No verified classification is available; the fixture denies eligibility until the source is resolved.

Synthetic certification fixture for teaching. It is not publication corpus data.

The result list alone cannot prove that retrieval was authorized. A display filter can hide titles after candidate generation while preserving their influence on score statistics, facets, vocabulary, suggestions, cache entries, generated context, and logs. The visible output can look correct while the pipeline has already observed restricted evidence.

The fixture also keeps unrated separate from unknown. Unrated means the source explicitly supplied that classification, so policy can permit or deny it. Unknown means no verified source value is available, so the record remains ineligible until the missing assertion is resolved. Treating both as an empty field would turn missing authorization evidence into an accidental fallback.

Permissioned retrieval moves the decision to the point where a record becomes eligible for search. The application constructs an authorization request with four inspectable inputs:

InputFilm fixtureEnterprise translation
Principalverified household profileemployee, customer, service identity
Actionretrieveread, search, summarize
Resourcefilm recordrow, document, field, object
Contextterritory, policy version, timetenant, purpose, network, policy version

This shape follows attribute-based access control as described in NIST SP 800-162. Roles, groups, relationships, and access lists can supply decision data. The search pipeline does not need to turn those policy models into prompt text. It needs a deterministic permit, deny, or indeterminate result at the resource boundary.

Constrain both retrieval paths

Lexical and dense retrieval fail differently, but they need the same eligible universe. A lexical filter might add a tenant or classification predicate to the query. A vector system might search a partition, pass a filter into approximate nearest-neighbor traversal, or run exact search over an already authorized subset. Hybrid retrieval can fuse only candidates that passed the common decision.

This is a candidate-generation rule rather than a ranking feature. A ranker may use authorization-safe attributes after eligibility is established. It must never receive a restricted record with a low score and rely on the final renderer to remove it.

Document, row, and field controls create different outputs. Document or row policy decides whether a record may enter the candidate set. Field policy decides which attributes of an eligible record can appear in features, snippets, prompts, and responses. PostgreSQL describes row security as an additional per-row predicate, while Elasticsearch documents separate document- and field-level controls. Both enforce at the protected data operation itself.

Test security and recall separately

A filtered dense query can enforce the right eligibility rule and still return weak neighbors. Selective filters change the graph or partition the approximate search must traverse. The Recall view compares approximate results with exact search over the same eligible subset. Both paths can exclude restricted records while only one recovers every useful eligible record.

That gives evaluation two independent assertions. Boundary enforcement means every returned candidate was eligible for this principal, action, resource, and context. Eligible recall means the approximate path recovered enough of the relevant records inside that authorized subset.

Elasticsearch’s filtered kNN guidance exposes the practical tradeoff through candidate counts and filter-aware search. Security does not excuse poor retrieval, and good recall does not excuse a scope violation.

Derived data inherits the restriction

An embedding remains derived from protected text. A cache entry remains derived from a scoped request. A trace can reveal restricted identifiers, counts, terms, or policy claims. Changing the representation does not make the source public.

Cache keys therefore need an authorization-class fingerprint or an equivalent partition. RFC 9111 restricts shared reuse of responses to requests carrying authorization because a response valid for one caller may be invalid for another. A search cache needs the same discipline across tenant, principal class, policy version, locale, schema, and other inputs that change eligibility.

Audit events should explain decisions without copying the entire credential or record. A useful event names a decision identifier, principal reference, action, resource reference, outcome, reason code, policy version, scope reference, and time. Raw profile claims and restricted text belong behind separate retention and access controls.

The chapter leaves the trajectory with two artifacts: an authorized candidate set and a minimal decision reference. Neither becomes permanent authority. A later cache read, memory lookup, delegated branch, or resumed checkpoint creates another protected decision. Chapter 14 follows the context required to make those decisions current.