Engineering · · 4 min read
Why does the right document disappear when an agent adds a filter?
Trace empty retrieval through identity, metadata types, query construction, and ranking before changing the model.
Your agent finds the deployment guide in a general search. Ask specifically for the payments project and the answer becomes "I could not find that." The document exists and the connection works; the extra detail breaks retrieval.
A LangChain report about metadata filtering describes this class of problem. Inspect the actual filter and stored metadata before changing the prompt.
Follow the document through the query#
Take one document the requesting user can read, save its stable source identifier, and inspect its indexed metadata. Compare the stored value with the query value: type, capitalization, nesting, and whether every chunk has the field. A project identifier stored as a string does not match the integer the agent supplied.
Pinecone documents its supported metadata operators. Other stores differ in syntax, and a filter accepted by your framework does not prove the provider received the intended expression.
Separate two constraints:
- Access restrictions come from authenticated identity and trusted application state.
- Relevance constraints come from the user's question.
The model can suggest that "payments" means a particular project but never invents the set of projects the user can access.
The distinction matters during recovery. Removing an optional date filter broadens a legitimate search; removing the access constraint changes who can see company information. Review the agent identity guidance when the two kinds of filter have become mixed.
Work through a small failure#
Take a repository guide indexed as six chunks. Five have project_id: "pay-core", and one chunk, containing the rollback procedure, has only project: "Payments". The general search returns it because it contains the question's exact phrase; the project-filtered search removes it.
Raising the result count cannot restore a document the filter excluded, and switching embedding models adds expensive activity around a metadata gap. Fix ingestion so every chunk inherits the authoritative project identifier, then reindex and verify its chunks.
In a second failure, all six chunks have correct metadata, and the agent turns "last quarter's deployment guide" into a filter on the last modification date. The guide was updated yesterday and disappears. The user wanted an older release's procedure, and modification time is not release applicability. The fix is a different metadata field or an explicit follow-up search.
The two failures look identical in chat and need different repairs.
| Check | Evidence to capture | Likely correction |
|---|---|---|
| Source exists | Direct permitted lookup | Repair ingestion if missing |
| Metadata matches | Stored value and type | Normalize at ingestion |
| Filter survives wrappers | Provider-bound query | Fix adapter translation |
| Meaning matches question | Why each filter was added | Revise query planning |
| Candidate survives ranking | Retrieved identifiers | Adjust ranking separately |
Diagnose with a controlled comparison#
Use the same user, collection, and document version throughout:
- Query by the document identifier.
- Apply only the mandatory access constraint.
- Add the project constraint.
- Add the time constraint.
One constraint at a time identifies the transition that loses the document.
Run this in a controlled test with known permitted documents. Do not add a production fallback that retries every failed query without filters: it hides ingestion defects and exposes data whenever a filter carries a security decision.
Keep the raw provider request in restricted debug storage, a safe summary in telemetry: fields applied, adapter version, surviving candidate count.
Make the repair survive the next connector change#
Add a fixture with an allowed document, a disallowed document, a document with missing metadata, and a document whose modification date differs from its effective period. Assert which identifiers return, and check the provider request as well as the final answer, because an eloquent answer can hide the wrong search.
Decide how ingestion treats missing required metadata. Quarantine the source until permission-bearing fields resolve. For optional relevance fields, keep the absence explicit so search can take a broader permitted path without inventing a value.
Your agent keeps its planning and conversation behavior; the repair lives in the connector and retrieval contract. Use Deixic's connection guidance and MCP gateway documentation to inspect the integration path, and keep the filter fixtures beside the connector configuration. Save the failing example so the next migration answers it correctly.