Engineering · · 4 min read
When a source is unavailable, should the agent use a fallback?
Choose fallback behavior by evidence, identity, freshness, and action consequence so an outage does not turn into a plausible wrong answer.
The agent cannot reach the support system and finishes an account brief from yesterday's summary. The user sees "no unresolved customer issues." The answer is fluent, and the missing source was the only place a new escalation would appear.
A fallback keeps a task moving only when the substitute still satisfies the job. A public LangChain issue about fallback compatibility shows that wiring a fallback can change the interface an agent receives. The question is whether the substitute preserves the evidence and behavior the user needs.
Separate three kinds of fallback#
- A transport fallback reaches the same authoritative system through another permitted path.
- A model fallback changes the reasoning service while using the same evidence and tools.
- A source fallback changes the information itself, such as replacing live support data with a cached brief.
Another network path preserves the job if identity and request semantics stay the same. Another model must handle the same tool and output contracts. Another source can be older, incomplete, or less authoritative and still answer confidently.
LangChain's middleware documentation describes model fallback mechanisms.
Define the missing fact#
For the account brief, the essential missing fact is whether there are unresolved support commitments now. A cached summary supplies background and cannot prove that no new issue appeared after it was created.
Keep the supported parts of the brief: account owner, renewal date, and historical context. Mark the current support section unavailable with the time of the last successful observation. The user decides whether to proceed or wait for a fresh check.
If a permitted direct lookup reaches the same support record while broad search is down, use it for the missing fact and keep the source identifier and observation time.
| Failure | Potential fallback | Condition for use |
|---|---|---|
| Search endpoint unavailable | Direct object lookup | Known object and same access rules |
| Model provider unavailable | Approved alternate model | Tool contract and data handling remain valid |
| Live source unavailable | Cached evidence | Age and missing-current-state limits are explicit |
| Permission denied | Different permitted evidence | No attempt to bypass the denied authority |
| Critical prerequisite unknown | No action fallback | Return the exact missing decision input |
Match the fallback to the action#
A historical summary can support a draft. It is inadequate for sending a customer promise, changing access, or approving a refund.
Take a refund workflow. A cached invoice proves that an invoice once existed and says nothing about the current refundable balance. The agent can prepare the request and explain the missing current check, and the write waits on the provider's current state and normal authorization.
Do not stop every task at the first outage. Continue independent work, preserve completed artifacts, and narrow the blocked portion to the fact or operation that needs the unavailable evidence.
Avoid fallback loops#
A fallback can fail too. Set a short, ordered list of alternatives with explicit reasons, and do not cycle between providers until one sounds complete.
Record which path supplied each material fact. If a model fallback lacks a required tool capability, fail that operation clearly. Silently dropping the tool and generating from prior text is the wrong recovery. Test a changed model interface the way you test a framework migration, using the framework migration checklist for tool arguments, result pairing, and unfinished tasks.
Test the answer under missing evidence#
Remove the critical source in a controlled test and ask the same question. The expected answer keeps verified facts, identifies the gap, and avoids the conclusion that needs the missing evidence. Add a stale cache, a permission denial, an alternate model without the required tool support, and a partial-result source, and verify each produces the intended recovery and no generic retry.
Version the fallback rules with the tool configuration and the missing-source fixture. Deixic's connection guidance covers connecting tools, and evidence history records decisions and results for calls routed through Deixic. The result is a small set of explicit substitution rules, including the cases where an honest gap is the fastest route to a correct finished task.