Engineering · · 4 min read
An agent framework upgrade needs more than a passing import
Test tool arguments, injected state, message pairing, streaming, and stored-task recovery when changing framework versions.
The application starts after the upgrade, the demo question gets an answer, and the team deploys. The first real handoff fails: state that used to be injected into a tool is now treated as a missing argument, and the model keeps trying to supply it.
A public LangGraph report about injected state and create_agent describes this kind of failure. It is version-specific and makes no claim about every current installation, but the lesson holds: test the integration contract at the framework boundary.
Start from behavior your users already rely on#
List the behaviors that make your agent useful: choosing the right tool, preserving identity, carrying state between steps, pairing tool results with calls, and returning a finished artifact. An import test covers none of them.
Read the migration guide for your version path; LangChain's v1 migration guide describes API changes and middleware-related concerns. Keep installed package versions with the test results so failures can be reproduced.
Do not combine a framework migration with a model change, connector rewrite, and prompt overhaul; together they make regressions hard to attribute. Keep a baseline running the same user request against old and new implementations.
Reproduce a handoff without a model#
An agent hands a support investigation to a billing specialist. The handoff tool receives application state and the current tool-call identifier, and the model supplies only business arguments. Invoke the tool through the framework's execution path with a deterministic fixture and verify that:
- The required state is injected.
- The resulting message is associated with the correct call.
- The next agent sees the intended subset of prior information.
If this fails, fix the adapter before experimenting with prompts. A prompt telling the model to include state can silence the validation error while letting untrusted generated text fill a field the application should supply.
| Contract | Migration test |
|---|---|
| Tool argument validation | Invalid input fails before side effects |
| Injected identity and state | Model arguments cannot replace application values |
| Tool result pairing | Every result reaches the intended call |
| Parallel calls | Results remain distinct and complete |
| Streaming | Partial output never becomes terminal success |
| Stored task recovery | Old state resumes or rejects with a migration path |
Exercise real failure paths#
Happy-path answers survive upgrades that change error handling. Test a denied operation, a malformed tool result, a timeout, and a throttle, and see whether the framework now raises, returns an error message, or retries internally.
These differences affect safety: a previously terminal permission error can become a repeated tool attempt, and a visible validation failure can disappear into a generic "something went wrong." Preserve the classification recovery code needs.
The retry ownership guide applies directly after a framework change, since new middleware or defaults can add an attempt layer beneath an application retry.
Include unfinished tasks in the migration#
A deployed agent rarely has only fresh conversations: some tasks wait for a human, some have a submitted provider job, and some have stored intermediate results. Decide how the new code reads them.
- Compatible state formats: prove it with saved fixtures.
- Formats needing transformation: version the transformation and keep the original data until validation succeeds.
- Impossible safe continuation: return a specific restart requirement while retaining completed artifacts and unresolved external operations.
Do not replay the whole workflow after a deserialization error when earlier steps wrote to external systems; resume from the last confirmed point or reconcile those writes first.
Compare the outcome, not sentence similarity#
Run representative jobs through both versions with controlled tool responses. Compare target objects, tool choices, arguments, sources, final artifacts, and stop conditions. Different wording is harmless; a different recipient or missing source is a regression.
Then run a small permitted integration test against the real connector; a mocked adapter cannot prove the provider accepts the new request shape.
Deixic's MCP gateway and SDK recovery guide provide integration references for routed work. Build the compatibility suite from the jobs people actually perform, including the failure and resume paths a passing import never exercises.