Engineering · · 4 min read
Your agent is under its limit. Why is the connector still throttled?
Account for shared provider quotas, interactive priorities, batch work, and retry-after before increasing agent concurrency.
One agent makes a modest number of requests and receives a rate-limit error. Its owner checks the task budget and finds nothing. Elsewhere, a background inventory shares the credential while other agents retry.
An agent's call budget and a provider's quota are separate constraints. The provider counts requests by application, tenant, user, resource, or endpoint, so a per-conversation budget cannot protect a shared allowance.
Microsoft Graph's throttling guidance describes 429 responses and honoring Retry-After. A public Microsoft Community report describes throttling below the limits the developer expected. Identify the affected quota and the traffic sharing it before buying more model capacity.
Identify the quota that is actually exhausted#
Log the endpoint, application identity, tenant, response code, provider request identifier, and retry delay, and keep credentials out. Group failures by the provider's documented quota scope, not only by agent name.
Separate connector throttling from model-provider throttling: a model limit blocks planning or generation, a connector limit blocks the business system, and raising one allowance does not fix the other.
Also distinguish a sustained rate ceiling from a concurrency or expensive-query limit. A few large requests create pressure a requests-per-minute graph hides, so record operation type and elapsed time beside request count to see whether inventory, search, or writes dominate.
Allocate the shared budget deliberately#
A company uses one connector for interactive account questions and a nightly document refresh. The refresh can finish gradually, while the account question has a person waiting. Without priorities in a shared queue, the background work delays the interactive job even when each agent behaves sensibly.
A shared scheduler reserves capacity for interactive reads, limits background concurrency, and applies provider delays before dispatch. Put these controls where all requests meet, and show operators which queue owns a delayed request.
Give each task an elapsed-time budget besides its request allowance. When an interactive task cannot finish in that window, return its partial result and name the missing source instead of leaving the user waiting.
| Work type | Scheduling choice | Recovery behavior |
|---|---|---|
| Interactive lookup | Reserved priority capacity | Delayed or partial answer, stated |
| Bulk refresh | Lower concurrency and checkpoints | Resume from saved progress |
| Sensitive write | Preserve operation identity | Reconcile before resubmitting |
| Status polling | Coalesce reads where valid | Increase interval within deadline |
| Repeated read | Reuse permitted fresh result | Recheck identity and freshness |
Stop the retry crowd#
When many agents receive the same throttle, their independent sleeps expire together. A shared scheduler records the provider's required delay at the relevant scope and dispatches nothing into that window.
Throttled agents wait on one shared schedule.
Do not ask each model to calculate its own backoff, which wastes generation and depends on the next sampled response. Schedule in code and return a deferred status.
Batching cuts connection overhead and leaves provider throttling in place. Check how the provider counts operations inside a batch, and handle failures per item, since retrying a whole batch can repeat successful writes.
Recover without discarding progress#
For a long refresh, persist the last completed page or cursor only after its results are durable. If throttling interrupts the next page, resume from that checkpoint instead of restarting discovery, and keep the cursor tied to its scope and identity.
For a brief that needs several systems, keep the sources already read and label the throttled source: a missing support status must not become “no support issues.” The inventory completeness guide explains why a partial collection needs a visible limit.
Test the shared condition#
One agent cannot show fairness under a shared quota. Use controlled fake responses to run an interactive lookup beside a bulk refresh and a polling task, and assert that Retry-After is honored, concurrency stays under its cap, and no task retries past its deadline.
Inspect provider calls as well as tool invocations, because an SDK can issue hidden retries. The retry ownership guide traces that multiplication.
Quota scheduling is a provider-specific integration responsibility. Deixic's connection guidance covers setting up connected tools. Save the quota scope, priority policy, and failure fixture as shared engineering knowledge for the next agent.