Why we built our own agent sandbox

Daytona, Modal, and E2B each provide isolated compute for agent code.

We built Sandboxwich because an evaluation run needed a narrower contract. The runtime had to claim a cell that had never served another tenant, bind it to one execution, and prevent it from returning to inventory. If teardown could not be proven, the cell had to remain quarantined.

That requirement changed the state machine, the retry protocol, the credentials inside the pod, and the pool reconciler.

Documented warm-start and lifecycle behavior#

A clean filesystem at claim time cannot prove that a previous process never observed a credential, changed in-memory state, or influenced a daemon that survived cleanup.

A cell gets one lease#

The sterile-cell state machine begins with ready(generation=1, never exposed). A successful claim atomically changes it to leased(generation=2) and binds the lease to the tenant and execution. Its only terminal outcomes are destroyed and quarantined.

ready, generation 1, never exposed
                 |
                 | atomic claim
                 v
leased, generation 2, tenant-bound
                 |
                 | release or expiry
                 v
              stopping
              /      \
     deletion proven  deletion uncertain
            /            \
           v              v
      destroyed       quarantined

No route transitions a leased, destroyed, or quarantined cell back to ready. The pool creates a replacement instead.

The Sandboxwich record shows no earlier tenant lease for that runtime identity.

A retry cannot claim twice#

Network failures make a claim endpoint ambiguous. The caller can lose the response after the server commits the lease. Retrying a generic create request may allocate a second sandbox; changing the request under the same identifier can bind the wrong execution.

Sandboxwich treats claim_id as a durable fence. The first request stores a digest of the complete claim. An exact retry returns the same lease and attestation. A retry that changes the binding or TTL receives 409. If the original fenced claim found no inventory, its retries remain empty even when a cell becomes available later.

Without a durable empty result, a delayed retry could acquire authority after the caller had timed out and moved on.

Daytona documents distributed locking in its control plane, Modal supports unique names for running sandboxes, and E2B tells webhook consumers to deduplicate lifecycle events by event ID. In the public create-interface documentation we reviewed on 18 August 2026, we did not find Sandboxwich's combination of a caller claim fence, full-request digest, generation check, and durable empty replay. We put that application-specific protocol in our control plane.

The lease is scoped to the evaluation#

A Sandboxwich lease binds:

• cell and lease IDs;
• lease generation and expiry;
• signed release set, runtime class, and policy digest;
• organization, workspace, thread, and runner session.

The control plane signs that tuple with HMAC-SHA256 and persists only the attestation's SHA-256 digest.

Inside a prewarmed pod, a credential-bearing control sidecar validates the lease. It sends a sanitized activation over mTLS to a launcher that has no Sandboxwich API credential. The raw lease attestation stays in the control sidecar.

E2B provides a public comparison through its private-beta workload identity. It issues short-lived JWT-SVIDs for a project, sandbox, and execution, and its egress proxy can substitute a token without exposing the token value to sandbox code. An external service can authenticate the workload after validating the token's signature and claims. Our attestation also carries the pool generation, release tuple, evaluation execution binding, and one-shot lease state.

Verify the admitted pod#

Sandboxwich can place Kubernetes workloads behind gVisor or Kata Containers through runtimeClassName. Writing that field into a manifest is insufficient evidence that the isolation boundary exists. A mutating admission webhook can rewrite or remove it.

After Kubernetes admits the pod, the worker reads the pod back from the API server and checks spec.runtimeClassName. It repeats the check before command reuse. A missing or mismatched value produces the terminal runtime_class_boundary_unverified failure and teardown.

Their hosted APIs do not expose Kubernetes runtimeClassName. Modal documents gVisor as part of its managed isolation, while Daytona and E2B expose their own container, VM, and bring-your-own-compute abstractions. We operate this Kubernetes boundary, so the verification belongs in our worker.

Cleanup is a capacity rule#

A timed-out delete call does not prove that the provider stopped the workload. Replacing that cell immediately could leave both the old workload and its replacement alive.

The pool therefore counts stopping and cleanup_pending cells against capacity. Cleanup continues after an API restart and when the configured pool target is zero. The control plane marks the provider absent only when the completed stop matches the expected lease, generation, and disposition. Uncertain deletion ends in quarantine rather than reuse.

Selection boundary#

A workload that requires a persistent session conflicts with the sterile-cell state machine because a released cell cannot resume. Our evaluation runtime instead requires a record that the cell has no prior tenant exposure, claim replay, execution-scoped authority, an admitted runtime class, and provider-confirmed deletion.

Current limits#

Sandboxwich is pre-1.0. Its capability matrix marks the Kubernetes provider paths experimental, and Kata's virtual_machine isolation still needs a recorded live conformance run on infrastructure with nested virtualization. The HMAC assertion authenticates control-plane state. It contains no TPM or confidential-computing evidence.

The implementation and contract are public in evalops/sandboxwich. Start with the sterile-cell design, then read the pool reconciler and the HTTP contract tests.