One of our engineering regression tests succeeds only after it gets Cargo to pass a test whose source says it should fail.

It builds a Rust test against a dependency that returns 1, then changes the expected value to 2. It copies the changed source into the build directory and runs Cargo again. The fixture checks that the destination contains the failing assertion. Cargo returns success anyway, reusing the previously compiled test.

We keep this reproduction beside our verification tooling. It tests an assumption that a Git hash and an exit code cannot establish on their own: the build system actually consumed the source we asked it to test.

How the wrong test stays green#

Our remote verifier synchronizes a committed checkout into a leased build slot and runs the verification command there. The slot retains compiled artifacts between runs. Reusing a slot gives Cargo an opportunity to skip work for unchanged inputs.

The failure came from synchronizing archived source while preserving its modification times. Changed files arrived with timestamps older than Cargo's recorded build.

For ordinary source inputs, Cargo's timestamp freshness check compares source modification times with a dep-info file that anchors the previous build. A changed file carrying an older timestamp can leave that check satisfied. Cargo documents this mechanism.

In our reproduction, the source bytes were correct and the command really ran. Cargo reused the old passing test executable. --locked also remained enabled: pinning dependency resolution did nothing to make this source change newer.

Our earlier post about verification receipts covered results that become stale after an agent edits the code. This failure happens before the result is recorded. The verifier needs to establish that its cached executable corresponds to the synchronized source.

Two synchronization flags#

The verifier now synchronizes with:

rsync -az --checksum --no-times --delete ...

The options address separate problems. --checksum makes the transfer decision compare contents, catching changes even when file size and modification time match. --no-times overrides archive mode's timestamp preservation, giving transferred files destination timestamps. The rsync manual describes both options.

An unchanged file keeps its destination timestamp. A changed file gets copied with a fresh timestamp, making its change visible to Cargo's freshness check.

That fixes subsequent transfers. Our regression also demonstrates an existing state those flags cannot repair.

Why existing slots need a migration#

Suppose the old synchronization has already copied the failing source into a warm slot. Its bytes match the sender. Its timestamp still predates the cached build.

The new checksum comparison sees identical contents and skips the transfer. Since nothing transfers, --no-times has no timestamp to refresh. Cargo runs the old passing executable again.

Our fixture explicitly tests this sequence. It applies the corrected flags to the affected slot and asserts that the false green persists. A test starting with a clean destination would have missed this migration requirement.

So the verifier also performs a one-time timestamp refresh when adopting a slot. It walks synchronized regular files and updates their modification times before running the verification command.

The refresh has specific limits:

It excludes .git, target, and node_modules.
It skips symlinks and avoids traversing symlinked directories.
It writes a versioned marker beside the source directory after the walk succeeds. Synchronization with --delete cannot remove that sibling marker.
It runs under the same slot lease as synchronization and verification. A refresh error stops the verification command.

The marker makes the refresh a migration. Touching the source on each run would force repeated recompilation of unchanged code.

The regression checks the cache too#

The offline integration fixture uses real Cargo and rsync in temporary directories, with a dependency stored in a local Git repository. It extracts the synchronization flags and migration script from the verifier's implementation, so the probe exercises the configuration we use.

It checks both correctness and cache preservation. After reproducing the false green, the timestamp migration must make Cargo compile the changed source and report the failing assertion. The dependency's compiled artifact must retain its hash and timestamp. Files under an excluded cache directory and an external symlink target must remain untouched.

Then the fixture changes the assertion back, verifies success, and repeats an unchanged synchronization. That rerun must keep the source timestamp and skip recompiling the fixture crate. Another edit with the same size and timestamp must rebuild and fail.

Finally, it makes normalization fail and checks that a command chained after it with && never executes.

Test the assumptions behind the result#

Receipts produced under the old assumptions still require fresh verification; changing the verifier cannot retroactively establish what those commands compiled.

If your verification pipeline reuses build directories, add a probe that changes a passing assertion into a failing one while restoring an older source timestamp. Run it against both a clean slot and a slot that has already received the changed bytes. Check that unchanged reruns still reuse the expected artifacts.