`nix copy --to` caches the narinfo it uploaded, so the pusher rejects its own paths from caches that sign on serve

## Describe the bug `nix copy --to` records the narinfo it uploaded in the client's own narinfo caches, and what it records is the *locally constructed* one, signatures included. If the client does not sign locally, that is an entry with no signatures at all, cached for `narinfo-cache-positive-ttl` (30 days by default). https://github.com/NixOS/nix/blob/master/src/libstore/binary-cache-store.cc#L124-L137 ```cpp void BinaryCacheStore::writeNarInfo(ref<NarInfo> narInfo) { auto narInfoFile = narInfoFileFor(narInfo->path); upsertFile(narInfoFile, narInfo->to_string(*this), "text/x-nix-narinfo"); pathInfoCache->lock()->upsert(narInfo->path, PathInfoCacheValue{.value = std::shared_ptr<NarInfo>(narInfo)}); if (diskCache) diskCache->upsertNarInfo( config.getReference().render(/*FIXME withParams=*/false), std::string(narInfo->path.hashPart()), std::shared_ptr<NarInfo>(narInfo)); } ``` This assumes the cache serves back exactly what was uploaded. That does not hold for caches which add signatures when they serve, which several hosted Nix caches do so that clients can push without holding a signing key. For those, the signature only ever exists in the server's response, and the uploader has just cached a copy without it. The uploader then refuses its own pushed paths: ``` warning: ignoring substitute for '/nix/store/kcjnwrb4l2r36089lrm6vni7a504dhn8-example' from 'https://cache.example.com', as it's not signed by any of the keys in 'trusted-public-keys' ``` and rebuilds them from source until the entry expires. In practice this is reached through `post-build-hook`, so every build silently poisons the narinfo cache for the path it just built, indefinitely. ### Relationship to existing issues This is the same failure as #4258 (a cached narinfo without the signature the cache actually serves) and produces the same confusing warning as #8254. What does not appear in either is that **Nix itself writes such entries, on every upload, without anything going wrong operationally**: - In #4258 the trigger was an operator pushing unsigned paths and fixing it later; the stale entry was collateral. - In #8254 the discussion landed on entries cached from a *different* substituter ([comment](https://github.com/NixOS/nix/issues/8254#issuecomment-1584089143)). Here nobody made a mistake and only one cache is involved. `nix copy --to` populates the cache with an unsigned entry as a matter of course, which makes this reproducible on demand rather than incidental, and gives a single place to fix it. ## Steps To Reproduce With a binary cache that signs narinfos when serving, its public key in `trusted-public-keys`, and no local `secret-key-files`: ```console $ nix build --file ./example.nix --no-link --print-out-paths /nix/store/kcjnwrb4l2r36089lrm6vni7a504dhn8-example $ nix copy --to 'https://cache.example.com' /nix/store/kcjnwrb4l2r36089lrm6vni7a504dhn8-example # The server has a valid signature: $ curl -s https://cache.example.com/kcjnwrb4l2r36089lrm6vni7a504dhn8.narinfo | grep Sig Sig: example-cache-1:8btJ9osjmzLLdpjTz4jaJ4FippzUKODtgvJCmLNKc8rtAdT2NgiS6SCO1fWJkQ5KRe/70bWDPqVLrRau9a8HDQ== # The client cached it without one: $ sqlite3 ~/.cache/nix/binary-cache-v7.sqlite \ "SELECT hashPart, quote(sigs) FROM NARs WHERE hashPart='kcjnwrb4l2r36089lrm6vni7a504dhn8';" kcjnwrb4l2r36089lrm6vni7a504dhn8|'' # So substituting it back fails: $ nix store delete /nix/store/kcjnwrb4l2r36089lrm6vni7a504dhn8-example $ nix build --file ./example.nix --no-link --max-jobs 0 warning: ignoring substitute for '/nix/store/kcjnwrb4l2r36089lrm6vni7a504dhn8-example' from 'https://cache.example.com', as it's not signed by any of the keys in 'trusted-public-keys' error: Cannot build '/nix/store/...-example.drv'. Reason: local builds are disabled (max-jobs = 0) # Bypassing the cached narinfo makes it work: $ nix build --file ./example.nix --no-link --max-jobs 0 --option narinfo-cache-positive-ttl 0 copying path '/nix/store/kcjnwrb4l2r36089lrm6vni7a504dhn8-example' from 'https://cache.example.com'... ``` Note how little of this points at the cause. The narinfo on the server verifies against a trusted key, any *other* machine substitutes the path happily, and the cache logs nothing, because it is never asked. Only the machine that pushed is affected, so the natural integration test for a binary cache (push on A, pull on B) does not cover it either. ## Expected behavior Uploading a path should not leave the client unable to substitute that same path back from the cache it uploaded to. Options, roughly in increasing order of invasiveness: 1. Do not populate the *disk* cache from `writeNarInfo`, only the in-memory `pathInfoCache`. What it writes is a guess about what the server will serve; this stops that guess from outliving the process by up to 30 days. 2. Only cache the uploaded narinfo when it carries at least one signature, since an unsigned entry is the one that can be rejected later. 3. Re-read the narinfo back from the cache after upload and cache that. Correct in general, but costs a round-trip per uploaded path. 4. Treat a cached narinfo that fails the signature check as a cache miss and re-fetch before warning. This would also cover #4258 and #8254, where the stale entry does not come from an upload. (1) looks the most proportionate for this issue; (4) is the more general fix. Separately, the warning text would be much easier to act on if it distinguished "this narinfo came from our own cache" from "we fetched this and it was unsigned". As written it sends people to inspect the server, which is the one place where nothing is wrong. Compare #6672. ## Metadata - Nix version: 2.34.8 - Nix system: x86_64-linux - Relevant settings: `require-sigs = true` (default), `narinfo-cache-positive-ttl = 2592000` (default), no `secret-key-files` ## Workaround For anyone who lands here first: give the upload a cache directory you throw away, so it cannot write to yours. ```bash XDG_CACHE_HOME="$(mktemp -d)" export XDG_CACHE_HOME trap 'rm -rf "$XDG_CACHE_HOME"' EXIT nix copy --to 'https://cache.example.com' $OUT_PATHS ``` Signing locally with `secret-key-files` and trusting your own key also avoids it, since then the entry the client caches carries a signature it trusts. Either way, entries already written stay until they expire, so clear `~/.cache/nix/binary-cache-v*.sqlite` once afterwards. If the pushing was done by a `post-build-hook`, the entries belong to the daemon's cache (`~root/.cache/nix` on NixOS), not to yours, which is a good way to lose an afternoon.