store URI: introduce multiple signatures support

[picnoir]

Add a secretKeyFiles URI parameter in the store URIs receiving a coma-separated list of Nix signing keyfiles.

For instance:

nix copy --to "file:///tmp/store?secret-keys=/tmp/key1,/tmp/key2" \
  "$(nix build --print-out-paths nixpkgs#hello)"

The keys passed through this new store URI parameter are merged with the key specified in the secretKeyFile parameter, if any.

Motivation

We'd like to rotate the signing key for cache.nixos.org. To simplify the transition, we'd like to sign the new paths with two keys: the new one and the current one. With this, the cache can support nix configurations only trusting the new key and legacy configurations only trusting the current key.

See NixOS/rfcs#149 for more informations behind the motivation.

Context

---

Add 👍 to pull requests you find important.

The Nix maintainer team uses a GitHub project board to schedule and track reviews.

[picnoir]

Erf, miss-clicked, I wanted to open this as a draft, sorry for the ping :(

So, it seems to work, but I only tested that in a ad-hoc fashion for now:

#!/usr/bin/env bash
set -euo pipefail

tmpstore=$(mktemp -d)
trap 'rm -rf $tmpstore' EXIT
./nix key generate-secret --key-name cache.alternativebit.fr > key1
./nix key generate-secret --key-name cache.alternativebit.fr > key2
hellopath=$(nix build --print-out-paths nixpkgs#hello)
./nix copy --to "file://${tmpstore}?secret-keys=$PWD/key1,$PWD/key2" "$hellopath"
tree "$tmpstore"
for file in "$tmpstore"/*.narinfo; do
    if [ $(cat "$file" | grep -E  '^Sig: cache.alternativebit.fr' | wc -l) -ne 2 ]; then
        echo "ERROR: Cannot find 2 signatures in ${file}"
        cat "${file}"
        exit 1
    else
        echo "Found 2 signatures in ${file}"
    fi
done

~/code-root/github.com/NixOS/nix/build/src/nix (pic/multisign*) » ./test-multisign.sh
/tmp/tmp.PhBc2e298N
├── log
├── m2047a1xwgblgkrnbxz0yilkaqfrbf2b.narinfo
├── nar
│   ├── 015k427rhdd5zl2fv6ld00nak9x028j94h0hzypry7bgcmsmxzjg.nar.xz
│   ├── 05wlgdfa54n8fgyjscnr0r8bafmmcmc94h4xqwbdxibi9f0sxaj5.nar.xz
│   ├── 0incw0bg5gx0glrkyif4wpsfscanbakpkap3j5bz1vgb1v5q8aa5.nar.xz
│   ├── 1jy4xxwvkv184mqm2awr7d5xihbaggrcjzxvqjiwva6i8axm11gn.nar.xz
│   └── 1yacv71g68dhknaz3q68jjhgxapi2g824wqapf5daig2i5fzyr12.nar.xz
├── nix-cache-info
├── nj19yxkqf0iqjqn4x6dbglsvqk5bgsbs.narinfo
├── realisations
├── rmy663w9p7xb202rcln4jjzmvivznmz8.narinfo
├── y2xxdhhjy2l5mgpm3d0rw2wxmpd61my4.narinfo
└── y4qpcibkj767szhjb58i2sidmz8m24hb.narinfo

4 directories, 11 files
Found 2 signatures in /tmp/tmp.PhBc2e298N/m2047a1xwgblgkrnbxz0yilkaqfrbf2b.narinfo
Found 2 signatures in /tmp/tmp.PhBc2e298N/nj19yxkqf0iqjqn4x6dbglsvqk5bgsbs.narinfo
Found 2 signatures in /tmp/tmp.PhBc2e298N/rmy663w9p7xb202rcln4jjzmvivznmz8.narinfo
Found 2 signatures in /tmp/tmp.PhBc2e298N/y2xxdhhjy2l5mgpm3d0rw2wxmpd61my4.narinfo
Found 2 signatures in /tmp/tmp.PhBc2e298N/y4qpcibkj767szhjb58i2sidmz8m24hb.narinfo

This script should be converted in a proper functional test, but I'm not familiar with those yet. I'll have a look and see how it works in the next few days.

Once we convert this into a proper test, we can undraft this PR.

Reviews welcome in the meantime.

[Ericson2314]

#11139 my PR will allow JSON for lists, FYI.

[Mic92]

@picnoir could you extend ./tests/functional/signing.sh with this store path syntax?

[picnoir]

#11139 my PR will allow JSON for lists, FYI.

Ok. I'm going to add a ad-hoc JSON-style list parser for this¹. Unless we already have access to a proper JSON parser? I'll have a look. (Edit: there is)

That way, we don't create a direct dependency between #11139 and this PR, and we won't have to support multiple syntax for this URI param.

Footnotes

1. Fancy way of saying I'm going to add a [ prefix and ] suffix. :P ↩

[picnoir]

Added the functional test as suggested by mic92.

@Ericson2314 for the JSON syntax part, would you prefer me to adopt the JSON format from the get go in this PR or would you prefer adding that in #11139 ?

[picnoir]

I tried to implement this JSON configuration here: picnoir@903a2aa

Two issues:

1. The parser is rejecting the URL before reaching the key parsing. I guess the square brackets should be URI-encoded? Not a fan.
2. It's weird to use from the CLI. We need to escape the JSON double ticks. See details in the above commit message.

---

Un-drafting the PR with the comma-separated style for now.

[Ericson2314]

Sorry @picnoir, I was not clear. What I meant was that I want to get that one working, land it, and then your change should be an easy to make on top. I didn't mean to ask you to go implement JSON settings values for this yourself.

[picnoir]

Performance note: we'll be computing the nar fingerprint once for each key:

nix/src/libstore/path-info.cc

Line 25 in 903a2aa

std::string ValidPathInfo::fingerprint(const Store & store) const

.

We should probably memoize the fingerprint here:

nix/src/libstore/realisation.cc

Line 119 in 903a2aa

signatures.insert(signer.signDetached(fingerprint()));

and sign it for each key.

---

Edit: I have a WIP for that, will finish it tonight.

[picnoir]

Note for readers: the pointer indirection is necessary because of the Signer abstraction.

We're not really using this abstraction through and it could potentially be removed? It was introduced for the remote signing mechanism, but the rest did not land and seem stalled.

[Mic92]

Note for readers: the pointer indirection is necessary because of the Signers abstraction.

We're not really using this abstraction through and it could potentially be removed? It was introduced for the remote signing mechanism, but the rest did not land and seem stalled.

Since you also proposed to work on the remote signer, maybe we can remove it for a bit longer.