Initial signing support (#1345)

* Add CLI options

* Add manifest types

* Thread signature policy through to fetchers

* Thread signing section through from metadata

* Implement signing validation

* Clippy

* Attempt testing

* Yes and

* Why

* fmt

* Update crates/bin/src/args.rs

Co-authored-by: Jiahao XU <Jiahao_XU@outlook.com>

* Update crates/binstalk-fetchers/src/gh_crate_meta.rs

Co-authored-by: Jiahao XU <Jiahao_XU@outlook.com>

* Update crates/bin/src/args.rs

Co-authored-by: Jiahao XU <Jiahao_XU@outlook.com>

* Update crates/binstalk-fetchers/src/signing.rs

Co-authored-by: Jiahao XU <Jiahao_XU@outlook.com>

* Update crates/binstalk-fetchers/src/signing.rs

Co-authored-by: Jiahao XU <Jiahao_XU@outlook.com>

* Update crates/binstalk-fetchers/src/signing.rs

Co-authored-by: Jiahao XU <Jiahao_XU@outlook.com>

* Update crates/binstalk-fetchers/src/signing.rs

Co-authored-by: Jiahao XU <Jiahao_XU@outlook.com>

* fixes

* Finish feature

* Document

* Include all fields in the signing.file template

* Readme document

* Review fixes

* Fail on non-utf8 sig

* Thank goodness for tests

* Run test in ci

* Add rsign2 commands

* Log utf8 error

* Update e2e-tests/signing.sh

Co-authored-by: Jiahao XU <Jiahao_XU@outlook.com>

* Fix `e2e-tests/signing.sh` MacOS CI failure

Move the tls cert creation into `signing.sh` and sleep for 10s to wait
for https server to start.

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Refactor e2e-tests-signing files

 - Use a tempdir generated by `mktemp` for all certificates-related
   files
 - Put other checked-in files into `e2e-tests/signing`

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Fixed `e2e-tests-signing` connection err in MacOS CI

Wait for server to start up by trying to connect to it.

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Fix `e2e-tests-signing` passing `-subj` to `openssl` on Windows

Use single quote instead of double quote to avoid automatic expansion
from bash

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Fix `e2e-tests-signing` waiting for server to startup

Remove `timeout` since it is not supported on MacOS.

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Try to fix windows CI by setting `MSYS_NO_PATHCONV=1` on `openssl` cmds

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Fixed `e2e-tests-signing` on windows

By using double `//` for the value passed to option `-subj`

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Fixed infinite loop in `signing/wait-for-server` on Windows

Pass `--ssl-revoke-best-effort` to prevent schannel from checking ssl
revocation status.

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Add cap on retry attempt in `signing/wait-for-server.sh`

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Let `singing/server.py` print output to stderr

so that we can see the error message there.

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Fix running `signing/server.py` on MacOS CI

use `python3` since macos-latest still has python2 installed and
`python` is a symlink to `python2` there.

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

---------

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
Co-authored-by: Jiahao XU <Jiahao_XU@outlook.com>
This commit is contained in:
Félix Saparelli 2023-09-23 16:02:56 +12:00 committed by GitHub
parent efbd20857b
commit 32beba507b
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
29 changed files with 723 additions and 150 deletions

95
SIGNING.md Normal file
View file

@ -0,0 +1,95 @@
# Signature support
Binstall supports verifying signatures of downloaded files.
At the moment, only one algorithm is supported, but this is expected to improve as time goes.
This feature requires adding to the Cargo.toml metadata: no autodiscovery here!
## Minimal example
Generate a [minisign](https://jedisct1.github.io/minisign/) keypair:
```console
minisign -G -p signing.pub -s signing.key
# or with rsign2:
rsign generate -p signing.pub -s signing.key
```
In your Cargo.toml, put:
```toml
[package.metadata.binstall.signing]
algorithm = "minisign"
pubkey = "RWRnmBcLmQbXVcEPWo2OOKMI36kki4GiI7gcBgIaPLwvxe14Wtxm9acX"
```
Replace the value of `pubkey` with the public key in your `signing.pub`.
Save the `signing.key` as a secret in your CI, then use it when building packages:
```console
tar cvf package-name.tar.zst your-files # or however
minisign -S -s signing.key -x package-name.tar.zst.sig -m package-name.tar.zst
# or with rsign2:
rsign sign -s signing.key -x package-name.tar.zst.sig package-name.tar.zst
```
Upload both your package and the matching `.sig`.
Now when binstall downloads your packages, it will also download the `.sig` file and use the `pubkey` in the Cargo.toml to verify the signature.
If the signature has a trusted comment, it will print it at install time.
## Reference
- `algorithm`: required, see below.
- `pubkey`: required, must be the public key.
- `file`: optional, a template to specify the URL of the signature file. Defaults to `{ url }.sig` where `{ url }` is the download URL of the package.
### Minisign
`algorithm` must be `"minisign"`.
The legacy signature format is not supported.
The `pubkey` must be in the same format as minisign generates.
It may or may not include the untrusted comment; it's ignored by Binstall so we recommend not.
## Just-in-time signing
To reduce the risk of a key being stolen, this scheme supports just-in-time signing.
The idea is to generate a keypair when releasing, use it for signing the packages, save the key in the Cargo.toml before publishing to a registry, and then discard the private key when it's done.
That way, there's no key to steal nor to store securely, and every release is signed by a different key.
And because crates.io is immutable, it's impossible to overwrite the key.
There is one caveat to keep in mind: with the scheme as described above, Binstalling with `--git` may not work:
- If the Cargo.toml in the source contains a partially-filled `[...signing]` section, Binstall will fail.
- If the section contains a different key than the ephemeral one used to sign the packages, Binstall will refuse to install what it sees as corrupt packages.
- If the section is missing entirely, Binstall will work, but of course signatures won't be checked.
The solution here is either:
- Commit the Cargo.toml with the ephemeral public key to the repo when publishing.
- Omit the `[...signing]` section in the source, and write the entire section on publish instead of just filling in the `pubkey`; signatures won't be checked for `--git` installs.
- Instruct your users to use `--skip-signatures` if they want to install with `--git`.
## Why not X? (Sigstore, GPG, signify, with SSH keys, ...)
We're open to pull requests adding algorithms!
We're especially interested in Sigstore for a better implementation of "just-in-time" signing (which it calls "keyless").
We chose minisign as the first supported algorithm as it's lightweight, fairly popular, and has zero options to choose from.
## There's a competing project that does package signature verification differently!
[Tell use about it](https://github.com/cargo-bins/cargo-binstall/issues/1)!
We're not looking to fracture the ecosystem here, and will gladly implement support if something exists already.
We'll also work with others in the space to eventually formalise this beyond Binstall, for example around the `cargo-dist.json` metadata format.
## What's the relationship to crate/registry signing?
There isn't one.
Crate signing is something we're also interested in, and if/when it materialises we'll add support in Binstall for the bits that concern us, but by nature package signing is not related to (source) crate signing.