Initial signing support (#1345)

* Add CLI options

* Add manifest types

* Thread signature policy through to fetchers

* Thread signing section through from metadata

* Implement signing validation

* Clippy

* Attempt testing

* Yes and

* Why

* fmt

* Update crates/bin/src/args.rs

Co-authored-by: Jiahao XU <Jiahao_XU@outlook.com>

* Update crates/binstalk-fetchers/src/gh_crate_meta.rs

Co-authored-by: Jiahao XU <Jiahao_XU@outlook.com>

* Update crates/bin/src/args.rs

Co-authored-by: Jiahao XU <Jiahao_XU@outlook.com>

* Update crates/binstalk-fetchers/src/signing.rs

Co-authored-by: Jiahao XU <Jiahao_XU@outlook.com>

* Update crates/binstalk-fetchers/src/signing.rs

Co-authored-by: Jiahao XU <Jiahao_XU@outlook.com>

* Update crates/binstalk-fetchers/src/signing.rs

Co-authored-by: Jiahao XU <Jiahao_XU@outlook.com>

* Update crates/binstalk-fetchers/src/signing.rs

Co-authored-by: Jiahao XU <Jiahao_XU@outlook.com>

* fixes

* Finish feature

* Document

* Include all fields in the signing.file template

* Readme document

* Review fixes

* Fail on non-utf8 sig

* Thank goodness for tests

* Run test in ci

* Add rsign2 commands

* Log utf8 error

* Update e2e-tests/signing.sh

Co-authored-by: Jiahao XU <Jiahao_XU@outlook.com>

* Fix `e2e-tests/signing.sh` MacOS CI failure

Move the tls cert creation into `signing.sh` and sleep for 10s to wait
for https server to start.

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Refactor e2e-tests-signing files

 - Use a tempdir generated by `mktemp` for all certificates-related
   files
 - Put other checked-in files into `e2e-tests/signing`

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Fixed `e2e-tests-signing` connection err in MacOS CI

Wait for server to start up by trying to connect to it.

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Fix `e2e-tests-signing` passing `-subj` to `openssl` on Windows

Use single quote instead of double quote to avoid automatic expansion
from bash

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Fix `e2e-tests-signing` waiting for server to startup

Remove `timeout` since it is not supported on MacOS.

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Try to fix windows CI by setting `MSYS_NO_PATHCONV=1` on `openssl` cmds

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Fixed `e2e-tests-signing` on windows

By using double `//` for the value passed to option `-subj`

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Fixed infinite loop in `signing/wait-for-server` on Windows

Pass `--ssl-revoke-best-effort` to prevent schannel from checking ssl
revocation status.

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Add cap on retry attempt in `signing/wait-for-server.sh`

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Let `singing/server.py` print output to stderr

so that we can see the error message there.

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Fix running `signing/server.py` on MacOS CI

use `python3` since macos-latest still has python2 installed and
`python` is a symlink to `python2` there.

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

---------

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
Co-authored-by: Jiahao XU <Jiahao_XU@outlook.com>
This commit is contained in:
Félix Saparelli 2023-09-23 16:02:56 +12:00 committed by GitHub
parent efbd20857b
commit 32beba507b
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
29 changed files with 723 additions and 150 deletions

View file

@ -19,7 +19,7 @@ use tracing::{debug, error, info, instrument, warn};
use crate::{
bins,
errors::{BinstallError, VersionParseError},
fetchers::{Data, Fetcher, TargetData},
fetchers::{Data, Fetcher, SignaturePolicy, TargetData},
helpers::{
self, cargo_toml::Manifest, cargo_toml_workspace::load_manifest_from_workspace,
download::ExtractedFiles, remote::Client, target_triple::TargetTriple,
@ -83,6 +83,10 @@ async fn resolve_inner(
return Ok(Resolution::AlreadyUpToDate);
};
if opts.signature_policy == SignaturePolicy::Require && !package_info.signing {
return Err(BinstallError::MissingSignature(package_info.name));
}
let desired_targets = opts
.desired_targets
.get()
@ -126,6 +130,7 @@ async fn resolve_inner(
opts.gh_api_client.clone(),
data.clone(),
target_data,
opts.signature_policy,
);
(fetcher.clone(), AutoAbortJoinHandle::new(fetcher.find()))
}),
@ -216,36 +221,11 @@ async fn download_extract_and_verify(
// Download and extract it.
// If that fails, then ignore this fetcher.
let extracted_files = fetcher.fetch_and_extract(bin_path).await?;
debug!("extracted_files = {extracted_files:#?}");
// Build final metadata
let meta = fetcher.target_meta();
#[cfg(incomplete)]
{
// Fetch and check package signature if available
if let Some(pub_key) = meta.as_ref().map(|m| m.pub_key.clone()).flatten() {
debug!("Found public key: {pub_key}");
// Generate signature file URL
let mut sig_ctx = ctx.clone();
sig_ctx.format = "sig".to_string();
let sig_url = sig_ctx.render(&pkg_url)?;
debug!("Fetching signature file: {sig_url}");
// Download signature file
let sig_path = temp_dir.join(format!("{pkg_name}.sig"));
download(&sig_url, &sig_path).await?;
// TODO: do the signature check
unimplemented!()
} else {
warn!("No public key found, package signature could not be validated");
}
}
// Verify that all non-optional bin_files exist
let bin_files = collect_bin_files(
fetcher,
@ -357,6 +337,7 @@ struct PackageInfo {
version: Version,
repo: Option<String>,
overrides: BTreeMap<String, PkgOverride>,
signing: bool,
}
struct Bin {
@ -465,6 +446,7 @@ impl PackageInfo {
} else {
Ok(Some(Self {
overrides: mem::take(&mut meta.overrides),
signing: meta.signing.is_some(),
meta,
binaries,
name,