cargo-binstall/SUPPORT.md
Félix Saparelli 32beba507b
Initial signing support (#1345)
* Add CLI options

* Add manifest types

* Thread signature policy through to fetchers

* Thread signing section through from metadata

* Implement signing validation

* Clippy

* Attempt testing

* Yes and

* Why

* fmt

* Update crates/bin/src/args.rs

Co-authored-by: Jiahao XU <Jiahao_XU@outlook.com>

* Update crates/binstalk-fetchers/src/gh_crate_meta.rs

Co-authored-by: Jiahao XU <Jiahao_XU@outlook.com>

* Update crates/bin/src/args.rs

Co-authored-by: Jiahao XU <Jiahao_XU@outlook.com>

* Update crates/binstalk-fetchers/src/signing.rs

Co-authored-by: Jiahao XU <Jiahao_XU@outlook.com>

* Update crates/binstalk-fetchers/src/signing.rs

Co-authored-by: Jiahao XU <Jiahao_XU@outlook.com>

* Update crates/binstalk-fetchers/src/signing.rs

Co-authored-by: Jiahao XU <Jiahao_XU@outlook.com>

* Update crates/binstalk-fetchers/src/signing.rs

Co-authored-by: Jiahao XU <Jiahao_XU@outlook.com>

* fixes

* Finish feature

* Document

* Include all fields in the signing.file template

* Readme document

* Review fixes

* Fail on non-utf8 sig

* Thank goodness for tests

* Run test in ci

* Add rsign2 commands

* Log utf8 error

* Update e2e-tests/signing.sh

Co-authored-by: Jiahao XU <Jiahao_XU@outlook.com>

* Fix `e2e-tests/signing.sh` MacOS CI failure

Move the tls cert creation into `signing.sh` and sleep for 10s to wait
for https server to start.

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Refactor e2e-tests-signing files

 - Use a tempdir generated by `mktemp` for all certificates-related
   files
 - Put other checked-in files into `e2e-tests/signing`

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Fixed `e2e-tests-signing` connection err in MacOS CI

Wait for server to start up by trying to connect to it.

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Fix `e2e-tests-signing` passing `-subj` to `openssl` on Windows

Use single quote instead of double quote to avoid automatic expansion
from bash

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Fix `e2e-tests-signing` waiting for server to startup

Remove `timeout` since it is not supported on MacOS.

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Try to fix windows CI by setting `MSYS_NO_PATHCONV=1` on `openssl` cmds

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Fixed `e2e-tests-signing` on windows

By using double `//` for the value passed to option `-subj`

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Fixed infinite loop in `signing/wait-for-server` on Windows

Pass `--ssl-revoke-best-effort` to prevent schannel from checking ssl
revocation status.

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Add cap on retry attempt in `signing/wait-for-server.sh`

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Let `singing/server.py` print output to stderr

so that we can see the error message there.

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Fix running `signing/server.py` on MacOS CI

use `python3` since macos-latest still has python2 installed and
`python` is a symlink to `python2` there.

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

---------

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
Co-authored-by: Jiahao XU <Jiahao_XU@outlook.com>
2023-09-23 04:02:56 +00:00

8.1 KiB

Support for cargo binstall

binstall works with existing CI-built binary outputs, with configuration via [package.metadata.binstall] keys in the relevant crate manifest. When configuring binstall you can test against a local manifest with --manifest-path=PATH argument to use the crate and manifest at the provided PATH, skipping crate discovery and download.

To get started, check the default first, only add a [package.metadata.binstall] section to your Cargo.toml if the default does not work for you.

As an example, the configuration would be like this:

[package.metadata.binstall]
pkg-url = "{ repo }/releases/download/v{ version }/{ name }-{ target }-v{ version }{ archive-suffix }"
bin-dir = "{ name }-{ target }-v{ version }/{ bin }{ binary-ext }"
pkg-fmt = "tgz"

With the following configuration keys:

  • pkg-url specifies the package download URL for a given target/version, templated
  • bin-dir specifies the binary path within the package, templated (with an .exe suffix on windows)
  • pkg-fmt overrides the package format for download/extraction (defaults to: tgz)

pkg-url and bin-dir are templated to support different names for different versions / architectures / etc. Template variables use the format { VAR } where VAR is the name of the variable, \{ for literal {, \} for literal } and \\ for literal \, with the following variables available:

  • name is the name of the crate/package
  • version is the crate version (per --version and the crate manifest)
  • repo is the repository linked in Cargo.toml
  • bin is the name of a specific binary, inferred from the crate configuration
  • target is the rust target name (defaults to your architecture, but can be overridden using the --target command line option if required()
  • archive-suffix is the filename extension of the package archive format that includes the prefix ., e.g. .tgz for tgz or .exe/"" for bin.
  • archive-format is the soft-deprecated filename extension of the package archive format that does not include the prefix ., e.g. tgz for tgz or exe/"" for bin.
  • binary-ext is the string .exe if the target is for Windows, or the empty string otherwise
  • format is a soft-deprecated alias for archive-format in pkg-url, and alias for binary-ext in bin-dir; in the future, this may warn at install time.
  • target-family: Operating system of the target from target_lexicon::OperatingSystem
  • target-arch: Architecture of the target, universal on {universal, universal2}-apple-darwin, otherwise from target_lexicon::Architecture
  • target-libc: ABI environment of the target from target_lexicon::Environment
  • target-vendor: Vendor of the target from target_lexicon::Vendor

pkg-url, pkg-fmt and bin-dir can be overridden on a per-target basis if required, for example, if your x86_64-pc-windows-msvc builds use zip archives this could be set via:

[package.metadata.binstall.overrides.x86_64-pc-windows-msvc]
pkg-fmt = "zip"

Defaults

By default, binstall will try all supported package formats and would do the same for bin-dir.

It will first extract the archives, then iterate over the following list, finding the first dir that exists:

  • { name }-{ target }-v{ version }
  • { name }-{ target }-{ version }
  • { name }-{ version }-{ target }
  • { name }-v{ version }-{ target }
  • { name }-{ target }
  • { name }-{ version }
  • { name }-v{ version }
  • { name }

Then it will concat the dir with "{ bin }{ binary-ext }" and use that as the final bin-dir.

name here is name of the crate, bin is the cargo binary name and binary-ext is .exe on windows and empty on other platforms).

The default value for pkg-url will depend on the repository of the package.

It is set up to work with GitHub releases, GitLab releases, bitbucket downloads and source forge downloads.

If your package already uses any of these URLs, you shouldn't need to set anything.

The URLs are derived from a set of filenames and a set of paths, which are "multiplied together": every filename appended to every path. The filenames are:

  • { name }-{ target }-{ version }{ archive-suffix }
  • { name }-{ target }-v{ version }{ archive-suffix }
  • { name }-{ version }-{ target }{ archive-suffix }
  • { name }-v{ version }-{ target }{ archive-suffix }
  • { name }_{ target }_{ version }{ archive-suffix }
  • { name }_{ target }_v{ version }{ archive-suffix }
  • { name }_{ version }_{ target }{ archive-suffix }
  • { name }_v{ version }_{ target }{ archive-suffix }
  • { name }-{ target }{ archive-suffix } ("versionless")
  • { name }_{ target }{ archive-suffix } ("versionless")

The paths are:

for GitHub

  • { repo }/releases/download/{ version }/
  • { repo }/releases/download/v{ version }/

for GitLab

  • { repo }/-/releases/{ version }/downloads/binaries/
  • { repo }/-/releases/v{ version }/downloads/binaries/

Note that this uses the Permanent links to release assets feature of GitLab EE: it requires you to create an asset as a link with a filepath, which, as of writing, can only be set using GitLab's API.

for BitBucket

  • { repo }/downloads/

Binaries must be uploaded to the project's "Downloads" page on BitBucket.

Also note that as there are no per-release downloads, the "versionless" filename is not considered here.

for SourceForge

  • { repo }/files/binaries/{ version }
  • { repo }/files/binaries/v{ version }

The URLs also have /download appended as per SourceForge's schema.

Binary must be uploaded to the "File" page of your project, under the directory binaries/v{ version }.

Others

For all other situations, binstall does not provide a default pkg-url and you need to manually specify it.

QuickInstall

QuickInstall is an unofficial repository of prebuilt binaries for Crates, and binstall has built-in support for it! If your crate is built by QuickInstall, it will already work with binstall. However, binaries as configured above take precedence when they exist.

Examples

For example, the default configuration (as shown above) for a crate called radio-sx128x (version: v0.14.1-alpha.5 on x86_64 linux) would be interpolated to:

  • A download URL of https://github.com/rust-iot/rust-radio-sx128x/releases/download/v0.14.1-alpha.5/rust-radio-sx128x-x86_64-unknown-linux-gnu-v0.14.1-alpha.5.tgz
  • Containing a single binary file rust-radio-sx128x-x86_64-unknown-linux-gnu-v0.14.1-alpha.5/rust-radio-x86_64-unknown-linux-gnu
  • Installed to$HOME/.cargo/bin/rust-radio-sx128x-v0.14.1-alpha.5
  • With a symlink from $HOME/.cargo/bin/rust-radio-sx128x

If the package name does not match the crate name

As is common with libraries/utilities (and the radio-sx128x example), this can be overridden by specifying the pkg-url:

[package.metadata.binstall]
pkg-url = "{ repo }/releases/download/v{ version }/sx128x-util-{ target }-v{ version }{ archive-suffix }"

Which provides a download URL of https://github.com/rust-iot/rust-radio-sx128x/releases/download/v0.14.1-alpha.5/sx128x-util-x86_64-unknown-linux-gnu-v0.14.1-alpha.5.tgz

If the package structure differs from the default

Were the package to contain binaries in the form name-target[.exe], this could be overridden using the bin-dir key:

[package.metadata.binstall]
bin-dir = "{ bin }-{ target }{ binary-ext }"

Which provides a binary path of: sx128x-util-x86_64-unknown-linux-gnu[.exe]. It is worth noting that binary names are inferred from the crate, so long as cargo builds them this should just work.