Binary installation for rust projects
Find a file
Jiahao XU 1dbd2460a3
Support private github repository (#1690)
* Refactor: Create new crate binstalk-git-repo-api

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Fix CI lint warnings

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Fix `just check`: Rm deleted features from `cargo-hack` check

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Refactor: Extract  new mod error

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Ret artifact url in `has_release_artifact`

So that we can use it to download from private repositories.

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Move `test_graph_ql_error_type` to mod `error`

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Fix running `cargo test` in `binstalk-git-repo-api``

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Remove unnecessary import in mod `error::test`

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Rename mod `request`` to `release_artifacts`

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Impl draft version of fetching repo info

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Refactor: Move `HasReleaseArtifacts` failure variants into `GhApiError`

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Refactor: Use `GhRepo` in `GhRelease`

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Fix testing

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Refactor: Return `'static` future

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Refactor: Make sure `'static` Future is returned

To make it easier to create generic function

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Add logging to unit testing

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Fix unit testing

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Refactor: Extract new fn `GhApiClient::do_fetch`

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Rm unused `percent_encode_http_url_path`

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Fix `cargo test` run on CI

`cargo test` run all tests in one process.

As such, `set_global_default` would fail on the second call.

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Optimize `GhApiClient::do_fetch`: Avoid unnecessary restful API call

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Refactor: Rm param `auth_token` for restful API fn

which is always set to `None`

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Impl new API `GhApiClient::get_repo_info`

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Fix unit test for `GhApiClient::get_repo_info`

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Refactor testing: Parameter-ize testing

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Parallelise `test_get_repo_info`

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Refactor: Create parameter-ised `test_has_release_artifact`

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Parallelize `test_has_release_artifact`

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Refactor: `gh_api_client::test::create_client` shall not be `async`

as there is no `.await` in it.

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Return `Url` in `GhApiClient::has_release_artifact`

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Impl new API `GhApiClient::download_artifact`

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Remove unused deps added to binstalk-git-repo-api

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Fix clippy lints

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Add new API `GhApiClient::remote_client`

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Add `GhApiClient::has_gh_token`

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Add `GhRepo::try_extract_from_url`

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Rename `ReleaseArtifactUrl` to `GhReleaseArtifactUrl`

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Add new fn `Download::with_data_verifier`

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* feature: Support private repository

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Fix clippy lints

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Add e2e-test/private-github-repo

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Fix clippy lints

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Fix `launch_baseline_find_tasks`: Retry on rate limit

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Fix test failure: Retry on rate limit

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Temporarily enable debug output for e2e-test-private-github-repo

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Fix `get_repo_info`: Retry on rate limit

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Improve `debug!` logging

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Add more debug logging

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Add more debugging

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Add more debug logging

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Apply suggestions from code review

* Fix compilation

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Fix cargo fmt

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Add crate binstalk-git-repo-api to release-pr.yml

* Update crates/binstalk-git-repo-api/Cargo.toml

* Apply suggestions from code review

* Update crates/binstalk/Cargo.toml

---------

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
2024-06-10 06:02:12 +00:00
.github Support private github repository (#1690) 2024-06-10 06:02:12 +00:00
crates Support private github repository (#1690) 2024-06-10 06:02:12 +00:00
e2e-tests Support private github repository (#1690) 2024-06-10 06:02:12 +00:00
.editorconfig Split crates and clean up structure of codebase (#294) 2022-08-20 11:24:12 +00:00
.gitignore Migrate CI and builds to Just, add "full" builds (#660) 2023-01-08 16:27:36 +11:00
action.yml Add GitHub Action declaration for installing cargo-binstall (#1269) 2023-08-09 20:50:16 +00:00
Cargo.lock Support private github repository (#1690) 2024-06-10 06:02:12 +00:00
Cargo.toml Support private github repository (#1690) 2024-06-10 06:02:12 +00:00
cleanup-cache.sh ci: Rm merge queue cache in cache-cleanup.yml (#1129) 2023-06-05 12:54:01 +00:00
install-from-binstall-release.ps1 Use $Env:CARGO_HOME in ps1 install script (#1509) 2023-11-20 07:22:38 +00:00
install-from-binstall-release.sh Fix install-from-binstall-release.sh for Git Bash users on Windows (#1562) 2024-01-13 02:22:51 +00:00
justfile Support private github repository (#1690) 2024-06-10 06:02:12 +00:00
README.md Renew readme (#1524) 2023-12-13 06:39:08 +00:00
rust-toolchain.toml Add rust-toolchain.toml (#1147) 2023-06-17 05:51:56 +00:00
SIGNING.md Fix release build again (#1400) 2023-09-26 11:17:17 +00:00
SUPPORT.md Initial signing support (#1345) 2023-09-23 04:02:56 +00:00
zigbuild-requirements.txt Fix CI: Require ziglang < 0.11 (#1250) 2023-08-06 04:53:47 +00:00

Cargo B(inary)Install

Binstall provides a low-complexity mechanism for installing Rust binaries as an alternative to building from source (via cargo install) or manually downloading packages. This is intended to work with existing CI artifacts and infrastructure, and with minimal overhead for package maintainers.

Binstall works by fetching the crate information from crates.io and searching the linked repository for matching releases and artifacts, falling back to the quickinstall third-party artifact host, to alternate targets as supported, and finally to cargo install as a last resort.

CI build GitHub tag Crates.io

You may want to see this page as it was when the latest version was published.

Usage

$ cargo binstall radio-sx128x@0.14.1-alpha.5
 INFO resolve: Resolving package: 'radio-sx128x@=0.14.1-alpha.5'
 WARN The package radio-sx128x v0.14.1-alpha.5 (x86_64-unknown-linux-gnu) has been downloaded from github.com
 INFO This will install the following binaries:
 INFO   - sx128x-util (sx128x-util-x86_64-unknown-linux-gnu -> /home/.cargo/bin/sx128x-util)
Do you wish to continue? yes/[no]
? yes
 INFO Installing binaries...
 INFO Done in 2.838798298s

Binstall aims to be a drop-in replacement for cargo install in many cases, and supports similar options.

For unattended use (e.g. in CI), use the --no-confirm flag. For additional options please see cargo binstall --help.

Installation

If you already have it

To upgrade cargo-binstall, use cargo binstall cargo-binstall!

Quickly

Here are one-liners for downloading and installing a pre-compiled cargo-binstall binary.

Linux and macOS

curl -L --proto '=https' --tlsv1.2 -sSf https://raw.githubusercontent.com/cargo-bins/cargo-binstall/main/install-from-binstall-release.sh | bash

Windows

Set-ExecutionPolicy Unrestricted -Scope Process; iex (iwr "https://raw.githubusercontent.com/cargo-bins/cargo-binstall/main/install-from-binstall-release.ps1").Content

Manually

Download the relevant package for your system below, unpack it, and move the cargo-binstall executable into $HOME/.cargo/bin:

OS Arch URL
Linux x86_64 https://github.com/cargo-bins/cargo-binstall/releases/latest/download/cargo-binstall-x86_64-unknown-linux-musl.tgz
Linux armv7 https://github.com/cargo-bins/cargo-binstall/releases/latest/download/cargo-binstall-armv7-unknown-linux-musleabihf.tgz
Linux arm64 https://github.com/cargo-bins/cargo-binstall/releases/latest/download/cargo-binstall-aarch64-unknown-linux-musl.tgz
Mac Intel https://github.com/cargo-bins/cargo-binstall/releases/latest/download/cargo-binstall-x86_64-apple-darwin.zip
Mac Apple Silicon https://github.com/cargo-bins/cargo-binstall/releases/latest/download/cargo-binstall-aarch64-apple-darwin.zip
Mac Universal
(both archs)
https://github.com/cargo-bins/cargo-binstall/releases/latest/download/cargo-binstall-universal-apple-darwin.zip
Windows Intel/AMD https://github.com/cargo-bins/cargo-binstall/releases/latest/download/cargo-binstall-x86_64-pc-windows-msvc.zip
Windows ARM 64 https://github.com/cargo-bins/cargo-binstall/releases/latest/download/cargo-binstall-aarch64-pc-windows-msvc.zip

From source

With a recent Rust installed:

cargo install cargo-binstall

In GitHub Actions

We provide a first-party, minimal action that installs the latest version of Binstall:

  - uses: cargo-bins/cargo-binstall@main

For more features, we recommend the excellent taiki-e/install-action, which has dedicated support for selected tools and uses Binstall for everything else.

Companion tools

These are useful third-party tools which work well with Binstall.

cargo-update

While you can upgrade crates explicitly by running cargo binstall again, cargo-update takes care of updating all tools as needed. It automatically uses Binstall to install the updates if it is present.

cargo-run-bin

Binstall and cargo install both install tools globally by default, which is fine for system-wide tools. When installing tooling for a project, however, you may prefer to both scope the tools to that project and control their versions in code. That's where cargo-run-bin comes in, with a dedicated section in your Cargo.toml and a short cargo subcommand. When Binstall is available, it installs from binary whenever possible... and you can even manage Binstall itself with cargo-run-bin!

Unsupported crates

Binstall is generally smart enough to auto-detect artifacts in most situations. However, if a package fails to install, you can manually specify the pkg-url, bin-dir, and pkg-fmt as needed at the command line, with values as documented in SUPPORT.md.

$ cargo-binstall \
  --pkg-url="{ repo }/releases/download/{ version }/{ name }-{ version }-{ target }.{ archive-format }" \
  --pkg-fmt="txz" \
  crate_name

Maintainers wanting to make their users' life easier can add explicit Binstall metadata to Cargo.toml to locate the appropriate binary package for a given version and target.

Signatures

We have initial, limited support for maintainers to specify a signing public key and where to find package signatures. With this enabled, Binstall will download and verify signatures for that package.

You can use --only-signed to refuse to install packages if they're not signed.

If you like to live dangerously (please don't use this outside testing), you can use --skip-signatures to disable checking or even downloading signatures at all.

FAQ

Why use this?

Because wget-ing releases is frustrating, cargo install takes a not inconsequential portion of forever on constrained devices, and often putting together actual packages is overkill.

Why use the cargo manifest?

Crates already have these, and they already contain a significant portion of the required information. Also, there's this great and woefully underused (IMO) [package.metadata] field.

Is this secure?

Yes and also no?

We have initial support for verifying signatures, but not a lot of the ecosystem produces signatures at the moment. See #1 to discuss more on this.

We always pull the metadata from crates.io over HTTPS, and verify the checksum of the crate tar. We also enforce using HTTPS with TLS >= 1.2 for the actual download of the package files.

Compared to something like a curl ... | sh script, we're not running arbitrary code, but of course the crate you're downloading a package for might itself be malicious!

What do the error codes mean?

You can find a full description of errors including exit codes here: https://docs.rs/binstalk/latest/binstalk/errors/enum.BinstallError.html

Are debug symbols available?

Yes! Extra pre-built packages with a .full suffix are available and contain split debuginfo, documentation files, and extra binaries like the detect-wasi utility.


If you have ideas/contributions or anything is not working the way you expect (in which case, please include an output with --log-level debug) and feel free to open an issue or PR.