* Perform artifact discovery in sequential
Run different `fetcher.find()` in sequential
* FuturesResolver: Fallback to other future if one error
* Fix typo
* Apply cargo fmt
* Parallelise `<QuickInstall as Fetcher>::find`
Check for signature in parallel to the package
* Download signature in `<QuickInstall as Fetcher>::find`
So that the signature download can be done in parallel.
* Bump msrv for binstalk-fetchers to 1.70
* Update crates/binstalk-fetchers/src/futures_resolver.rs
Co-authored-by: Félix Saparelli <felix@passcod.name>
Signed-off-by: Jiahao XU <30436523+NobodyXu@users.noreply.github.com>
* cargo fmt
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
---------
Signed-off-by: Jiahao XU <30436523+NobodyXu@users.noreply.github.com>
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
Co-authored-by: Félix Saparelli <felix@passcod.name>
* Add new dep zeroize
* Use Zeroizing to avoid leaking the token
* Optimize gh-auth-token
Spawn it as a task, and only await it
when using GhApiClient
* Fix binstalk-git-repo-api unit tests
* Bump msrv of binstalk-downloader, binstalk and cargo-binstall to 1.79
* Update From<io::Error> for DownloadError
to use io::Error::downcast
* Update From<io::Error> for BinstallError
* fix dmt
* Use binary name when searching for artifacts
When there is a single binary declared in the manifest and
it differs from the package name, add it to the list of handles
used for pre-built artifact fetching.
* Simplify `binary_name` assignment with a `match`
* Add e2e test
* Only attempt to use the binary name with `GhCrateMeta` fetcher
* Avoid too much over-allocating.
Technically it should also check if gh-crate-meta resolver is enabled, but it is unlikely for it to be disabled and overallocating for extra n-target should be fine, it is an improvement over doubling the space allocated if the binary_name is Some.
* Fix fmt in crates/binstalk/src/ops/resolve.rs
---------
Co-authored-by: Jiahao XU <Jiahao_XU@outlook.com>
* Fix fallback to `cargo-install`: Use the default target
which is guaranteed to work, instead of providing a custom `--target`
from `detect-targets`.
The one from `detect-targets` might report musl even if the `rustc`
reports gnu, since OSes like NixOS could use a custom glibc location
that prevent glibc binaries built for other OSes be run for them.
However, if we are compiling the crate and `rustc` defaults to glibc,
then it should be guaranteed to work.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Impl fn `DesiredTargets::get_initialized`
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Fix fallback to `cargo-install`: Use user-provided override targets
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
---------
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Add CLI options
* Add manifest types
* Thread signature policy through to fetchers
* Thread signing section through from metadata
* Implement signing validation
* Clippy
* Attempt testing
* Yes and
* Why
* fmt
* Update crates/bin/src/args.rs
Co-authored-by: Jiahao XU <Jiahao_XU@outlook.com>
* Update crates/binstalk-fetchers/src/gh_crate_meta.rs
Co-authored-by: Jiahao XU <Jiahao_XU@outlook.com>
* Update crates/bin/src/args.rs
Co-authored-by: Jiahao XU <Jiahao_XU@outlook.com>
* Update crates/binstalk-fetchers/src/signing.rs
Co-authored-by: Jiahao XU <Jiahao_XU@outlook.com>
* Update crates/binstalk-fetchers/src/signing.rs
Co-authored-by: Jiahao XU <Jiahao_XU@outlook.com>
* Update crates/binstalk-fetchers/src/signing.rs
Co-authored-by: Jiahao XU <Jiahao_XU@outlook.com>
* Update crates/binstalk-fetchers/src/signing.rs
Co-authored-by: Jiahao XU <Jiahao_XU@outlook.com>
* fixes
* Finish feature
* Document
* Include all fields in the signing.file template
* Readme document
* Review fixes
* Fail on non-utf8 sig
* Thank goodness for tests
* Run test in ci
* Add rsign2 commands
* Log utf8 error
* Update e2e-tests/signing.sh
Co-authored-by: Jiahao XU <Jiahao_XU@outlook.com>
* Fix `e2e-tests/signing.sh` MacOS CI failure
Move the tls cert creation into `signing.sh` and sleep for 10s to wait
for https server to start.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Refactor e2e-tests-signing files
- Use a tempdir generated by `mktemp` for all certificates-related
files
- Put other checked-in files into `e2e-tests/signing`
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Fixed `e2e-tests-signing` connection err in MacOS CI
Wait for server to start up by trying to connect to it.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Fix `e2e-tests-signing` passing `-subj` to `openssl` on Windows
Use single quote instead of double quote to avoid automatic expansion
from bash
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Fix `e2e-tests-signing` waiting for server to startup
Remove `timeout` since it is not supported on MacOS.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Try to fix windows CI by setting `MSYS_NO_PATHCONV=1` on `openssl` cmds
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Fixed `e2e-tests-signing` on windows
By using double `//` for the value passed to option `-subj`
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Fixed infinite loop in `signing/wait-for-server` on Windows
Pass `--ssl-revoke-best-effort` to prevent schannel from checking ssl
revocation status.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Add cap on retry attempt in `signing/wait-for-server.sh`
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Let `singing/server.py` print output to stderr
so that we can see the error message there.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Fix running `signing/server.py` on MacOS CI
use `python3` since macos-latest still has python2 installed and
`python` is a symlink to `python2` there.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
---------
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
Co-authored-by: Jiahao XU <Jiahao_XU@outlook.com>
* feat: Improve resolution logging
Fixed#1336
Log target of the pre-built binaries which will be installed.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Fix args parsing: `quiet` & `verbose` cannot be set at the same time
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* feat: Implies `--log-level debug` if `--verbose` is set
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
---------
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
`binstalk-downloader` contains stuff about http(s) before the
git code is moved into it and now it becomes http and git.
While git indeed uses http stuff, which is why I decided to put
it into binstalk-downloader, it is more than just downloading
since it is stateful (can be cached locally and updated)
where as http is stateless.
Also `binstalk-downloader`'s codegen time now increases
dramatically and it also creates extra dependencies for
binstalk-fetchers, delaying its execution.
The git code also don't use anything from `binstalk-downloader`
at all, it makes sense to be an independent crate.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Add `error` logging if achieve doesn't contain bin required
So that users know that this is an upstream issue and will report to
upstream.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Update crates/binstalk/src/ops/resolve.rs
---------
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
- Make `binstalk::bins` private.
- Move mod `signal` into `crates/bin`
- Make items in `crates/bin/src/lib.rs` private if possible to reduce
its API generation time.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
feat: `git::Repository` support cancellation.
To make sure users can cancel git operation via signal, e.g. when the
git operation fail or users no longer want to install.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
Refactor: Extract mod `fs` as crate `atomic-install`
To speedup codegen of `binstalk` and also publish a new crate that could
be reused.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
Fixed#1183
Since the crate tarball could be downloaded from a different set of
servers than where the cargo registry is hosted, verifying the checksum
is necessary to verify its integrity.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
Previously it will load the root `Cargo.toml` and treat it as the
manifest for the crate, now it will check its `package.name` and would
search for the workspace if the `package.name` does not match the crate
name.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* perform a bare git cloen to avoid worktree checkouts
This will be way faster on windows
* feat: use the git repository directly to obtain crate information
* Apply code review changes
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Impl `fmt::Display` for `helpers::GitUrl`
and also refactor implementation of `git::Repository::{shallow_clone,
shallow_clone_bare}`.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Fix clippy lint warnings
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Fix typo
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
---------
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
Co-authored-by: Jiahao XU <Jiahao_XU@outlook.com>
Fixed#885
Now we can take advantage of new argument `--registry` and
env overrides:
- `CARGO_REGISTRIES_DEFAULT` if `--registry` is not specified
- `CARGO_REGISTRIES_{registry_name}_INDEX` for the registry index url
We can also read from `.cargo/config.toml` for:
- default registry and registries configurations
- additional CA bundle `http.cainfo`
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
instead of `CompactString` since `target_lexicon::{OperatingSystem,
Architecture, Environment}::into_str()` will return a `&'static str`
most of the time.
Also updated `SUPPORT.md`.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
Fixed#775
- Add dep target-lexicon v0.12.7
- Add `target-{family, arch, libc, vendor}` to
`package.metadata.binstall`.
For `{universal, universal2}-apple-darwin`, the `target-arch` is set to
`universal`.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
Same as `cargo-install`'s `--no-track`.
It is also passed to `cargo-install` if it is invoked.
Also fixed `fs::atomic_symlink_file` which on Windows could fallback to
non-atomic install if symlinking failed.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
If `version_req` requests a specific version instead of a range, then
there is no need to pull all versions available from
https://crates.io/api/v1/crates
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
Fixed#838
- Add new key `subcrate` for rendering `pkg-url`
- Add new release paths in GitHub, GitLab & SourceForge using key `subcrate` for auto-detection
- Add subcrate detection for GitHub and GitLab
- Add `debug!` when using gh api token in `GhApiClient::new`
- Add subcrate testing to `e2e-tests/subcrate.sh`
- Bump cargo-release to 0.24.9 in e2e-tests/live.sh
to fix test failure on MacOS without libssl installed in `/usr/local/`.
- Optimize GhCrateMeta: Detect subcrate and repo-host in `Data::get_repo_info`
to cache the result and avoid duplicate works, this also makes the code
more ergonomic by removing the need to some `unwrap()` plus making it
more efficient since we don't need to clone the url just to modify it.
- Add instrument to `Data::get_repo_info`
- Fix `shellcheck` err in `e2e-tests/*.sh`
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
`leon_macros::template!` can parse template at compile-time.
It accepts a utf-8 string literal and uses `leon` internally to parse it, then generate code that evaluates to `Template<'static>`.
- Exclude fuzz from crate leon when publishing
- Impl fn-like proc-macro `leon_macros::template!`
- Add dep `leon-macros` to binstalk
- Use `leon_macros::template!` in `binstalk::fetchers::gh_crate_meta::hosting`
- Add doc for `leon-macros` in `leon`
- Improve `std::fmt::Display` impl for `leon::ParseError`
- Fixed broken infra link in leon
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
Fixed#851
* Add new dep leon to crate binstalk
* Add new variant `BinstallError::Template{Parse, Render}Error`
* Use `leon::Template` in mod `bins`
* Use `leon::Template` in mod `fetchers::gh_crate_meta`
* Refactor mod `bins`: Rm unused associated fn & fields from `Context`
* Simplify unit testing in mod `fetchers::gh_crate_meta`
* Rm soft-deprecated field `fetchers::gh_crate_meta::Context::format`
and change the `match` to resolve `archive` => `self.archive_format`.
* Make macro_rules `leon::template!` far easier to use
* Construct `leon::Template<'_>` as constant in `gh_crate_meta::hosting`
* Simplify `leon::Values` trait
Change its method `get_value` signature to
```rust
fn get_value(&self, key: &str) -> Option<Cow<'_, str>>;
```
Now, `ValuesFn` also accepts non-`'static` function, but now
`leon::Values` is only implemented for `&ValuesFn<F>` now.
This makes it a little bit more cumbersome to use but I think it's a
reasonable drawback.
* Rm `Send` bound req from `ValuesFn`
* Impl new fn `leon::Template::cast`
for casting `Template<'s>` to `Template<'t>` where `'s: 't`
* Rename `leon::Template::has_keys` => `has_any_of_keys`
* Make checking whether format related keys are present more robust
* Optimize `GhCrateMeta::launch_baseline_find_tasks`: Skip checking all fmt ext
if none of the format related keys ("format", "archive-format",
"archive-suffix") are present.
* Only ret `.exe` in `PkgFmt::extensions` on windows
by adding a new param `is_windows: bool`
* Improve debug msg in `GhCrateMeta::fetch_and_extract`
* Add warnings to `GhCrateMeta::find`
* Rm dep tinytemplate
* `impl<'s, 'rhs: 's> ops::AddAssign<&Template<'rhs>> for Template<'s>`
* `impl<'s, 'rhs: 's> ops::AddAssign<Template<'rhs>> for Template<'s>`
* `impl<'s, 'item: 's> ops::AddAssign<Item<'item>> for Template<'s>`
* `impl<'s, 'item: 's> ops::AddAssign<&Item<'item>> for Template<'s>`
* `impl<'s, 'rhs: 's> ops::Add<Template<'rhs>> for Template<'s>` (improved existing `Add` impl)
* `impl<'s, 'rhs: 's> ops::Add<&Template<'rhs>> for Template<'s>`
* `impl<'s, 'item: 's> ops::Add<Item<'item>> for Template<'s>`
* `impl<'s, 'item: 's> ops::Add<&Item<'item>> for Template<'s>`
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
Co-authored-by: Félix Saparelli <felix@passcod.name>
Fixed https://github.com/cargo-bins/cargo-quickinstall/issues/195
- Fix `fetchers::QuickInstall`: Stop sending stats for `universal-apple-darwin`
since quickinstall only supports targets officially supports by rust.
- Only send stats report to quickinstall if the `Fetcher::find` is `.await`ed on
This prevents stats report to be sent for cases where the `QuickInstall` fetcher
is actually unused, e.g. resolved to `GhCrateMeta` or other `QuickInstall` fetcher
with different target.
This also reduces amount of http requests created in background.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
- Refactor: Extract new fn `binstalk::helpers::remote::does_url_exist`
- Use new quickinstall release schema in `binstalk::fetchers::QuickInstall`
- Optimize `fetchers::QuickInstall`: Generate url once in `Fetcher::new`
Avoid repeated string allocation plus `Url` parsing.
This also makes changing package_url and stats_url easier.
- Optimize `QuickInstall::report`: Use HEAD instead of GET
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
