* docstrings for things that I was curious about
* report to new stats server, with status
* Arc<Self> needs a clone
* fix stats url in docs
* fix stats url handling
* Send telemetry report to quickinstall if no pre-built is found
quickinstall telemtry is overloaded cargo-bins/cargo-quickinstall#268, it only needs report when the pre-built is not available in upstream.
Signed-off-by: Jiahao XU <30436523+NobodyXu@users.noreply.github.com>
* Fix resolve.rs
And optimize sending telemtry: Remove unnecessary `<Arc as Clone>::clone`
Signed-off-by: Jiahao XU <30436523+NobodyXu@users.noreply.github.com>
* Fix fmt of resolve.rs
Signed-off-by: Jiahao XU <30436523+NobodyXu@users.noreply.github.com>
---------
Signed-off-by: Jiahao XU <30436523+NobodyXu@users.noreply.github.com>
* Add new option `--disable-quick-install-stats`
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Send quickinstall stats in parallel
This commit also make sure'the stats is always sent for
each fetcher, regardless of whether it is picked or not,
to make sure that the quick-install stats collection gets
the full infomration of possible targets.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Rename option to `--disable-telemetry`
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Update args.rs
Co-authored-by: Félix Saparelli <felix@passcod.name>
Signed-off-by: Jiahao XU <30436523+NobodyXu@users.noreply.github.com>
* Update args.rs
Signed-off-by: Jiahao XU <30436523+NobodyXu@users.noreply.github.com>
---------
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
Signed-off-by: Jiahao XU <30436523+NobodyXu@users.noreply.github.com>
Co-authored-by: Félix Saparelli <felix@passcod.name>
* Perform artifact discovery in sequential
Run different `fetcher.find()` in sequential
* FuturesResolver: Fallback to other future if one error
* Fix typo
* Apply cargo fmt
* Parallelise `<QuickInstall as Fetcher>::find`
Check for signature in parallel to the package
* Download signature in `<QuickInstall as Fetcher>::find`
So that the signature download can be done in parallel.
* Bump msrv for binstalk-fetchers to 1.70
* Update crates/binstalk-fetchers/src/futures_resolver.rs
Co-authored-by: Félix Saparelli <felix@passcod.name>
Signed-off-by: Jiahao XU <30436523+NobodyXu@users.noreply.github.com>
* cargo fmt
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
---------
Signed-off-by: Jiahao XU <30436523+NobodyXu@users.noreply.github.com>
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
Co-authored-by: Félix Saparelli <felix@passcod.name>
* Add new dep zeroize
* Use Zeroizing to avoid leaking the token
* Optimize gh-auth-token
Spawn it as a task, and only await it
when using GhApiClient
* Fix binstalk-git-repo-api unit tests
* Bump msrv of binstalk-downloader, binstalk and cargo-binstall to 1.79
* Update From<io::Error> for DownloadError
to use io::Error::downcast
* Update From<io::Error> for BinstallError
* fix dmt
* Use binary name when searching for artifacts
When there is a single binary declared in the manifest and
it differs from the package name, add it to the list of handles
used for pre-built artifact fetching.
* Simplify `binary_name` assignment with a `match`
* Add e2e test
* Only attempt to use the binary name with `GhCrateMeta` fetcher
* Avoid too much over-allocating.
Technically it should also check if gh-crate-meta resolver is enabled, but it is unlikely for it to be disabled and overallocating for extra n-target should be fine, it is an improvement over doubling the space allocated if the binary_name is Some.
* Fix fmt in crates/binstalk/src/ops/resolve.rs
---------
Co-authored-by: Jiahao XU <Jiahao_XU@outlook.com>
* Fix fallback to `cargo-install`: Use the default target
which is guaranteed to work, instead of providing a custom `--target`
from `detect-targets`.
The one from `detect-targets` might report musl even if the `rustc`
reports gnu, since OSes like NixOS could use a custom glibc location
that prevent glibc binaries built for other OSes be run for them.
However, if we are compiling the crate and `rustc` defaults to glibc,
then it should be guaranteed to work.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Impl fn `DesiredTargets::get_initialized`
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Fix fallback to `cargo-install`: Use user-provided override targets
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
---------
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Add CLI options
* Add manifest types
* Thread signature policy through to fetchers
* Thread signing section through from metadata
* Implement signing validation
* Clippy
* Attempt testing
* Yes and
* Why
* fmt
* Update crates/bin/src/args.rs
Co-authored-by: Jiahao XU <Jiahao_XU@outlook.com>
* Update crates/binstalk-fetchers/src/gh_crate_meta.rs
Co-authored-by: Jiahao XU <Jiahao_XU@outlook.com>
* Update crates/bin/src/args.rs
Co-authored-by: Jiahao XU <Jiahao_XU@outlook.com>
* Update crates/binstalk-fetchers/src/signing.rs
Co-authored-by: Jiahao XU <Jiahao_XU@outlook.com>
* Update crates/binstalk-fetchers/src/signing.rs
Co-authored-by: Jiahao XU <Jiahao_XU@outlook.com>
* Update crates/binstalk-fetchers/src/signing.rs
Co-authored-by: Jiahao XU <Jiahao_XU@outlook.com>
* Update crates/binstalk-fetchers/src/signing.rs
Co-authored-by: Jiahao XU <Jiahao_XU@outlook.com>
* fixes
* Finish feature
* Document
* Include all fields in the signing.file template
* Readme document
* Review fixes
* Fail on non-utf8 sig
* Thank goodness for tests
* Run test in ci
* Add rsign2 commands
* Log utf8 error
* Update e2e-tests/signing.sh
Co-authored-by: Jiahao XU <Jiahao_XU@outlook.com>
* Fix `e2e-tests/signing.sh` MacOS CI failure
Move the tls cert creation into `signing.sh` and sleep for 10s to wait
for https server to start.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Refactor e2e-tests-signing files
- Use a tempdir generated by `mktemp` for all certificates-related
files
- Put other checked-in files into `e2e-tests/signing`
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Fixed `e2e-tests-signing` connection err in MacOS CI
Wait for server to start up by trying to connect to it.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Fix `e2e-tests-signing` passing `-subj` to `openssl` on Windows
Use single quote instead of double quote to avoid automatic expansion
from bash
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Fix `e2e-tests-signing` waiting for server to startup
Remove `timeout` since it is not supported on MacOS.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Try to fix windows CI by setting `MSYS_NO_PATHCONV=1` on `openssl` cmds
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Fixed `e2e-tests-signing` on windows
By using double `//` for the value passed to option `-subj`
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Fixed infinite loop in `signing/wait-for-server` on Windows
Pass `--ssl-revoke-best-effort` to prevent schannel from checking ssl
revocation status.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Add cap on retry attempt in `signing/wait-for-server.sh`
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Let `singing/server.py` print output to stderr
so that we can see the error message there.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Fix running `signing/server.py` on MacOS CI
use `python3` since macos-latest still has python2 installed and
`python` is a symlink to `python2` there.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
---------
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
Co-authored-by: Jiahao XU <Jiahao_XU@outlook.com>
* feat: Improve resolution logging
Fixed#1336
Log target of the pre-built binaries which will be installed.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Fix args parsing: `quiet` & `verbose` cannot be set at the same time
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* feat: Implies `--log-level debug` if `--verbose` is set
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
---------
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
`binstalk-downloader` contains stuff about http(s) before the
git code is moved into it and now it becomes http and git.
While git indeed uses http stuff, which is why I decided to put
it into binstalk-downloader, it is more than just downloading
since it is stateful (can be cached locally and updated)
where as http is stateless.
Also `binstalk-downloader`'s codegen time now increases
dramatically and it also creates extra dependencies for
binstalk-fetchers, delaying its execution.
The git code also don't use anything from `binstalk-downloader`
at all, it makes sense to be an independent crate.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Add `error` logging if achieve doesn't contain bin required
So that users know that this is an upstream issue and will report to
upstream.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Update crates/binstalk/src/ops/resolve.rs
---------
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
- Make `binstalk::bins` private.
- Move mod `signal` into `crates/bin`
- Make items in `crates/bin/src/lib.rs` private if possible to reduce
its API generation time.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
feat: `git::Repository` support cancellation.
To make sure users can cancel git operation via signal, e.g. when the
git operation fail or users no longer want to install.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
Refactor: Extract mod `fs` as crate `atomic-install`
To speedup codegen of `binstalk` and also publish a new crate that could
be reused.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
Fixed#1183
Since the crate tarball could be downloaded from a different set of
servers than where the cargo registry is hosted, verifying the checksum
is necessary to verify its integrity.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
Previously it will load the root `Cargo.toml` and treat it as the
manifest for the crate, now it will check its `package.name` and would
search for the workspace if the `package.name` does not match the crate
name.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* perform a bare git cloen to avoid worktree checkouts
This will be way faster on windows
* feat: use the git repository directly to obtain crate information
* Apply code review changes
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Impl `fmt::Display` for `helpers::GitUrl`
and also refactor implementation of `git::Repository::{shallow_clone,
shallow_clone_bare}`.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Fix clippy lint warnings
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Fix typo
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
---------
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
Co-authored-by: Jiahao XU <Jiahao_XU@outlook.com>
Fixed#885
Now we can take advantage of new argument `--registry` and
env overrides:
- `CARGO_REGISTRIES_DEFAULT` if `--registry` is not specified
- `CARGO_REGISTRIES_{registry_name}_INDEX` for the registry index url
We can also read from `.cargo/config.toml` for:
- default registry and registries configurations
- additional CA bundle `http.cainfo`
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
instead of `CompactString` since `target_lexicon::{OperatingSystem,
Architecture, Environment}::into_str()` will return a `&'static str`
most of the time.
Also updated `SUPPORT.md`.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
Fixed#775
- Add dep target-lexicon v0.12.7
- Add `target-{family, arch, libc, vendor}` to
`package.metadata.binstall`.
For `{universal, universal2}-apple-darwin`, the `target-arch` is set to
`universal`.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
Same as `cargo-install`'s `--no-track`.
It is also passed to `cargo-install` if it is invoked.
Also fixed `fs::atomic_symlink_file` which on Windows could fallback to
non-atomic install if symlinking failed.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
If `version_req` requests a specific version instead of a range, then
there is no need to pull all versions available from
https://crates.io/api/v1/crates
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
Fixed#838
- Add new key `subcrate` for rendering `pkg-url`
- Add new release paths in GitHub, GitLab & SourceForge using key `subcrate` for auto-detection
- Add subcrate detection for GitHub and GitLab
- Add `debug!` when using gh api token in `GhApiClient::new`
- Add subcrate testing to `e2e-tests/subcrate.sh`
- Bump cargo-release to 0.24.9 in e2e-tests/live.sh
to fix test failure on MacOS without libssl installed in `/usr/local/`.
- Optimize GhCrateMeta: Detect subcrate and repo-host in `Data::get_repo_info`
to cache the result and avoid duplicate works, this also makes the code
more ergonomic by removing the need to some `unwrap()` plus making it
more efficient since we don't need to clone the url just to modify it.
- Add instrument to `Data::get_repo_info`
- Fix `shellcheck` err in `e2e-tests/*.sh`
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
`leon_macros::template!` can parse template at compile-time.
It accepts a utf-8 string literal and uses `leon` internally to parse it, then generate code that evaluates to `Template<'static>`.
- Exclude fuzz from crate leon when publishing
- Impl fn-like proc-macro `leon_macros::template!`
- Add dep `leon-macros` to binstalk
- Use `leon_macros::template!` in `binstalk::fetchers::gh_crate_meta::hosting`
- Add doc for `leon-macros` in `leon`
- Improve `std::fmt::Display` impl for `leon::ParseError`
- Fixed broken infra link in leon
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
