* fix: normalize GitHub URLs ending in .git to not ending in .git
* Refactor `Data::get_repo_info`
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Fix `get_repo_info` for repo with `.git` suffix
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Add e2e-tests to cover it
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Always try geting the redirected url
This would help:
- redirect public gh repo `.git` to its canonical form
- redirect public gh repo, which has been recently renamed
- cases where redirection is needed to get the real repo
This commit make it fallbacks to the previou surl, if getting
the redirected url fail, in case the repository is private.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Add more e2e-tests
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Optimize: Do not try redirect if gh_get_repo_info fail
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Minor refactor
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
---------
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
Co-authored-by: Jiahao XU <Jiahao_XU@outlook.com>
* Use binary name when searching for artifacts
When there is a single binary declared in the manifest and
it differs from the package name, add it to the list of handles
used for pre-built artifact fetching.
* Simplify `binary_name` assignment with a `match`
* Add e2e test
* Only attempt to use the binary name with `GhCrateMeta` fetcher
* Avoid too much over-allocating.
Technically it should also check if gh-crate-meta resolver is enabled, but it is unlikely for it to be disabled and overallocating for extra n-target should be fine, it is an improvement over doubling the space allocated if the binary_name is Some.
* Fix fmt in crates/binstalk/src/ops/resolve.rs
---------
Co-authored-by: Jiahao XU <Jiahao_XU@outlook.com>
* CI: Use runner `macos-14` which uses M1 machine
`macos-14` provides 3-core M1, 7G mem and 14G stroage, while `macos-latest`
(`macos-13` as of writing) provides 4-core intel, 14G mem and 14G
storage.
While the memory is cut by half and core count reduce by 1, I believe
that it would still speedup the CI since M1 is much more powerful than
previous Intel CPU used in `macos-13`.
It would also allow us to run tests on aarch64-apple-darwin and on
universal-apple-darwin, if necessary.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* CI: Fix `e2e-test-subcrate`: Bump `cargo-audit` ver to v0.18.3
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
---------
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Add CLI options
* Add manifest types
* Thread signature policy through to fetchers
* Thread signing section through from metadata
* Implement signing validation
* Clippy
* Attempt testing
* Yes and
* Why
* fmt
* Update crates/bin/src/args.rs
Co-authored-by: Jiahao XU <Jiahao_XU@outlook.com>
* Update crates/binstalk-fetchers/src/gh_crate_meta.rs
Co-authored-by: Jiahao XU <Jiahao_XU@outlook.com>
* Update crates/bin/src/args.rs
Co-authored-by: Jiahao XU <Jiahao_XU@outlook.com>
* Update crates/binstalk-fetchers/src/signing.rs
Co-authored-by: Jiahao XU <Jiahao_XU@outlook.com>
* Update crates/binstalk-fetchers/src/signing.rs
Co-authored-by: Jiahao XU <Jiahao_XU@outlook.com>
* Update crates/binstalk-fetchers/src/signing.rs
Co-authored-by: Jiahao XU <Jiahao_XU@outlook.com>
* Update crates/binstalk-fetchers/src/signing.rs
Co-authored-by: Jiahao XU <Jiahao_XU@outlook.com>
* fixes
* Finish feature
* Document
* Include all fields in the signing.file template
* Readme document
* Review fixes
* Fail on non-utf8 sig
* Thank goodness for tests
* Run test in ci
* Add rsign2 commands
* Log utf8 error
* Update e2e-tests/signing.sh
Co-authored-by: Jiahao XU <Jiahao_XU@outlook.com>
* Fix `e2e-tests/signing.sh` MacOS CI failure
Move the tls cert creation into `signing.sh` and sleep for 10s to wait
for https server to start.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Refactor e2e-tests-signing files
- Use a tempdir generated by `mktemp` for all certificates-related
files
- Put other checked-in files into `e2e-tests/signing`
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Fixed `e2e-tests-signing` connection err in MacOS CI
Wait for server to start up by trying to connect to it.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Fix `e2e-tests-signing` passing `-subj` to `openssl` on Windows
Use single quote instead of double quote to avoid automatic expansion
from bash
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Fix `e2e-tests-signing` waiting for server to startup
Remove `timeout` since it is not supported on MacOS.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Try to fix windows CI by setting `MSYS_NO_PATHCONV=1` on `openssl` cmds
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Fixed `e2e-tests-signing` on windows
By using double `//` for the value passed to option `-subj`
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Fixed infinite loop in `signing/wait-for-server` on Windows
Pass `--ssl-revoke-best-effort` to prevent schannel from checking ssl
revocation status.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Add cap on retry attempt in `signing/wait-for-server.sh`
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Let `singing/server.py` print output to stderr
so that we can see the error message there.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Fix running `signing/server.py` on MacOS CI
use `python3` since macos-latest still has python2 installed and
`python` is a symlink to `python2` there.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
---------
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
Co-authored-by: Jiahao XU <Jiahao_XU@outlook.com>
Fixed#885
Now we can take advantage of new argument `--registry` and
env overrides:
- `CARGO_REGISTRIES_DEFAULT` if `--registry` is not specified
- `CARGO_REGISTRIES_{registry_name}_INDEX` for the registry index url
We can also read from `.cargo/config.toml` for:
- default registry and registries configurations
- additional CA bundle `http.cainfo`
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
Same as `cargo-install`'s `--no-track`.
It is also passed to `cargo-install` if it is invoked.
Also fixed `fs::atomic_symlink_file` which on Windows could fallback to
non-atomic install if symlinking failed.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
Fixed#838
- Add new key `subcrate` for rendering `pkg-url`
- Add new release paths in GitHub, GitLab & SourceForge using key `subcrate` for auto-detection
- Add subcrate detection for GitHub and GitLab
- Add `debug!` when using gh api token in `GhApiClient::new`
- Add subcrate testing to `e2e-tests/subcrate.sh`
- Bump cargo-release to 0.24.9 in e2e-tests/live.sh
to fix test failure on MacOS without libssl installed in `/usr/local/`.
- Optimize GhCrateMeta: Detect subcrate and repo-host in `Data::get_repo_info`
to cache the result and avoid duplicate works, this also makes the code
more ergonomic by removing the need to some `unwrap()` plus making it
more efficient since we don't need to clone the url just to modify it.
- Add instrument to `Data::get_repo_info`
- Fix `shellcheck` err in `e2e-tests/*.sh`
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
- Refactor: Extract new fn `binstalk::helpers::remote::does_url_exist`
- Use new quickinstall release schema in `binstalk::fetchers::QuickInstall`
- Optimize `fetchers::QuickInstall`: Generate url once in `Fetcher::new`
Avoid repeated string allocation plus `Url` parsing.
This also makes changing package_url and stats_url easier.
- Optimize `QuickInstall::report`: Use HEAD instead of GET
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
`bins::BinFile::new` does not include `binary_ext` in the
`BinFile::base_name`.
Also add regression e2e-test.
Fixed#801
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
Fixed#702
* Add new dep windows v0.44.0 for windows only
* Add regression test for #702
* Impl `win::replace_file`
* Use `ReplaceFileW` on windows if `fs::rename` in `fs::atomic_install` fails
* Improve logging and err messages
* Add more regression tests
* Make `BinFile::install_link` atomic: Do not remove file if dst exists
* Fallback to `ReplaceFileW` in `fs::atomic_symlink_file`
* Refactor: Extract new fn `fs::persist`
* Use `fs::persist` instead of `TempFile::persist` in `fs::atomic_install`, which fallbacks to `ReplaceFileW` on windows
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
Co-authored-by: Félix Saparelli <felix@passcod.name>
