* Update justfile to use lld on macOS and Linux
When cargo-zigbuild is not used
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Delete .cargo/config.toml
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Update justfile to use lld as long as cargo-zigbuild isn't used
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Disable lld again
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
---------
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Fix unit test for `GhApiClient`
Set client-side rate limit to 1 request per 200 ms
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Only run `detect-targets` test if changed
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Only run unit tests for crates that are changed
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Fix job `changed-files`
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Fix job `changed-files`
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Fix shell error in job `changed-files`
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* FIx separator for `changed-files`
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Fix passning of craetes to run
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Fix passing of `CARGO_NEXTEST_ADDITIONAL_ARGS`
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Fix deciding which crates to test on windows
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Fix just recipe `unit-tests`
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Split test into two jobs
so that they will have they own cache (since different feature flags are used).
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
---------
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Optimize binstalk-git-repo-api
Use a dedicated github token in CI
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Speedup CI using `cargo-nextest`
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Fix release profile override on Windows
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Optimize unit test in binstalk-registry
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Use `secrets.CI_RELEASE_TEST_GITHUB_TOKEN` for just-setup
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Fix unit testing in justfile
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Add retry on rate limit in unit testing
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Only use `CI_UNIT_TEST_GITHUB_TOKEN` in unit testing
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Fix `test_get_repo_info`: Retry on rate limit
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Fix `test_has_release_artifact_and_download_artifacts`
Retry on rate limit
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
---------
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Refactor: Mv leon and leon-macros into another repo
It's moved to https://github.com/cargo-bins/leon
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Fix CI: Rm `cargo-hack` check for `leon`
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
---------
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Fix ci: Install deps before caching and include `JUST*` env values in caching key
To make sure caching is done properly and does not accidentally include
(old) pre-built binaries that we don't want.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Fix missing `shell`
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Fix justfile target `check`: Call `cargo hack check` directly
without using the `CARGO=cargo-zigbuild` env since `leon` does not use
external C/C++ compiler anyway.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* CI: Dowgrade cargo-hack to 0.6.10
This is the last version working
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
---------
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Revert "release: cargo-binstall v1.4.0 (#1397)"
This reverts commit 99e8256018.
* --allow-dirty on publish
* Remove private key file after generation
* Write public key file to bin crate and to package
* Upload public key file to release
* Add CLI options
* Add manifest types
* Thread signature policy through to fetchers
* Thread signing section through from metadata
* Implement signing validation
* Clippy
* Attempt testing
* Yes and
* Why
* fmt
* Update crates/bin/src/args.rs
Co-authored-by: Jiahao XU <Jiahao_XU@outlook.com>
* Update crates/binstalk-fetchers/src/gh_crate_meta.rs
Co-authored-by: Jiahao XU <Jiahao_XU@outlook.com>
* Update crates/bin/src/args.rs
Co-authored-by: Jiahao XU <Jiahao_XU@outlook.com>
* Update crates/binstalk-fetchers/src/signing.rs
Co-authored-by: Jiahao XU <Jiahao_XU@outlook.com>
* Update crates/binstalk-fetchers/src/signing.rs
Co-authored-by: Jiahao XU <Jiahao_XU@outlook.com>
* Update crates/binstalk-fetchers/src/signing.rs
Co-authored-by: Jiahao XU <Jiahao_XU@outlook.com>
* Update crates/binstalk-fetchers/src/signing.rs
Co-authored-by: Jiahao XU <Jiahao_XU@outlook.com>
* fixes
* Finish feature
* Document
* Include all fields in the signing.file template
* Readme document
* Review fixes
* Fail on non-utf8 sig
* Thank goodness for tests
* Run test in ci
* Add rsign2 commands
* Log utf8 error
* Update e2e-tests/signing.sh
Co-authored-by: Jiahao XU <Jiahao_XU@outlook.com>
* Fix `e2e-tests/signing.sh` MacOS CI failure
Move the tls cert creation into `signing.sh` and sleep for 10s to wait
for https server to start.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Refactor e2e-tests-signing files
- Use a tempdir generated by `mktemp` for all certificates-related
files
- Put other checked-in files into `e2e-tests/signing`
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Fixed `e2e-tests-signing` connection err in MacOS CI
Wait for server to start up by trying to connect to it.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Fix `e2e-tests-signing` passing `-subj` to `openssl` on Windows
Use single quote instead of double quote to avoid automatic expansion
from bash
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Fix `e2e-tests-signing` waiting for server to startup
Remove `timeout` since it is not supported on MacOS.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Try to fix windows CI by setting `MSYS_NO_PATHCONV=1` on `openssl` cmds
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Fixed `e2e-tests-signing` on windows
By using double `//` for the value passed to option `-subj`
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Fixed infinite loop in `signing/wait-for-server` on Windows
Pass `--ssl-revoke-best-effort` to prevent schannel from checking ssl
revocation status.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Add cap on retry attempt in `signing/wait-for-server.sh`
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Let `singing/server.py` print output to stderr
so that we can see the error message there.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Fix running `signing/server.py` on MacOS CI
use `python3` since macos-latest still has python2 installed and
`python` is a symlink to `python2` there.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
---------
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
Co-authored-by: Jiahao XU <Jiahao_XU@outlook.com>
instead of `x86_64-apple-darwin` for better optimization since any
machine that support universal support x86_64h.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
feat: Scrap `--rate-limit` from env `BINSTALL_RATE_LIMIT` as a fallback
and set `BINSTALL_RATE_LIMIT` to `100/1` on CI.
Fixed#1196
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
for targets:
- x86_64-apple-darwin
- aarch64-apple-darwin
- x86_64-unknown-linux-gnu
- x86_64-unknown-linux-musl
which will use `zlib-ng` to speedup decompression and use assembly
version for sha1 checksum calculation on supported CPU.
Also enable feature `zlib-ng` on windows and
- aarch64-unknown-linux-gnu
- aarch64-unknown-linux-musl
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
Fixed#885
Now we can take advantage of new argument `--registry` and
env overrides:
- `CARGO_REGISTRIES_DEFAULT` if `--registry` is not specified
- `CARGO_REGISTRIES_{registry_name}_INDEX` for the registry index url
We can also read from `.cargo/config.toml` for:
- default registry and registries configurations
- additional CA bundle `http.cainfo`
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
After the merge of #1184, CI now takes 11m just to shallow clone
crates.io git index, which means that our user using alternative git
index would also be quite slow.
This commit speeds it up by building all dependencies with `-O3` in dev
and release build, in release build we build everything with `-O3`.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
- ci: Check feat powerset of leon & binstalk-downloader in `ci.yml`
- fix leon feature `cli`: Enable dep `miette` in feature `cli`
- fix binstalk-downloader when default feature is disabled and no other
tls related feature is enabled (breaking change due to replace of
`tls::Version` with newtype `TLSVersion`).
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
Same as `cargo-install`'s `--no-track`.
It is also passed to `cargo-install` if it is invoked.
Also fixed `fs::atomic_symlink_file` which on Windows could fallback to
non-atomic install if symlinking failed.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
Fixed#1038
sccache is not very effective at caching dependencies, the external
C/C++ is recompiled in every ci and it takes a lot of time.
Compilation of other Rust dependencies also takes quite some time and
sccache is not helping at all, so I decided to switch to
`Swatinem/rust-cache@v2`.
The downside of the new caching method is that a new cache conntaining
part of the `.cargo/` and `target/` will be created whenver
`Cargo.lock`/`Cargo.toml` changes, but it can still reuse the old cache
to create new caching.
This is acceptable given that `sccache` often fails to reuse
cache due to rate limiting from GHA, since it is not designed for use
like a s3 object pool, and `sccache` will create a lot of new cache
artifacts for a given branch that cannot be reused in main and would
have to cleanup via a cronjob.
Edit:
rust 1.70 uses llvm 16.0, however ubuntu-latest still uses llvm 15.0
As such, during release-build, cross-lang-lto failed due to llvm is too
old.
Temporarily disable linker-plugin-lto to fix this.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
`cargo-test` cannot share its artifacts with `cargo-build` and vice
versa, plus running `e2e-tests` in release build is enough to find out
potential miscompilation.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
Fixed#838
- Add new key `subcrate` for rendering `pkg-url`
- Add new release paths in GitHub, GitLab & SourceForge using key `subcrate` for auto-detection
- Add subcrate detection for GitHub and GitLab
- Add `debug!` when using gh api token in `GhApiClient::new`
- Add subcrate testing to `e2e-tests/subcrate.sh`
- Bump cargo-release to 0.24.9 in e2e-tests/live.sh
to fix test failure on MacOS without libssl installed in `/usr/local/`.
- Optimize GhCrateMeta: Detect subcrate and repo-host in `Data::get_repo_info`
to cache the result and avoid duplicate works, this also makes the code
more ergonomic by removing the need to some `unwrap()` plus making it
more efficient since we don't need to clone the url just to modify it.
- Add instrument to `Data::get_repo_info`
- Fix `shellcheck` err in `e2e-tests/*.sh`
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
Avoid checking dev-deps for jobs `*-check` and lint in workflow `ci.yml`
We just want to make sure cargo-binstall can compile for the target, we
don't want to pull in the extra dependencies.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
to make the pre-built artifacts auditable by `cargo-audit` and other
tools.
* Add new env `JUST_USE_AUDITABLE` to enable use of `cargo-auditable`
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
- Remove `tools: cross` input to `actions/just-setup`
since we use `cargo-zigbuild` for cross compiling instead of `cross-rs`
now.
- Only install `cargo-zigbuild` in `just ci-install-deps` if `JUST_USE_CARGO_ZIGBUILD` is enabled
- Include hash of `rustc -vV` in `build-cache`
Switching between different versions of `rustc` will cause all crates to be
rebuilt using new `rustc`.
- Replace input `cache` with `index-cache` and `build-cache` in `actions/just-setup`
for better control of caching behavior
- Reuse workflow `test`'s `build-cache` in workflow `lint`
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Run e2e-tests for release build
* Print `RUSTFLAGS` for justfile target build & check
* Fix CI: Disable miropt level 4
* Fix CI: Disable -Zbuild-std
* Disable `-Zgcc-ld=lld` and `-Zshare-generics` so that we can switch back to stable
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
Fixed#806
- Add new feature flag `cross-lang-fat-lto` and enable it on release for linux
- Enable `-C linker-plugin-lto` for linux
- Only use `-Z gcc-ld=lld` on non-windows targets when `cargo-zigbuild` is not enabled
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
Enable feature `pkg-config` and `log_release_max_level_debug` for debug
build and enable feature `static`, `trust-dns` and
`log_release_max_level_debug` for release build.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
`bins::BinFile::new` does not include `binary_ext` in the
`BinFile::base_name`.
Also add regression e2e-test.
Fixed#801
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
