Commit graph

8 commits

Author SHA1 Message Date
Jiahao XU
b854f3f52c
feature: Merge disable strategies (#1868)
* feat: Merge --disable-strategies with ones in cargo manifest

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Update doc

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Update e2e-test-strategies

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Fix typo in option doc in crates/bin/src/args.rs

Signed-off-by: Jiahao XU <30436523+NobodyXu@users.noreply.github.com>

---------

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
Signed-off-by: Jiahao XU <30436523+NobodyXu@users.noreply.github.com>
2024-08-07 14:04:00 +00:00
Jiahao XU
ee94b8b639
Fix: --strategies on CLI do not seem to override disabled-strategies in the manifest (#1857)
* Fix cli override in entry.rs

Forward `args.disabled_strategies`

Signed-off-by: Jiahao XU <30436523+NobodyXu@users.noreply.github.com>

* Fix `args::parse`: Do not free disabled_strategies

Signed-off-by: Jiahao XU <30436523+NobodyXu@users.noreply.github.com>

* Fix passing of cli_overrides

Signed-off-by: Jiahao XU <30436523+NobodyXu@users.noreply.github.com>

* Create strategies-test-override-Cargo.toml

Signed-off-by: Jiahao XU <30436523+NobodyXu@users.noreply.github.com>

* Add e2e-tests for cli-overrides

Signed-off-by: Jiahao XU <30436523+NobodyXu@users.noreply.github.com>

* Fix entry.rs

Signed-off-by: Jiahao XU <30436523+NobodyXu@users.noreply.github.com>

* fix entry.rs

Signed-off-by: Jiahao XU <30436523+NobodyXu@users.noreply.github.com>

* Fix entry.rs

Signed-off-by: Jiahao XU <30436523+NobodyXu@users.noreply.github.com>

* Update strategies.sh

Signed-off-by: Jiahao XU <30436523+NobodyXu@users.noreply.github.com>

* Compute cli_overrides in args::parse

Signed-off-by: Jiahao XU <30436523+NobodyXu@users.noreply.github.com>

* fix use of args::parse main_impl.rs

Signed-off-by: Jiahao XU <30436523+NobodyXu@users.noreply.github.com>

* Update entry.rs

Signed-off-by: Jiahao XU <30436523+NobodyXu@users.noreply.github.com>

* Fix args::parse

Signed-off-by: Jiahao XU <30436523+NobodyXu@users.noreply.github.com>

* Fix typo in args.rs

Signed-off-by: Jiahao XU <30436523+NobodyXu@users.noreply.github.com>

* Fix args.rs

Signed-off-by: Jiahao XU <30436523+NobodyXu@users.noreply.github.com>

* Fix fmt in args.rs

Signed-off-by: Jiahao XU <30436523+NobodyXu@users.noreply.github.com>

* fix fmt in main_impl.rs

Signed-off-by: Jiahao XU <30436523+NobodyXu@users.noreply.github.com>

* update e2e-test-strategies

Signed-off-by: Jiahao XU <30436523+NobodyXu@users.noreply.github.com>

* Update e2e-tests/strategies.sh

Signed-off-by: Jiahao XU <30436523+NobodyXu@users.noreply.github.com>

* Update e2e-tests/strategies.sh

Make sure both --strategies and --disable-strategies is tested

Signed-off-by: Jiahao XU <30436523+NobodyXu@users.noreply.github.com>

* Update strategies.sh

Signed-off-by: Jiahao XU <30436523+NobodyXu@users.noreply.github.com>

---------

Signed-off-by: Jiahao XU <30436523+NobodyXu@users.noreply.github.com>
2024-08-03 05:27:23 +00:00
Jiahao XU
3f29fbe83a
Feature: SupportdDisable of strategies for crate using Cargo.toml (#1828)
* Refactor: Move `Strategy` to `binstalk-types`

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Add serialisation test for `Strategy`

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Add support to disable strategies via crate `Cargo.toml`

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Add e2e-test

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Fix `Cargo.toml` disabled strategy checking for compile strategy

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Optimize `resolve_inner`: Cache meta override

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Add compile-time length checking for `Strategy`

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* More optimization

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Fix order of override: cli options alwayus takes precedence

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Add missing manifest for e2e-test

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

---------

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
2024-07-23 14:05:22 +00:00
Kristof Mattei
fdfc89c287
fix: normalize GitHub URLs ending in .git to not ending in .git (#1804)
* fix: normalize GitHub URLs ending in .git to not ending in .git

* Refactor `Data::get_repo_info`

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Fix `get_repo_info` for repo with `.git` suffix

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Add e2e-tests to cover it

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Always try geting the redirected url

This would help:
 - redirect public gh repo `.git` to its canonical form
 - redirect public gh repo, which has been recently renamed
 - cases where redirection is needed to get the real repo

This commit make it fallbacks to the previou surl, if getting
the redirected url fail, in case the repository is private.

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Add more e2e-tests

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Optimize: Do not try redirect if gh_get_repo_info fail

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Minor refactor

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

---------

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
Co-authored-by: Jiahao XU <Jiahao_XU@outlook.com>
2024-07-14 14:45:41 +00:00
Jiahao XU
1dbd2460a3
Support private github repository (#1690)
* Refactor: Create new crate binstalk-git-repo-api

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Fix CI lint warnings

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Fix `just check`: Rm deleted features from `cargo-hack` check

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Refactor: Extract  new mod error

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Ret artifact url in `has_release_artifact`

So that we can use it to download from private repositories.

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Move `test_graph_ql_error_type` to mod `error`

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Fix running `cargo test` in `binstalk-git-repo-api``

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Remove unnecessary import in mod `error::test`

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Rename mod `request`` to `release_artifacts`

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Impl draft version of fetching repo info

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Refactor: Move `HasReleaseArtifacts` failure variants into `GhApiError`

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Refactor: Use `GhRepo` in `GhRelease`

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Fix testing

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Refactor: Return `'static` future

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Refactor: Make sure `'static` Future is returned

To make it easier to create generic function

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Add logging to unit testing

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Fix unit testing

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Refactor: Extract new fn `GhApiClient::do_fetch`

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Rm unused `percent_encode_http_url_path`

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Fix `cargo test` run on CI

`cargo test` run all tests in one process.

As such, `set_global_default` would fail on the second call.

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Optimize `GhApiClient::do_fetch`: Avoid unnecessary restful API call

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Refactor: Rm param `auth_token` for restful API fn

which is always set to `None`

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Impl new API `GhApiClient::get_repo_info`

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Fix unit test for `GhApiClient::get_repo_info`

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Refactor testing: Parameter-ize testing

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Parallelise `test_get_repo_info`

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Refactor: Create parameter-ised `test_has_release_artifact`

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Parallelize `test_has_release_artifact`

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Refactor: `gh_api_client::test::create_client` shall not be `async`

as there is no `.await` in it.

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Return `Url` in `GhApiClient::has_release_artifact`

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Impl new API `GhApiClient::download_artifact`

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Remove unused deps added to binstalk-git-repo-api

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Fix clippy lints

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Add new API `GhApiClient::remote_client`

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Add `GhApiClient::has_gh_token`

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Add `GhRepo::try_extract_from_url`

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Rename `ReleaseArtifactUrl` to `GhReleaseArtifactUrl`

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Add new fn `Download::with_data_verifier`

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* feature: Support private repository

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Fix clippy lints

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Add e2e-test/private-github-repo

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Fix clippy lints

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Fix `launch_baseline_find_tasks`: Retry on rate limit

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Fix test failure: Retry on rate limit

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Temporarily enable debug output for e2e-test-private-github-repo

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Fix `get_repo_info`: Retry on rate limit

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Improve `debug!` logging

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Add more debug logging

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Add more debugging

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Add more debug logging

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Apply suggestions from code review

* Fix compilation

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Fix cargo fmt

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Add crate binstalk-git-repo-api to release-pr.yml

* Update crates/binstalk-git-repo-api/Cargo.toml

* Apply suggestions from code review

* Update crates/binstalk/Cargo.toml

---------

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
2024-06-10 06:02:12 +00:00
Félix Saparelli
32beba507b
Initial signing support (#1345)
* Add CLI options

* Add manifest types

* Thread signature policy through to fetchers

* Thread signing section through from metadata

* Implement signing validation

* Clippy

* Attempt testing

* Yes and

* Why

* fmt

* Update crates/bin/src/args.rs

Co-authored-by: Jiahao XU <Jiahao_XU@outlook.com>

* Update crates/binstalk-fetchers/src/gh_crate_meta.rs

Co-authored-by: Jiahao XU <Jiahao_XU@outlook.com>

* Update crates/bin/src/args.rs

Co-authored-by: Jiahao XU <Jiahao_XU@outlook.com>

* Update crates/binstalk-fetchers/src/signing.rs

Co-authored-by: Jiahao XU <Jiahao_XU@outlook.com>

* Update crates/binstalk-fetchers/src/signing.rs

Co-authored-by: Jiahao XU <Jiahao_XU@outlook.com>

* Update crates/binstalk-fetchers/src/signing.rs

Co-authored-by: Jiahao XU <Jiahao_XU@outlook.com>

* Update crates/binstalk-fetchers/src/signing.rs

Co-authored-by: Jiahao XU <Jiahao_XU@outlook.com>

* fixes

* Finish feature

* Document

* Include all fields in the signing.file template

* Readme document

* Review fixes

* Fail on non-utf8 sig

* Thank goodness for tests

* Run test in ci

* Add rsign2 commands

* Log utf8 error

* Update e2e-tests/signing.sh

Co-authored-by: Jiahao XU <Jiahao_XU@outlook.com>

* Fix `e2e-tests/signing.sh` MacOS CI failure

Move the tls cert creation into `signing.sh` and sleep for 10s to wait
for https server to start.

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Refactor e2e-tests-signing files

 - Use a tempdir generated by `mktemp` for all certificates-related
   files
 - Put other checked-in files into `e2e-tests/signing`

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Fixed `e2e-tests-signing` connection err in MacOS CI

Wait for server to start up by trying to connect to it.

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Fix `e2e-tests-signing` passing `-subj` to `openssl` on Windows

Use single quote instead of double quote to avoid automatic expansion
from bash

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Fix `e2e-tests-signing` waiting for server to startup

Remove `timeout` since it is not supported on MacOS.

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Try to fix windows CI by setting `MSYS_NO_PATHCONV=1` on `openssl` cmds

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Fixed `e2e-tests-signing` on windows

By using double `//` for the value passed to option `-subj`

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Fixed infinite loop in `signing/wait-for-server` on Windows

Pass `--ssl-revoke-best-effort` to prevent schannel from checking ssl
revocation status.

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Add cap on retry attempt in `signing/wait-for-server.sh`

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Let `singing/server.py` print output to stderr

so that we can see the error message there.

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Fix running `signing/server.py` on MacOS CI

use `python3` since macos-latest still has python2 installed and
`python` is a symlink to `python2` there.

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

---------

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
Co-authored-by: Jiahao XU <Jiahao_XU@outlook.com>
2023-09-23 04:02:56 +00:00
Jiahao XU
ca00cbaccc
feat: Support install directly from git repo (#1162)
Fixed #3

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
2023-06-24 01:01:31 +00:00
Félix Saparelli
aea9df602c
Migrate CI and builds to Just, add "full" builds (#660) 2023-01-08 16:27:36 +11:00