* Rm unnecessary warning msg "Failed to read git credential file"
Fixed#1476
If `gh auth token` executed successfully and binstall obtained a gh
token from it, then there's no reason to issue any warning msg.
Only when binstall cannot read from `.git-credential` and
`gh auth token` failed does binstall need to issue warning.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Fix clippy warning
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
---------
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Add CLI options
* Add manifest types
* Thread signature policy through to fetchers
* Thread signing section through from metadata
* Implement signing validation
* Clippy
* Attempt testing
* Yes and
* Why
* fmt
* Update crates/bin/src/args.rs
Co-authored-by: Jiahao XU <Jiahao_XU@outlook.com>
* Update crates/binstalk-fetchers/src/gh_crate_meta.rs
Co-authored-by: Jiahao XU <Jiahao_XU@outlook.com>
* Update crates/bin/src/args.rs
Co-authored-by: Jiahao XU <Jiahao_XU@outlook.com>
* Update crates/binstalk-fetchers/src/signing.rs
Co-authored-by: Jiahao XU <Jiahao_XU@outlook.com>
* Update crates/binstalk-fetchers/src/signing.rs
Co-authored-by: Jiahao XU <Jiahao_XU@outlook.com>
* Update crates/binstalk-fetchers/src/signing.rs
Co-authored-by: Jiahao XU <Jiahao_XU@outlook.com>
* Update crates/binstalk-fetchers/src/signing.rs
Co-authored-by: Jiahao XU <Jiahao_XU@outlook.com>
* fixes
* Finish feature
* Document
* Include all fields in the signing.file template
* Readme document
* Review fixes
* Fail on non-utf8 sig
* Thank goodness for tests
* Run test in ci
* Add rsign2 commands
* Log utf8 error
* Update e2e-tests/signing.sh
Co-authored-by: Jiahao XU <Jiahao_XU@outlook.com>
* Fix `e2e-tests/signing.sh` MacOS CI failure
Move the tls cert creation into `signing.sh` and sleep for 10s to wait
for https server to start.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Refactor e2e-tests-signing files
- Use a tempdir generated by `mktemp` for all certificates-related
files
- Put other checked-in files into `e2e-tests/signing`
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Fixed `e2e-tests-signing` connection err in MacOS CI
Wait for server to start up by trying to connect to it.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Fix `e2e-tests-signing` passing `-subj` to `openssl` on Windows
Use single quote instead of double quote to avoid automatic expansion
from bash
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Fix `e2e-tests-signing` waiting for server to startup
Remove `timeout` since it is not supported on MacOS.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Try to fix windows CI by setting `MSYS_NO_PATHCONV=1` on `openssl` cmds
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Fixed `e2e-tests-signing` on windows
By using double `//` for the value passed to option `-subj`
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Fixed infinite loop in `signing/wait-for-server` on Windows
Pass `--ssl-revoke-best-effort` to prevent schannel from checking ssl
revocation status.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Add cap on retry attempt in `signing/wait-for-server.sh`
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Let `singing/server.py` print output to stderr
so that we can see the error message there.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Fix running `signing/server.py` on MacOS CI
use `python3` since macos-latest still has python2 installed and
`python` is a symlink to `python2` there.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
---------
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
Co-authored-by: Jiahao XU <Jiahao_XU@outlook.com>
* feat: Improve resolution logging
Fixed#1336
Log target of the pre-built binaries which will be installed.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Fix args parsing: `quiet` & `verbose` cannot be set at the same time
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* feat: Implies `--log-level debug` if `--verbose` is set
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
---------
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Fix GitHub token auto discovery
Fixed#1333
- Rm dep `gh-token` since it is broken and we can simply run
`gh auth token` in `cargo-binstall` instead.
- binstalk-downloader: Make sure GitHub token is at least 40B long
and other than the `_`, composes of only alphanumeric characters.
- Warn on failure to read `git/credential` files
- Optimize `try_from_home` to avoid heap allocation of `PathBuf`
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Fix typo and clippy
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Simplify `is_valid_gh_token` & `is_ascii_alphanumeric` impl
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Improve err msg in `get_inner`
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Improve err msg of `cargo_binstall::gh_token::get`
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
---------
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
- Move implementation of `main` into the library part of
`cargo-binstall` to speedup codegen.
- Move `manifests.rs` into `binstalk-manifests`
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
- Make `binstalk::bins` private.
- Move mod `signal` into `crates/bin`
- Make items in `crates/bin/src/lib.rs` private if possible to reduce
its API generation time.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
Refactor: Extract mod `fs` as crate `atomic-install`
To speedup codegen of `binstalk` and also publish a new crate that could
be reused.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
Fixed#1183
Since the crate tarball could be downloaded from a different set of
servers than where the cargo registry is hosted, verifying the checksum
is necessary to verify its integrity.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
feat: Scrap `--rate-limit` from env `BINSTALL_RATE_LIMIT` as a fallback
and set `BINSTALL_RATE_LIMIT` to `100/1` on CI.
Fixed#1196
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
Fixed#885
Now we can take advantage of new argument `--registry` and
env overrides:
- `CARGO_REGISTRIES_DEFAULT` if `--registry` is not specified
- `CARGO_REGISTRIES_{registry_name}_INDEX` for the registry index url
We can also read from `.cargo/config.toml` for:
- default registry and registries configurations
- additional CA bundle `http.cainfo`
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
- ci: Check feat powerset of leon & binstalk-downloader in `ci.yml`
- fix leon feature `cli`: Enable dep `miette` in feature `cli`
- fix binstalk-downloader when default feature is disabled and no other
tls related feature is enabled (breaking change due to replace of
`tls::Version` with newtype `TLSVersion`).
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
Same as `cargo-install`'s `--no-track`.
It is also passed to `cargo-install` if it is invoked.
Also fixed `fs::atomic_symlink_file` which on Windows could fallback to
non-atomic install if symlinking failed.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
`leon_macros::template!` can parse template at compile-time.
It accepts a utf-8 string literal and uses `leon` internally to parse it, then generate code that evaluates to `Template<'static>`.
- Exclude fuzz from crate leon when publishing
- Impl fn-like proc-macro `leon_macros::template!`
- Add dep `leon-macros` to binstalk
- Use `leon_macros::template!` in `binstalk::fetchers::gh_crate_meta::hosting`
- Add doc for `leon-macros` in `leon`
- Improve `std::fmt::Display` impl for `leon::ParseError`
- Fixed broken infra link in leon
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
Partially resolve#731
Using symlinks not only add unnecessary bloat to users' `$CARGO_HOME/bin` directory, but it actually breaks `sccache`, which inspects its binary name and decides how to act on.
For `sccache` to function, it must be invoked directly or use a hard link.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
Fixed#859
* Impl `cargo_config`
* Use `install.root` in `$CARGO_HOME/.cargo/config.toml`
before fallback to `cargo_home`.
* Improve logging in `get_cargo_roots_path`
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
The startup/init code in `entry::install_crates` performs a lot of blocking operations, from testing if dir exists to reading from files and there is no `.await` in there until after the startup.
There are also a few cases where `block_in_place` should be called (e.g. loading manifests, loading TLS certificates) but is missing.
Most of the `Args` passed to `entry::install_crates` are actually consumed before the first `.await` point, so performing startup/init code eagerly would make the generated future much smaller, reduce codegen and also makes it easier to optimize.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
- Add option `--no-discover-github-token` for disabling this behavior
- Add new dep gh-token v0.1.0 to crates/bin
- Extract github-token from git-credentials or gh config if `--github-token` or
environment variable `GITHUB_TOKEN` is not present.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
It also uses `max_stable_version` in the json downloaded from https://crates.io/api/v1/crates/$name if possible, which is equivalent to the version shown on https://crates.io/crates/$name .
- Add new feat `json` to `binstalk-downloader`
- Impl new async fn `Response::json`
- use `Response::json` in `GhApiClient` impl
- Mark all err types in binstalk-downloader as `non_exhaustive`
- Ret `remote::Error` in `remote::Certificate::{from_pem, from_der}` instead of `ReqwestError`.
- Refactor `BinstallError`: Merge variant `Unzip`, `Reqwest` & `Http`
into one variant `Download`.
- Manually download and parse json from httos://crates.io/api/v1
- Remove unused deps `crates_io_api`
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
Fixed#776
- Add new feature gh-api-client to binstalk-downloader
- Impl new type `binstalk_downloader::remote::{RequestBuilder, Response}`
- Impl `binstalk_downloader::gh_api_client::GhApiClient`, exposed if `cfg(feature = "gh-api-client")` and add e2e and unit tests for it
- Use `binstalk_downloader::gh_api_client::GhApiClient` to speedup `cargo-binstall`
- Add new option `--github-token` to supply the token for GitHub restful API, or read from env variable `GITHUB_TOKEN` if not present.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
for specifying root ceritificates used for https connnections.
And remove old environment variable `CARGO_HTTP_CAINFO`, `SSL_CERT_FILE`
and `SSL_CERT_PATH` to avoid accidentally setting them, especially in CI
env.
Also:
- Rm fn `binstalk_downloader::Certificate::from_env`
- Enable feature `env` of dep `clap` in `crates/bin`
- Add new dep `file-format` v0.14.0 to `crates/bin`
- Use `file-format` to determine pem/der file format when loading root certs
- Rm fn `binstalk_downloader::Certificate::open` and enum `binstalk_downloader::OpenCertificateError`
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Support for custom root cert in `binstalk_downloader::remote::Client`
* Support adding root cert via env `CARGO_HTTP_CAINFO`, `SSL_CERT_{FILE, PATH}`
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
Fixed#779#791
- Retry request on timeout
- Retry for `StatusCode::{REQUEST_TIMEOUT, GATEWAY_TIMEOUT}`
- Add `DEFAULT_RETRY_DURATION_FOR_RATE_LIMIT` for 503/429
if 503/429 does not give us a header or give us an invalid header on
when to retry, we would default to
`DEFAULT_RETRY_DURATION_FOR_RATE_LIMIT`.
- Fix `Client::get_redirected_final_url`: Retry using `GET` on status code 400..405 + 410
- Rename remote_exists => remote_gettable & support fallback to GET
if HEAD fails due to status code 400..405 + 410.
- Improve `Client::get_stream`: Include url & method in the err of the stream returned
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Enable feature "env" on dep clap in bin
* Read log_level from env `CARGO_BINSTALL_LOG_LEVEL` if arg not present
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
