This is for usage outside of binstall itself.
Sometimes it's useful to be able to specify more reqwest options, such as doing custom DNS resolution. This leaves the existing API identical but adds a couple methods that make it possible to construct a client with custom options.
Signed-off-by: Félix Saparelli <felix@passcod.name>
* Use rc-zip-sync for zip extraction
Fixed#1080
In this commit, binstalk-downloader is updated to
- first download the zip into a temporary file, since
there is no correct way to extract zip from a stream.
- then use rc-zip-sync to read from the zip and extract
it to filesystem.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Fix returned `ExtractedFiles` in `do_extract_zip`
Signed-off-by: Jiahao XU <30436523+NobodyXu@users.noreply.github.com>
* Fix clippy in zip_extraction.rs
Signed-off-by: Jiahao XU <30436523+NobodyXu@users.noreply.github.com>
---------
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
Signed-off-by: Jiahao XU <30436523+NobodyXu@users.noreply.github.com>
* Bump msrv of binstalk-downloader, binstalk and cargo-binstall to 1.79
* Update From<io::Error> for DownloadError
to use io::Error::downcast
* Update From<io::Error> for BinstallError
* fix dmt
* Bump msrv of binstalk-downloader to 1.70.0
* Fix trait bound in multiple places
for copy_file_to_mpsc
* Rm super-trait trait bounds for FusedStream
* Bump detect-targets msrv to 1.62.0
* Bump cargo-binstall msrv to 1.70.0
* Bump dep reqwest in binstalk-downloader from 0.11.26 to 0.12.0
reqwest 0.11.27 is still used in gix, but it will be upgraded in
next gix release.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Fix feature selection: Forward tls feature to `simple-git`
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
---------
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
Fixed#1575
Always consume the stream if a `data_verifier` is provided in
`binstalk_download::Download`
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
It's renamed to hickory-resolver.
Also enabled new feature `hickory-resolver/dns-over-h3` if `rustls` is
enabled.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Bump dep trust-dns-reslver to v0.23.0
Since we no longer enables `reqwest/trust-dns-resolver` anymore, we
don't need to keep this dependency is sync with upstream `reqwest`.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Fix compilation
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
---------
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
Fixed#1414
There are many different kinds of gh tokens and they can change the
format at anytime, so we can't check it based on its formats.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Add CLI options
* Add manifest types
* Thread signature policy through to fetchers
* Thread signing section through from metadata
* Implement signing validation
* Clippy
* Attempt testing
* Yes and
* Why
* fmt
* Update crates/bin/src/args.rs
Co-authored-by: Jiahao XU <Jiahao_XU@outlook.com>
* Update crates/binstalk-fetchers/src/gh_crate_meta.rs
Co-authored-by: Jiahao XU <Jiahao_XU@outlook.com>
* Update crates/bin/src/args.rs
Co-authored-by: Jiahao XU <Jiahao_XU@outlook.com>
* Update crates/binstalk-fetchers/src/signing.rs
Co-authored-by: Jiahao XU <Jiahao_XU@outlook.com>
* Update crates/binstalk-fetchers/src/signing.rs
Co-authored-by: Jiahao XU <Jiahao_XU@outlook.com>
* Update crates/binstalk-fetchers/src/signing.rs
Co-authored-by: Jiahao XU <Jiahao_XU@outlook.com>
* Update crates/binstalk-fetchers/src/signing.rs
Co-authored-by: Jiahao XU <Jiahao_XU@outlook.com>
* fixes
* Finish feature
* Document
* Include all fields in the signing.file template
* Readme document
* Review fixes
* Fail on non-utf8 sig
* Thank goodness for tests
* Run test in ci
* Add rsign2 commands
* Log utf8 error
* Update e2e-tests/signing.sh
Co-authored-by: Jiahao XU <Jiahao_XU@outlook.com>
* Fix `e2e-tests/signing.sh` MacOS CI failure
Move the tls cert creation into `signing.sh` and sleep for 10s to wait
for https server to start.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Refactor e2e-tests-signing files
- Use a tempdir generated by `mktemp` for all certificates-related
files
- Put other checked-in files into `e2e-tests/signing`
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Fixed `e2e-tests-signing` connection err in MacOS CI
Wait for server to start up by trying to connect to it.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Fix `e2e-tests-signing` passing `-subj` to `openssl` on Windows
Use single quote instead of double quote to avoid automatic expansion
from bash
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Fix `e2e-tests-signing` waiting for server to startup
Remove `timeout` since it is not supported on MacOS.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Try to fix windows CI by setting `MSYS_NO_PATHCONV=1` on `openssl` cmds
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Fixed `e2e-tests-signing` on windows
By using double `//` for the value passed to option `-subj`
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Fixed infinite loop in `signing/wait-for-server` on Windows
Pass `--ssl-revoke-best-effort` to prevent schannel from checking ssl
revocation status.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Add cap on retry attempt in `signing/wait-for-server.sh`
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Let `singing/server.py` print output to stderr
so that we can see the error message there.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Fix running `signing/server.py` on MacOS CI
use `python3` since macos-latest still has python2 installed and
`python` is a symlink to `python2` there.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
---------
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
Co-authored-by: Jiahao XU <Jiahao_XU@outlook.com>
* Fix GitHub token auto discovery
Fixed#1333
- Rm dep `gh-token` since it is broken and we can simply run
`gh auth token` in `cargo-binstall` instead.
- binstalk-downloader: Make sure GitHub token is at least 40B long
and other than the `_`, composes of only alphanumeric characters.
- Warn on failure to read `git/credential` files
- Optimize `try_from_home` to avoid heap allocation of `PathBuf`
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Fix typo and clippy
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Simplify `is_valid_gh_token` & `is_ascii_alphanumeric` impl
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Improve err msg in `get_inner`
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Improve err msg of `cargo_binstall::gh_token::get`
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
---------
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
`binstalk-downloader` contains stuff about http(s) before the
git code is moved into it and now it becomes http and git.
While git indeed uses http stuff, which is why I decided to put
it into binstalk-downloader, it is more than just downloading
since it is stateful (can be cached locally and updated)
where as http is stateless.
Also `binstalk-downloader`'s codegen time now increases
dramatically and it also creates extra dependencies for
binstalk-fetchers, delaying its execution.
The git code also don't use anything from `binstalk-downloader`
at all, it makes sense to be an independent crate.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
feat: `git::Repository` support cancellation.
To make sure users can cancel git operation via signal, e.g. when the
git operation fail or users no longer want to install.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
Fixed#1183
Since the crate tarball could be downloaded from a different set of
servers than where the cargo registry is hosted, verifying the checksum
is necessary to verify its integrity.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
I planned to replace `futures-util` with `futures-lite`, but it turns
out that `reqwest` actually depends on `futures-util`, so there is no
point removing it and introduce yet another dependency.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
By accepting `&mut dyn DataVerifier` for users to pass any callback that
uses `digest::Digest`/`digest::Mac`, `sigstore` or whatever they want.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
Replace use of `PhantomData::default()` in `src/download.rs` with
`PhantomData` since it is a unit struct.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
When installing `cargo-expand` v1.0.59, I got an error message:
```
Failed to parse http response body as Json: invalid type: null, expected a string at line
1 column 90
```
This is because `GraphQLPageInfo::end_cursor` can actually be `null`, so
I change its type to `Option<CompactString>`.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
- ci: Check feat powerset of leon & binstalk-downloader in `ci.yml`
- fix leon feature `cli`: Enable dep `miette` in feature `cli`
- fix binstalk-downloader when default feature is disabled and no other
tls related feature is enabled (breaking change due to replace of
`tls::Version` with newtype `TLSVersion`).
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
- Increase `DEFAULT_RETRY_DURATION` to 5 minutes, since GitHub enforces
rate limit on an hourly basis.
- Refactor `check_for_status` & `fetch_release_artifacts_restful_api`
- Optimize `percent_decode_http_url_path`: Avoid `percent_decode_str`
if there is no `%` in the `input`.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
Fixed#838
- Add new key `subcrate` for rendering `pkg-url`
- Add new release paths in GitHub, GitLab & SourceForge using key `subcrate` for auto-detection
- Add subcrate detection for GitHub and GitLab
- Add `debug!` when using gh api token in `GhApiClient::new`
- Add subcrate testing to `e2e-tests/subcrate.sh`
- Bump cargo-release to 0.24.9 in e2e-tests/live.sh
to fix test failure on MacOS without libssl installed in `/usr/local/`.
- Optimize GhCrateMeta: Detect subcrate and repo-host in `Data::get_repo_info`
to cache the result and avoid duplicate works, this also makes the code
more ergonomic by removing the need to some `unwrap()` plus making it
more efficient since we don't need to clone the url just to modify it.
- Add instrument to `Data::get_repo_info`
- Fix `shellcheck` err in `e2e-tests/*.sh`
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
- Fix zip extraction code: Ensure dir is rwx and file is readable for curr user
- Add more integration test for `ExtractedFiles`
- Fix `bins::infer_bin_dir_template` introduced in #856
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
