This is for usage outside of binstall itself.
Sometimes it's useful to be able to specify more reqwest options, such as doing custom DNS resolution. This leaves the existing API identical but adds a couple methods that make it possible to construct a client with custom options.
Signed-off-by: Félix Saparelli <felix@passcod.name>
It's renamed to hickory-resolver.
Also enabled new feature `hickory-resolver/dns-over-h3` if `rustls` is
enabled.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
I planned to replace `futures-util` with `futures-lite`, but it turns
out that `reqwest` actually depends on `futures-util`, so there is no
point removing it and introduce yet another dependency.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
- ci: Check feat powerset of leon & binstalk-downloader in `ci.yml`
- fix leon feature `cli`: Enable dep `miette` in feature `cli`
- fix binstalk-downloader when default feature is disabled and no other
tls related feature is enabled (breaking change due to replace of
`tls::Version` with newtype `TLSVersion`).
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
Fixed#835
Using `HEAD` for this would often cause false negative that requires the `Client` to fallback to `GET`, which creates a lot of requests even if the url doesn't exist and then get cargo-binstall rate limited by GitHub/GitLab/etc.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
It also uses `max_stable_version` in the json downloaded from https://crates.io/api/v1/crates/$name if possible, which is equivalent to the version shown on https://crates.io/crates/$name .
- Add new feat `json` to `binstalk-downloader`
- Impl new async fn `Response::json`
- use `Response::json` in `GhApiClient` impl
- Mark all err types in binstalk-downloader as `non_exhaustive`
- Ret `remote::Error` in `remote::Certificate::{from_pem, from_der}` instead of `ReqwestError`.
- Refactor `BinstallError`: Merge variant `Unzip`, `Reqwest` & `Http`
into one variant `Download`.
- Manually download and parse json from httos://crates.io/api/v1
- Remove unused deps `crates_io_api`
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
Fixed#776
- Add new feature gh-api-client to binstalk-downloader
- Impl new type `binstalk_downloader::remote::{RequestBuilder, Response}`
- Impl `binstalk_downloader::gh_api_client::GhApiClient`, exposed if `cfg(feature = "gh-api-client")` and add e2e and unit tests for it
- Use `binstalk_downloader::gh_api_client::GhApiClient` to speedup `cargo-binstall`
- Add new option `--github-token` to supply the token for GitHub restful API, or read from env variable `GITHUB_TOKEN` if not present.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
for specifying root ceritificates used for https connnections.
And remove old environment variable `CARGO_HTTP_CAINFO`, `SSL_CERT_FILE`
and `SSL_CERT_PATH` to avoid accidentally setting them, especially in CI
env.
Also:
- Rm fn `binstalk_downloader::Certificate::from_env`
- Enable feature `env` of dep `clap` in `crates/bin`
- Add new dep `file-format` v0.14.0 to `crates/bin`
- Use `file-format` to determine pem/der file format when loading root certs
- Rm fn `binstalk_downloader::Certificate::open` and enum `binstalk_downloader::OpenCertificateError`
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Support for custom root cert in `binstalk_downloader::remote::Client`
* Support adding root cert via env `CARGO_HTTP_CAINFO`, `SSL_CERT_{FILE, PATH}`
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
Fixed#779#791
- Retry request on timeout
- Retry for `StatusCode::{REQUEST_TIMEOUT, GATEWAY_TIMEOUT}`
- Add `DEFAULT_RETRY_DURATION_FOR_RATE_LIMIT` for 503/429
if 503/429 does not give us a header or give us an invalid header on
when to retry, we would default to
`DEFAULT_RETRY_DURATION_FOR_RATE_LIMIT`.
- Fix `Client::get_redirected_final_url`: Retry using `GET` on status code 400..405 + 410
- Rename remote_exists => remote_gettable & support fallback to GET
if HEAD fails due to status code 400..405 + 410.
- Improve `Client::get_stream`: Include url & method in the err of the stream returned
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
`futures-util` has too many dependencies and it contains a lot of code
of which we only use a tiny bit of them.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
Fixed#747
- Add dep compact_str v0.6.1 to binstalk-downloader
- Impl new type `DelayRequest`
- Handle 503/429 with wait duration > `MAX_RETRY_DURATION` by simply taking the min
- Fix `Client::send_request_inner`: Ensure 503/429 get propagated to other requests
even if the current requests reach its maximum retry and decides to
simply return an error.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Avoid potential panicking in `args::parse` by using `Vec::get` instead of indexing
* Refactor: Simplify `opts::{resolve, install}` API
Many parameters can be shared and put into `opts::Options` intead and
that would also avoid a few `Arc<Path>`.
* Optimize `get_install_path`: Avoid cloning `install_path`
* Optimize `LazyJobserverClient`: Un`Arc` & remove `Clone` impl
to avoid additional boxing
* Optimize `find_version`: Avoid cloning `semver::Version`
* Optimize `GhCrateMeta::launch_baseline_find_tasks`
return `impl Iterator<Item = impl Future<Output = ...>>`
instead of `impl Iterator<Item = AutoAbortJoinHandle<...>>`
to avoid unnecessary spawning.
Each task spawned has to be boxed and then polled by tokio runtime.
They might also be moved.
While they increase parallelism, spawning these futures does not justify
the costs because:
- Each `Future` only calls `remote_exists`
- Each `remote_exists` call send requests to the same domain, which is
likely to share the same http2 connection.
Since the conn is shared anyway, spawning does not speedup anything
but merely add communication overhead.
- Plus the tokio runtime spawning cost
* Optimize `install_crates`: Destruct `Args` before any `.await` point
to reduce size of the future
* Refactor `logging`: Replace param `arg` with `log_level` & `json_output`
to avoid dep on `Args`
* Add dep strum & strum_macros to crates/bin
* Derive `strum_macros::EnumCount` for `Strategy`
* Optimize strategies parsing in `install_crates`
* Fix panic in `install_crates` when `Compile` is not the last strategy specified
* Optimize: Take `Vec<Self>` instead of slice in `CrateName::dedup`
* Refactor: Extract new fn `compute_resolvers`
* Refactor: Extract new fn `compute_paths_and_load_manifests`
* Refactor: Extract new fn `filter_out_installed_crates`
* Reorder `install_crates`: Only run target detection if args are valid
and there are some crates to be installed.
* Optimize `filter_out_installed_crates`: Avoid allocation
by returning an `Iterator`
* Fix user_agent of `remote::Client`: Let user specify it
* Refactor: Replace `UIThread` with `ui::confirm`
which is much simpler.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Box `HttpError` in `binsalk_downloader::remote::Error::Http`
as `HttpError` contains `Url` which are too big.
* Box `HttpError` in `BinstallError::Http` same as the previous commit.
* Box `TinyTemplateError` in `BinstallError::Template`
since `TinyTemplateError` is 56 bytes large where most of the other variants are below 40 bytes large.
* Rm unsed variant `BinstallError::VersionUnavailable`
* Box `CratesIoApiError` in `BinstallError::CratesIoApi`
It is 32 bytes large while other variants are below 40 bytes large.
* Improve err msg for `BinstallError::CrateContext`
* Rm unused variant `BinstallError::VersionReq`
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
Resolved#522
* Add dep tracing v0.1.37 to binstalk
* Use `tracing` instead of `log` for logging in binstalk
* Rm dev dep `env_logger` since `log` is no longer used
* Rm unused dep `log` from binstalk
* Replace use of `log` with `tracing` in crates/bin
* Enable feat std of dep log in crates/bin
* Add dep tracing v0.1.37 to binstalk-downloader
* Replace use of `log` with `tracing` in binstalk-downloader
* Rm unused dep `log` from binstalk-downlaoder
* Wrap `ops::{install, resolve}` in `tracing::instrument`
* Wrap `Download::and_{extract, visit_tar}` in `instrument`
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
