Fixed#775
- Add dep target-lexicon v0.12.7
- Add `target-{family, arch, libc, vendor}` to
`package.metadata.binstall`.
For `{universal, universal2}-apple-darwin`, the `target-arch` is set to
`universal`.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
Fixed https://github.com/cargo-bins/cargo-quickinstall/issues/195
- Fix `fetchers::QuickInstall`: Stop sending stats for `universal-apple-darwin`
since quickinstall only supports targets officially supports by rust.
- Only send stats report to quickinstall if the `Fetcher::find` is `.await`ed on
This prevents stats report to be sent for cases where the `QuickInstall` fetcher
is actually unused, e.g. resolved to `GhCrateMeta` or other `QuickInstall` fetcher
with different target.
This also reduces amount of http requests created in background.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
- Refactor: Extract new fn `binstalk::helpers::remote::does_url_exist`
- Use new quickinstall release schema in `binstalk::fetchers::QuickInstall`
- Optimize `fetchers::QuickInstall`: Generate url once in `Fetcher::new`
Avoid repeated string allocation plus `Url` parsing.
This also makes changing package_url and stats_url easier.
- Optimize `QuickInstall::report`: Use HEAD instead of GET
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
Fixed#776
- Add new feature gh-api-client to binstalk-downloader
- Impl new type `binstalk_downloader::remote::{RequestBuilder, Response}`
- Impl `binstalk_downloader::gh_api_client::GhApiClient`, exposed if `cfg(feature = "gh-api-client")` and add e2e and unit tests for it
- Use `binstalk_downloader::gh_api_client::GhApiClient` to speedup `cargo-binstall`
- Add new option `--github-token` to supply the token for GitHub restful API, or read from env variable `GITHUB_TOKEN` if not present.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
Fixed#779#791
- Retry request on timeout
- Retry for `StatusCode::{REQUEST_TIMEOUT, GATEWAY_TIMEOUT}`
- Add `DEFAULT_RETRY_DURATION_FOR_RATE_LIMIT` for 503/429
if 503/429 does not give us a header or give us an invalid header on
when to retry, we would default to
`DEFAULT_RETRY_DURATION_FOR_RATE_LIMIT`.
- Fix `Client::get_redirected_final_url`: Retry using `GET` on status code 400..405 + 410
- Rename remote_exists => remote_gettable & support fallback to GET
if HEAD fails due to status code 400..405 + 410.
- Improve `Client::get_stream`: Include url & method in the err of the stream returned
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
- Refactor: Mv fn `utils::asyncify` into mod `utils`
- Improve err msg for task failure in `utils::asyncify`
- Make sure `asyncify` always returns the same annoymous type
that implements `Future` if the `T` is same.
- Rewrite `extract_bin` to avoid `block_in_place`
support cancellation by dropping
- Rm unused dep scopeguard from binstalk-downloader
- Rewrite `extract_tar_based_stream` so that it is cancellable by dropping
- Unbox `extract_future` in `async_extracter::extract_zip`
- Refactor `Download` API: Remove `CancellationFuture` as param
since all futures returned by `Download::and_*` does not call
`block_in_place`, so they can be cancelled by drop instead of using this
cumbersome hack.
- Fix exports from mod `async_tar_visitor`
- Make `signal::{ignore_signals, wait_on_cancellation_signal}` private
- Rm the global variable `CANCELLED` in `wait_on_cancellation_signal`
and rm fn `wait_on_cancellation_signal_inner`
- Optimize `wait_on_cancellation_signal`: Avoid `tokio::select!` on `not(unix)`
- Rm unnecessary `tokio::select!` in `wait_on_cancellation_signal` on unix
Since `unix::wait_on_cancellation_signal_unix` already waits for ctrl + c signal.
- Optimize `extract_bin`: Send `Bytes` to blocking thread for zero-copy
- Optimize `extract_with_blocking_decoder`: Avoid dup monomorphization
- Box fut of `fetch_crate_cratesio` in `PackageInfo::resolve`
- Optimize `extract_zip_entry`: Spawn only one blocking task per fn call
by using a mspc queue for the data to be written to the `outfile`.
This would improve efficiency as using `tokio::fs::File` is expensive:
It spawns a new blocking task, which needs one heap allocation and then
pushed to a mpmc queue, and then wait for it to be done on every loop.
This also fix a race condition where the unix permission is set before
the whole file is written, which might be used by attackers.
- Optimize `extract_zip`: Use one `BytesMut` for entire extraction process
To avoid frequent allocation and deallocation.
- Optimize `extract_zip_entry`: Inc prob of reusing alloc in `BytesMut`
Performs the reserve before sending the buf over mpsc queue to
increase the possibility of reusing the previous allocation.
NOTE: `BytesMut` only reuses the previous allocation if it is the
only one holds the reference to it, which is either on the first
allocation or all the `Bytes` in the mpsc queue has been consumed,
written to the file and dropped.
Since reading from entry would have to wait for external file I/O,
this would give the blocking thread some time to flush `Bytes`
out.
- Disable unused feature fs of dep tokio
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Optimize `Fetcher::find`: Make it non-async to avoid boxing
and return `AutoAbortJoinHandle<...>` instead.
Since spawning a new task in tokio box the future anyway, boxing the
returned future again in `Fetcher::find` merely adds unnecessary
overheads.
* Optimize `QuickInstall::report`: Make it async fn instead of ret `JoinHandle`
This provides several benefits:
- On debug build, no task will be spawned
- The calls to `self.stats_url()` can be evaluated in parallel.
- The task now returns `()` so that the task spawned is smaller.
* Fix `QuickInstall::find`: `warn!` if quickinstall report fails
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Optimization: Rm `debug!` in `find_version`
printing all version iterated obviously doesn't help much in debugging
in the problem but rather just confusing.
Also this makes it hard for the compiler to optimize the iterators.
* Use let-else in `ManifestVisitor`
* Optimize `BinFile::preview_{bin, link}` for zero-copy
Return `impl Display` that lazily format instead of allocating a `String`
* Optimize `infer_bin_dir_template`: Generate dir lazily
* Optimize `find_version`: Lazily clone `version_req` only on err
* Refactor `find_version`: Use `bool::then_some`
* Add dep either v1.8.0 to binstalk
* Optimize `GhCrateMeta::find`: Avoid cloning and `Vec` creation
by using `Either`
* Optimize `ops::install::install_from_package`: Make it a regular fn
instead of async fn since it does not `.await` on any async fn.
* Optimize `QuickInstall`: Rm field `target`
since `Arc<Data>` already contains that field.
* Optimize `Fetcher`s: Extract new struct `TargetData`
so that `Data` can be shared by all fetchers, regardless of the target.
* Optimize `QuickInstall`: Rm unused field `data`
* Optimize `Resolution::print`: Replace branching with conditional move
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
Resolved#522
* Add dep tracing v0.1.37 to binstalk
* Use `tracing` instead of `log` for logging in binstalk
* Rm dev dep `env_logger` since `log` is no longer used
* Rm unused dep `log` from binstalk
* Replace use of `log` with `tracing` in crates/bin
* Enable feat std of dep log in crates/bin
* Add dep tracing v0.1.37 to binstalk-downloader
* Replace use of `log` with `tracing` in binstalk-downloader
* Rm unused dep `log` from binstalk-downlaoder
* Wrap `ops::{install, resolve}` in `tracing::instrument`
* Wrap `Download::and_{extract, visit_tar}` in `instrument`
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
From @passcod :
* Make bin-dir an Option
* Use cargo-release as a test case
* WIP: Try multiple default bin dirs
* Update bins.rs
* Update cargo_toml_binstall.rs
From @NobodyXu :
* Optimize & Prepare for support multi `bin-path`s
* Ensure quickinstall bin_dir work as usual.
* Rm dup entries in `FULL_FILENAMES`
* Add second version of `NOVERSION_FILENAMES`
* Impl multiple default `bin-dir`s
* Improve doc for `BinFile::from_product`
* Fix tests.sh
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
Co-authored-by: Félix Saparelli <felix@passcod.name>
* Refactor: Extract new fn `BinFile::check_source_exists`
* Impl new async fn `AutoAbortJoinHandle::flattened_join`
* Impl new fn `Fetcher::fetcher_name`
* Verify that `bin_files` exist in `resolve` stage
To ensure that the installation stage won't fail because of missing
binaries.
* Rm unused `MultiFecther`
* Simplify `Future` impl for `AutoAbortJoinHandle`
* Add new variant `BinstallError::CargoTomlMissingPackage`
* Replace `unwrap` in `resolve_inner` with proper error handling
* Make `Fetcher::new` as a regular function
instead of an `async` function.
* Ret `Arc<dyn Fetcher>` in trait fn `Fetcher::new`
* Refactor `resolve_inner`
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
