* Add new dep async_zip v0.0.9 to binstalk-downloader
with features "gzip", "zstd", "xz", "bzip2", "tokio".
* Refactor: Simplify `async_extracter::extract_*` API
* Refactor: Create newtype wrapper of `ZipError`
so that the zip can be upgraded without affecting API of this crate.
* Enable feature fs of dep tokio in binstalk-downloader
* Rewrite `extract_zip` to use `async_zip::read::stream::ZipFileReader`
which avoids writing the zip file to a temporary file and then read it
back into memory.
* Refactor: Impl new fn `await_on_option` and use it
* Optimize `tokio::select!`: Make them biased and check for cancellation first
to make cancellation takes effect ASAP.
* Rm unused dep zip from binstalk-downloader
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Add new dep tokio-tar v0.3.0 to binstalk-downloader
* Add new dep tokio-util v0.7.4 with feat io to binstalk-downloader
* Add dep async-trait v0.1.59 to binstalk-downloader
* Add new dep async-compression v0.3.15 to binstalk-downloader
with features "gzip", "zstd", "xz", "bzip2", "tokio".
* Rewrite `Download::and_visit_tar` to use `tokio-tar`
to avoid the cumbersome `block_in_place`.
* Apply temporary workaround: Rm use of let-else in mod visitor
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Add & Impl new fn `CratesToml::collect_into_crates_versions`
to iterate over crates listed in cargo_crates_v1, accessing their names
and versions.
* Re-export `CompactString`, `Version` & `Url` in binstalk-manifests
for convenience
* Fix `CratesToml::load_from_path`: Wrap `File` in `FileLock::new_shared`
to avoid concurrent write while reading the file.
* Filter out installed crates in cargo_install_v1_metadata
* Make match in `filter_out_installed_crates` more explicit
* Add new test `cargo_crates_v1::test::test_loading`
* Optimize `CratesToml`: Use `Vec` instead of `BTreeMap`
since we cannot simply call `BTreeMap::get` to find an entry for a crate
anyway.
This also accidentally fixed the CI.
* Impl new API `CratesToml::remove`
* Fix`CratesToml::append_to_path` by removing previous records of
the crates that are just updated.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Fix reporting parsing error from `args::parse`
Report it in `args::parse` by using `Args::command().error(...).exit()`
instead of returning `BinstallError`.
* Rm unused variant `BinstallError::OverrideOptionUsedWithMultiInstall`
* Refactor: Move `strategies` validation into `args::parse`
* Rm unused variant `BinstallError::InvalidStrategies`
* Add new unit test `args::test::verify_cli`
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Optimize `BinstallError::CratesIoApi`: Extract new type `errors::CratesIoApiError` and box it
Also improve `<CratesIoApiError as Display>::fmt` impl.
* Optimize `BinstallError::SubProcess`: Use `Box<str>` instead of `String`
* Optimize `BinstallError::CargoManifest`: Box `CargoTomlError`
* Optimize `BinstallError::VersionParse`: Extract `VersionParseError` and box it
Also improve `<VersionParseError as Display>::fmt` impl.
* Optimize `BinstallError::CrateContext`: Extract `CrateContextError` and box it in
* Optimize `install_from_source`: Only format `cmd` on err
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Optimize `Fetcher::find`: Make it non-async to avoid boxing
and return `AutoAbortJoinHandle<...>` instead.
Since spawning a new task in tokio box the future anyway, boxing the
returned future again in `Fetcher::find` merely adds unnecessary
overheads.
* Optimize `QuickInstall::report`: Make it async fn instead of ret `JoinHandle`
This provides several benefits:
- On debug build, no task will be spawned
- The calls to `self.stats_url()` can be evaluated in parallel.
- The task now returns `()` so that the task spawned is smaller.
* Fix `QuickInstall::find`: `warn!` if quickinstall report fails
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
as there is no need to randomize the first one to be polled.
For `cancel_on_user_sig_term` and `StreamReadable::fill_buf`, the
cancellation future should always to be polled first so that user would
feel responsive.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
- bin and binstalk uses new let-else lang feature
- dep zstd of binstalk-downloader just bumped its msrv to v1.64.0
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
So that the borrow checker know that `FuturesUnordered<F>` holds no
reference to any data and thus it can further optimize future returned
by `GhCrateMeta::find`.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
`cargo-binstall` should reject duplicate strategy in `--strategies` as it is illegal input and it cannot be interpreted reasonably.
* Reject duplicate strategy in `--strategies`
* Optimize `compute_resolvers`: Use `Vec::retain`
instead of `Iterator::filter` plus collecting into `Vec`.
* Optimize `compute_resolvers`: Reject `strategies.len() > Strategy::COUNT`
* Improve err msg for cases where user disabled all strategies
* Simplify `BinstallError::InvalidStrategies`: Takes `&'static str`
instead of `&'static &'static str` since other variants are larger than 8B.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Optimization: Rm `debug!` in `find_version`
printing all version iterated obviously doesn't help much in debugging
in the problem but rather just confusing.
Also this makes it hard for the compiler to optimize the iterators.
* Use let-else in `ManifestVisitor`
* Optimize `BinFile::preview_{bin, link}` for zero-copy
Return `impl Display` that lazily format instead of allocating a `String`
* Optimize `infer_bin_dir_template`: Generate dir lazily
* Optimize `find_version`: Lazily clone `version_req` only on err
* Refactor `find_version`: Use `bool::then_some`
* Add dep either v1.8.0 to binstalk
* Optimize `GhCrateMeta::find`: Avoid cloning and `Vec` creation
by using `Either`
* Optimize `ops::install::install_from_package`: Make it a regular fn
instead of async fn since it does not `.await` on any async fn.
* Optimize `QuickInstall`: Rm field `target`
since `Arc<Data>` already contains that field.
* Optimize `Fetcher`s: Extract new struct `TargetData`
so that `Data` can be shared by all fetchers, regardless of the target.
* Optimize `QuickInstall`: Rm unused field `data`
* Optimize `Resolution::print`: Replace branching with conditional move
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Avoid potential panicking in `args::parse` by using `Vec::get` instead of indexing
* Refactor: Simplify `opts::{resolve, install}` API
Many parameters can be shared and put into `opts::Options` intead and
that would also avoid a few `Arc<Path>`.
* Optimize `get_install_path`: Avoid cloning `install_path`
* Optimize `LazyJobserverClient`: Un`Arc` & remove `Clone` impl
to avoid additional boxing
* Optimize `find_version`: Avoid cloning `semver::Version`
* Optimize `GhCrateMeta::launch_baseline_find_tasks`
return `impl Iterator<Item = impl Future<Output = ...>>`
instead of `impl Iterator<Item = AutoAbortJoinHandle<...>>`
to avoid unnecessary spawning.
Each task spawned has to be boxed and then polled by tokio runtime.
They might also be moved.
While they increase parallelism, spawning these futures does not justify
the costs because:
- Each `Future` only calls `remote_exists`
- Each `remote_exists` call send requests to the same domain, which is
likely to share the same http2 connection.
Since the conn is shared anyway, spawning does not speedup anything
but merely add communication overhead.
- Plus the tokio runtime spawning cost
* Optimize `install_crates`: Destruct `Args` before any `.await` point
to reduce size of the future
* Refactor `logging`: Replace param `arg` with `log_level` & `json_output`
to avoid dep on `Args`
* Add dep strum & strum_macros to crates/bin
* Derive `strum_macros::EnumCount` for `Strategy`
* Optimize strategies parsing in `install_crates`
* Fix panic in `install_crates` when `Compile` is not the last strategy specified
* Optimize: Take `Vec<Self>` instead of slice in `CrateName::dedup`
* Refactor: Extract new fn `compute_resolvers`
* Refactor: Extract new fn `compute_paths_and_load_manifests`
* Refactor: Extract new fn `filter_out_installed_crates`
* Reorder `install_crates`: Only run target detection if args are valid
and there are some crates to be installed.
* Optimize `filter_out_installed_crates`: Avoid allocation
by returning an `Iterator`
* Fix user_agent of `remote::Client`: Let user specify it
* Refactor: Replace `UIThread` with `ui::confirm`
which is much simpler.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Optimize `Download::and_extract`: Avoid dup monomorphization
* Increase buffer size for binstall_crates_v1 to `4096 * 5`
* Optimize `opts::resolve`: Avoid unnecessary `clone`s
* Fix reserve in `opts::resolve`: Do not over-reserve
* Rename field `opts::Options::resolver` => `resolvers`
* Refactor: Extract new type `resolve::PackageInfo`
- which makes `opts::resolve_inner` easier to understand
- reduce number of parameters required for `download_extract_and_verify` and
`collect_bin_files`
- reducing size of future returned by `opts::resolve_inner` by dropping
`cargo_toml::{Manifest, Package}` as early as possible since
`Manifest` is 3000 Bytes large while `Package` is 600 Bytes large.
* Optimize `fetchers::Data`: Use `CompactString` for field name & version
since they are usually small enough to fit in inlined version of
`CompactString`.
* Optimize `gh_crate_meta`: Avoid unnecessary allocation
in `RepositoryHost::get_default_pkg_url_template`.
* Refacator: Use `Itertools::cartesian_product` in `apply_filenames_to_paths`
* Optimize `ops::resolve`: Avoid unnecessary `clone` & reduce future size
by calling `fetcher.target_meta()` to obtain final metadata after
downloaded and extracted the binaries.
* Optimize `ops::resolve`: Avoid unnecessary allocation
in `download_extract_and_verify`: Replace `Itertools::join` with
`Itertools::format` to avoid allocating the string.
* Fix disabling cargo-install fallback
* Simplify `BinFile::from_product`: Takes `&str` instead of `&Product`
since we only need `product.name`
* Rename `BinFile::from_product` => `BinFile::new`
* Refactor: Create newtype `ops::resolve::Bin`
so that we don't need to `unwrap()` on `Product::name`
and reduce memory usage.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
Add short name for no-confirm flag
Closes#536
The -y short name is a convention, often used by package managers (apt, yum, ...) and it is associated with a request of no interaction (thus often used in scripts, to automate workflows)
* Refactor: Extract new crate binstalk-types
* Optimize: Rm field `CrateInfo::other`
which also removes dep serde-tuple-vec-map and serde-json from
binstalk-types.
This also makes `CrateInfo` easier to use, more generic and can be used
over any `Serializer`, not just `serde_json::Value`.
* Mark all errors in `binstalk-manifests` as non_exhaustive
* Reduce size of `CvsParseError` by using `Box<str>`
instead of `String` for variant `UnknownSourceType`.
* Reduce size of `CratesTomlParseError` to 16 bytes on 64bit platform
by boxing variants `TomlWrite` and `CvsParse` as these two fields are
significantly larger than other variants.
* Unify import style in mod `binstall_crates_v1`
* Replace dep binstalk-manifests with binstalk-types in binstalk-downloader
to reduce its transitive dependencies and enables binstalk-downloader to
be built in parallel to binstak-manifests.
* Replace dep binstalk-manifests with binstalk-types in binstalk
to reduce transitive dependencies and enables binstalk to be built in
parallel to binstalk-manifests.
This is benefitial because binstalk-manifests pulls in toml_edit, which
could takes up to 15s to be built on M1 (7-9s for codegen).
* Add dep binstalk-manifests to crates/bin
* Update dependabot and GHA release-pr
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
