for specifying root ceritificates used for https connnections.
And remove old environment variable `CARGO_HTTP_CAINFO`, `SSL_CERT_FILE`
and `SSL_CERT_PATH` to avoid accidentally setting them, especially in CI
env.
Also:
- Rm fn `binstalk_downloader::Certificate::from_env`
- Enable feature `env` of dep `clap` in `crates/bin`
- Add new dep `file-format` v0.14.0 to `crates/bin`
- Use `file-format` to determine pem/der file format when loading root certs
- Rm fn `binstalk_downloader::Certificate::open` and enum `binstalk_downloader::OpenCertificateError`
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Support for custom root cert in `binstalk_downloader::remote::Client`
* Support adding root cert via env `CARGO_HTTP_CAINFO`, `SSL_CERT_{FILE, PATH}`
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
Fixed#779#791
- Retry request on timeout
- Retry for `StatusCode::{REQUEST_TIMEOUT, GATEWAY_TIMEOUT}`
- Add `DEFAULT_RETRY_DURATION_FOR_RATE_LIMIT` for 503/429
if 503/429 does not give us a header or give us an invalid header on
when to retry, we would default to
`DEFAULT_RETRY_DURATION_FOR_RATE_LIMIT`.
- Fix `Client::get_redirected_final_url`: Retry using `GET` on status code 400..405 + 410
- Rename remote_exists => remote_gettable & support fallback to GET
if HEAD fails due to status code 400..405 + 410.
- Improve `Client::get_stream`: Include url & method in the err of the stream returned
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Enable feature "env" on dep clap in bin
* Read log_level from env `CARGO_BINSTALL_LOG_LEVEL` if arg not present
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
Fixed#588
* Add new dep fs-lock v0.1.0 to crates/bin
* Refactor: Impl new type `Manifests` in crates/bin
for managing manifests.
* Fix#588 race cond updating `.crates.toml`
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Refactor: Extract new mod `ops::resolve::resolution`
* Refactor: Extract new type `ResolutionFetch`
* Refactor: Extract new type `ResolutionSource`
* Improve `Resolution::print`: Provides more details on which crate
is the resolution for.
* Make `Resolution::print` a pub fn
* Refactor: Extract new fn `ResolutionFetch::install`
* Refactor: Extract new async fn `ResolutionSource::install`
* Optimize `ResolutionSource::install` avoiding `OsStr::to_string_lossy`.
* Add new dep command-group v2.0.0 to binstalk with feat with-tokio
* Fix `ResolutionSource::install`: Use `AsyncCommandGroup::group_spawn`
instead of `tokio::process::Command::spawn` to ensure all the processes
spanwed by the `cargo` process can be terminated with one signal sent to
the `cargo` process.
* Fix printing resolution: Make sure they are printed without interleaving
* Refactor `entry::install_crates`
* Improve dry-run output for `ResolutionSource::install`
* Refactor: Extract new fn `ResolutionFetch::print`
* Refactor: Extract new fn `ResolutionSource::print`
* Optimize `Resolution`: Box unit variant `Fetch`
* Improve formatting of `tokio::process::Command`
Prints out sth like `cargo install ...` instead of the `Debug::fmt`.
* Improve dry-run output
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
Honor `.crates.toml` only since `cargo-uninstall` can only update
`.crates.toml`, not binstall's own manifest.
Fixed#625
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Add & Impl new fn `CratesToml::collect_into_crates_versions`
to iterate over crates listed in cargo_crates_v1, accessing their names
and versions.
* Re-export `CompactString`, `Version` & `Url` in binstalk-manifests
for convenience
* Fix `CratesToml::load_from_path`: Wrap `File` in `FileLock::new_shared`
to avoid concurrent write while reading the file.
* Filter out installed crates in cargo_install_v1_metadata
* Make match in `filter_out_installed_crates` more explicit
* Add new test `cargo_crates_v1::test::test_loading`
* Optimize `CratesToml`: Use `Vec` instead of `BTreeMap`
since we cannot simply call `BTreeMap::get` to find an entry for a crate
anyway.
This also accidentally fixed the CI.
* Impl new API `CratesToml::remove`
* Fix`CratesToml::append_to_path` by removing previous records of
the crates that are just updated.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Fix reporting parsing error from `args::parse`
Report it in `args::parse` by using `Args::command().error(...).exit()`
instead of returning `BinstallError`.
* Rm unused variant `BinstallError::OverrideOptionUsedWithMultiInstall`
* Refactor: Move `strategies` validation into `args::parse`
* Rm unused variant `BinstallError::InvalidStrategies`
* Add new unit test `args::test::verify_cli`
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
`cargo-binstall` should reject duplicate strategy in `--strategies` as it is illegal input and it cannot be interpreted reasonably.
* Reject duplicate strategy in `--strategies`
* Optimize `compute_resolvers`: Use `Vec::retain`
instead of `Iterator::filter` plus collecting into `Vec`.
* Optimize `compute_resolvers`: Reject `strategies.len() > Strategy::COUNT`
* Improve err msg for cases where user disabled all strategies
* Simplify `BinstallError::InvalidStrategies`: Takes `&'static str`
instead of `&'static &'static str` since other variants are larger than 8B.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Avoid potential panicking in `args::parse` by using `Vec::get` instead of indexing
* Refactor: Simplify `opts::{resolve, install}` API
Many parameters can be shared and put into `opts::Options` intead and
that would also avoid a few `Arc<Path>`.
* Optimize `get_install_path`: Avoid cloning `install_path`
* Optimize `LazyJobserverClient`: Un`Arc` & remove `Clone` impl
to avoid additional boxing
* Optimize `find_version`: Avoid cloning `semver::Version`
* Optimize `GhCrateMeta::launch_baseline_find_tasks`
return `impl Iterator<Item = impl Future<Output = ...>>`
instead of `impl Iterator<Item = AutoAbortJoinHandle<...>>`
to avoid unnecessary spawning.
Each task spawned has to be boxed and then polled by tokio runtime.
They might also be moved.
While they increase parallelism, spawning these futures does not justify
the costs because:
- Each `Future` only calls `remote_exists`
- Each `remote_exists` call send requests to the same domain, which is
likely to share the same http2 connection.
Since the conn is shared anyway, spawning does not speedup anything
but merely add communication overhead.
- Plus the tokio runtime spawning cost
* Optimize `install_crates`: Destruct `Args` before any `.await` point
to reduce size of the future
* Refactor `logging`: Replace param `arg` with `log_level` & `json_output`
to avoid dep on `Args`
* Add dep strum & strum_macros to crates/bin
* Derive `strum_macros::EnumCount` for `Strategy`
* Optimize strategies parsing in `install_crates`
* Fix panic in `install_crates` when `Compile` is not the last strategy specified
* Optimize: Take `Vec<Self>` instead of slice in `CrateName::dedup`
* Refactor: Extract new fn `compute_resolvers`
* Refactor: Extract new fn `compute_paths_and_load_manifests`
* Refactor: Extract new fn `filter_out_installed_crates`
* Reorder `install_crates`: Only run target detection if args are valid
and there are some crates to be installed.
* Optimize `filter_out_installed_crates`: Avoid allocation
by returning an `Iterator`
* Fix user_agent of `remote::Client`: Let user specify it
* Refactor: Replace `UIThread` with `ui::confirm`
which is much simpler.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Optimize `Download::and_extract`: Avoid dup monomorphization
* Increase buffer size for binstall_crates_v1 to `4096 * 5`
* Optimize `opts::resolve`: Avoid unnecessary `clone`s
* Fix reserve in `opts::resolve`: Do not over-reserve
* Rename field `opts::Options::resolver` => `resolvers`
* Refactor: Extract new type `resolve::PackageInfo`
- which makes `opts::resolve_inner` easier to understand
- reduce number of parameters required for `download_extract_and_verify` and
`collect_bin_files`
- reducing size of future returned by `opts::resolve_inner` by dropping
`cargo_toml::{Manifest, Package}` as early as possible since
`Manifest` is 3000 Bytes large while `Package` is 600 Bytes large.
* Optimize `fetchers::Data`: Use `CompactString` for field name & version
since they are usually small enough to fit in inlined version of
`CompactString`.
* Optimize `gh_crate_meta`: Avoid unnecessary allocation
in `RepositoryHost::get_default_pkg_url_template`.
* Refacator: Use `Itertools::cartesian_product` in `apply_filenames_to_paths`
* Optimize `ops::resolve`: Avoid unnecessary `clone` & reduce future size
by calling `fetcher.target_meta()` to obtain final metadata after
downloaded and extracted the binaries.
* Optimize `ops::resolve`: Avoid unnecessary allocation
in `download_extract_and_verify`: Replace `Itertools::join` with
`Itertools::format` to avoid allocating the string.
* Fix disabling cargo-install fallback
* Simplify `BinFile::from_product`: Takes `&str` instead of `&Product`
since we only need `product.name`
* Rename `BinFile::from_product` => `BinFile::new`
* Refactor: Create newtype `ops::resolve::Bin`
so that we don't need to `unwrap()` on `Product::name`
and reduce memory usage.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
Add short name for no-confirm flag
Closes#536
The -y short name is a convention, often used by package managers (apt, yum, ...) and it is associated with a request of no interaction (thus often used in scripts, to automate workflows)
* Refactor: Extract new crate binstalk-types
* Optimize: Rm field `CrateInfo::other`
which also removes dep serde-tuple-vec-map and serde-json from
binstalk-types.
This also makes `CrateInfo` easier to use, more generic and can be used
over any `Serializer`, not just `serde_json::Value`.
* Mark all errors in `binstalk-manifests` as non_exhaustive
* Reduce size of `CvsParseError` by using `Box<str>`
instead of `String` for variant `UnknownSourceType`.
* Reduce size of `CratesTomlParseError` to 16 bytes on 64bit platform
by boxing variants `TomlWrite` and `CvsParse` as these two fields are
significantly larger than other variants.
* Unify import style in mod `binstall_crates_v1`
* Replace dep binstalk-manifests with binstalk-types in binstalk-downloader
to reduce its transitive dependencies and enables binstalk-downloader to
be built in parallel to binstak-manifests.
* Replace dep binstalk-manifests with binstalk-types in binstalk
to reduce transitive dependencies and enables binstalk to be built in
parallel to binstalk-manifests.
This is benefitial because binstalk-manifests pulls in toml_edit, which
could takes up to 15s to be built on M1 (7-9s for codegen).
* Add dep binstalk-manifests to crates/bin
* Update dependabot and GHA release-pr
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Enable feat ansi of dep tracing-subscriber
* Rm use of `tracing_appender::non_blocking`
since `cargo-binstall` is so simple that it doesn't need it.
* Use `tracing::{error, info}` in `MainExit::report`
* Use `tracing::{error, warn}` in `BinstallError::report`
* Add dep supports-color v1.3.1 to crates/bin
* Enable ansi of `tracing_subscriber::fmt` if stdout supports it
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
Resolved#522
* Add dep tracing v0.1.37 to binstalk
* Use `tracing` instead of `log` for logging in binstalk
* Rm dev dep `env_logger` since `log` is no longer used
* Rm unused dep `log` from binstalk
* Replace use of `log` with `tracing` in crates/bin
* Enable feat std of dep log in crates/bin
* Add dep tracing v0.1.37 to binstalk-downloader
* Replace use of `log` with `tracing` in binstalk-downloader
* Rm unused dep `log` from binstalk-downlaoder
* Wrap `ops::{install, resolve}` in `tracing::instrument`
* Wrap `Download::and_{extract, visit_tar}` in `instrument`
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
Fixed#136
* Impl opt-out optioins in binstalk
* Replace field `Options::{gh_crate, quickinstall}_fetcher` with `resolver`
which can determine order of resolver used.
* Add new field `Args::{disable_}strategies`
* Add new variant `BinstallError::InvalidStrategies`
* Add variant `BinstallError::NoFallbackToCargoInstall`
* Add code for supporting strategies in mod entry
* Test `--disable-strategies` in `tests.sh`
* Test for `--strategies` in `tests.sh`
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Disable feat log-always of dep tracing
* Add dep tracing-log 0.1.3 with no feat
* Add new dep tracing-appender v0.2.2
* Add dep tracing-subscriber 0.3.16 with feat fmt and json
* Fix `MainExit::report`: Do not use `log::{error, warn}`
since `MainExit::report` might be called with no `log`ger, it can only
use `println!` and `eprintln!`.
* Use `tracing_subscriber::fmt` instead of `simple_log`
* Rm unused dep simplelog from crates/bin
* Fix `BinstallError::report`: Avoid `log::{warn, error}`
since they might be called after `tracing_appender::WorkerGuard` is
dropped.
* Make tracing output more readable to end users
* Add new dep tracing-core v0.1.30
* Add new dep once_cell v1.16.0
* Refactor: Extract new mod `logging`
* Add new option `Args::json_output`
* Fix `MainExit::report`: Ignore io error
* Fix `BinstallError::report`: Ignore IO error
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Fix integration test: Make sure `cargo build cargo-binstall` would not fail due to binary already present
* Disable feature `zlib-ng` for release build
* Disable build-std for release build on aarch64-unknown-linux-gnu
* Fix clippy warning in unit test CI
* Fix compile-settings.jq
* Fix caching: Rm `restores-keys`
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
which allows pkg-fmt, bin-dir and pkg-url to be specified while providing multiple crates to install.
Fixed#444
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Add new feature `log_release_max_level_debug`
* Fix calculation of `log_level` in `logging`
* Enable feature log_release_max_level_debug on CI release build
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Add new field `Args::roots`
* Use `env::var_os` to fetch `CARGO_INSTALL_ROOTS`
Previously, it uses `env::var`, which might reject valid path just
because it is not utf-8 string.
* Update manifest if `CARGO_INSTALL_ROOT` is specified
* Add new fn `install_path::get_cargo_roots_path`
* Fix updating manifest: Use `cargo_roots` instead of default path
* Rm `helpers::statics::cargo_home`
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Print `rerun-if-changed=build.rs` in `build.rs`
* Optimize compile-time: Extract `bin/src/lib.rs`
so that `cargo-binstall (lib)` can be compiled in parallel to other
deps.
* Refactor: Extract new mod `bin/src/bin_utils.rs`
* Extract new fn `MainExit::new`
* Refactor: Extract new fn `run_tokio_main`
* Handle `Runtime::new` err gracefully in `run_tokio_main`
instead of `panic!`ing, return the error as `BinstallError`
* Avoid mixing `eprintln` and `error` in `MainExit::report`
* Set profile for `build-override`
to speedup building of `build.rs` and proc macros.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
* Derive `Eq` for `PkgFmt`, `PkgFmtDecomposed` & `TarBasedFmt`
* Derive `Eq` for `PkgMeta`, `PkgOverride` & `BinMeta`
* Rm unnecessary reborrow in `install_from_source`
* Rm unnecessary `into()` in `args::parse`
* Fix `clippy` warning in `CratesToml::append_to_path`
* Fix clippy warning in `cratesio_url`
* Fix clippy warning in `detect_targets_linux`
* Add `clippy` and `rustfmt` to workflow `unit-test`
Because `clippy` would not check inactive code disabled by `cfg`, so we
have to also run `clippy` on multiple targets.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
