Commit graph

245 commits

Author SHA1 Message Date
Félix Saparelli
9d70e33337
Merge pull request #193 from NobodyXu/fix/macos-target-detection
Fix macos detection: Use built-time `TARGET` as fallback
2022-06-28 03:10:18 +12:00
Félix Saparelli
cc8144e06d
Merge pull request #192 from NobodyXu/optimize/parallelize-target-detection
Optimize/parallelize target detection
2022-06-23 21:41:15 +12:00
Jiahao XU
536f3b2c6f
Fix macos detection: Use super::TARGET as fallback
When `guess_host_triple` failed to detect the target.

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
2022-06-23 19:39:36 +10:00
Félix Saparelli
ee4cbaa3d5
Merge pull request #191 from NobodyXu/feature/target-detection-windows
feature:  Detect `*-pc-windows-gnu` targets in `targets.rs`
2022-06-23 21:34:37 +12:00
Jiahao XU
68d111f946
Use get_desired_targets in entry
instead of `detect_targets`, so that if `opts.targets` is
`None`, the future returned by `detect_targets` can be run
in parallel by using `tokio::spawn` with other async code
in `entry`, such as `fetch_crate_cratesio`.

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
2022-06-23 14:47:59 +10:00
Jiahao XU
be4b3ead97
Add new fn get_desired_targets & newtype DesiredTargets
so that if `opts.targets` is `None`, the future returned by
`detect_targets` can be run in parallel by using `tokio::spawn` with
other async code in `entry`, such as `fetch_crate_cratesio`.

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
2022-06-23 14:46:53 +10:00
Jiahao XU
8bc0f11569
Detect *-pc-windows-gnu targets in targets
And add fallback `*-pc-windows-msvc` to the returned targets.

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
2022-06-23 13:59:53 +10:00
Jiahao XU
691bc18dd0
Set min TLS ver to 1.2 for https only mode
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
2022-06-23 13:03:50 +10:00
Jiahao XU
9b62ebdae1
Fix https only mode: It should not be http2 only
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
2022-06-23 13:01:22 +10:00
Jiahao XU
087d544331
Initialize REQWESTGLOBALCONFIG in main::entry
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
2022-06-23 13:01:22 +10:00
Jiahao XU
a3ab3ec502
Add new static var REQWESTGLOBALCONFIG
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
2022-06-23 13:01:22 +10:00
Jiahao XU
64f468acd6
Add new opt https_only_mode & min_tls_version
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
2022-06-23 13:00:52 +10:00
Jiahao XU
eb7d460a9a
Add new enum TLSVersion which impl ArgEnum
and can be converted to `reqwest::tls::Version`

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
2022-06-23 13:00:52 +10:00
Jiahao XU
8e92db3dc6
Refactor: Extract new_reqwest_client(_builder)
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
2022-06-23 13:00:52 +10:00
Ryo Yamashita
1b9f8c0ffc
Add PkgFmt::Tbz2 2022-06-23 02:37:55 +09:00
Jiahao XU
2f38925ee4
Refactor From<io::Error> for BinstallError
Avoid one `expect`.

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
2022-06-21 14:10:27 +10:00
Jiahao XU
c5a2a89361
Rm the duplicate debug! in main.rs:214
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
2022-06-21 13:52:27 +10:00
Jiahao XU
74a6e137be
Refactor: Mv debug! into create_request
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
2022-06-21 13:47:00 +10:00
Jiahao XU
ad41756daa
Rename fmt.rs to format.rs
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
2022-06-21 13:45:56 +10:00
Jiahao XU
23bad39ba8
Refactor:Mv mod visitor vfs under crates_io
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
2022-06-21 13:44:12 +10:00
Jiahao XU
b6f15f2e5e
Refactor: Extract fetch_crate_cratesio out into mod
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
2022-06-21 13:42:39 +10:00
Jiahao XU
c916814e7e
Refactor: Extract find_version into mod
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
2022-06-21 13:40:36 +10:00
Jiahao XU
5d79af545b
Add doc for Vfs
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
2022-06-21 13:38:24 +10:00
Jiahao XU
784a24577b
Refactor: Rm extract_impl
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
2022-06-18 18:37:50 +10:00
Jiahao XU
db22d7d041
Fix codestyle in async_extracter
Use consistent codestyle for specifing trait bounds.

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
2022-06-18 18:35:01 +10:00
Jiahao XU
53bf76104b
Revert refactor: Put fetch_crate_cratesio back to drivers
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
2022-06-18 18:27:08 +10:00
Jiahao XU
b14b71135e
Revert refactor: Put find_version back to drivers
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
2022-06-18 18:25:50 +10:00
Jiahao XU
a5879e3d65
Rm unnecessary to_owned call in extract_*
It was called before because `spawn_blocking` requires that, but we now
switches to `block_in_place` which no longer needs this.

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
2022-06-18 18:24:01 +10:00
Jiahao XU
9de8a4841f
Update doc of mod async_extracter
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
2022-06-18 18:12:48 +10:00
Jiahao XU
b152358175
Rm unused trait bound in extract_impl
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
2022-06-18 18:12:13 +10:00
Jiahao XU
621a641529
Rm unused items in mod helpers
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
2022-06-18 18:10:22 +10:00
Jiahao XU
c15d99c6f0
Run downloader & extracter on the same thread
This have the following advantage:
 - Remove the mpsc channel, which:
    - Remove synchronization required for mpsc.
    - Remove the internal buffering of the mpsc channel, which avoid potentially OOM situation.
 - Improve data locality since it no longer needs to be sent over thread.
 - It uses `block_in_place` to avoid creating an additional blocking
   thread.

The disadvantages would be that the downloader can no longer be run in parallel to the extracter.

If the bottleneck is the decompressor, then the downloader should also pause and wait
for the decompressor to consume the data.

But if the bottleneck is the network, then that might be an issue.

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
2022-06-18 18:07:46 +10:00
Jiahao XU
aba1ba7b6d
Manually impl From<io::Error> for BinstallError
so that if the `io::Error` wraps a `BinstallError`, we would just unwrap
it and return the inner `BinstallError`.

Otherwise, just wrap the `io::Error` in a `BinstallError`.

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
2022-06-18 18:05:15 +10:00
Jiahao XU
1161a60968
Simplify create_tar_decoder: Ret io::Result
instead of `Result<Archive, BinstallError>`

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
2022-06-18 17:56:17 +10:00
Jiahao XU
0480e99460
Impl newtype StreamReadable
It wraps a `Stream<Item = Result<Bytes, E>>` and implements `Read` and
`BufRead` on it so that it can be used on sync context.

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
2022-06-18 17:29:50 +10:00
Jiahao XU
39ab334da5
Add a simple optimization to normalize_path
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
2022-06-15 17:52:49 +10:00
Jiahao XU
30b9a78520
Optimize normalize_path: Avoid copy if possible
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
2022-06-15 17:45:50 +10:00
Jiahao XU
282805c3ac
Add reference to the src of path_ext
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
2022-06-15 17:23:00 +10:00
Jiahao XU
9eb1128f9f
Rm unused DummyVisitor in extract_tar_based_stream
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
2022-06-13 19:32:55 +10:00
Jiahao XU
2091345ce0
Refactor: Mv TarEntriesVisitor to mod async_extracter
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
2022-06-13 01:14:22 +10:00
Jiahao XU
467f7f6834
Refactor: Call create_tar_decoder directly
in `extract_tar_based_stream*`.

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
2022-06-13 01:12:21 +10:00
Jiahao XU
9584c8d35e
Refactor: Extract create_tar_decoder
from `extract_compressed_from_readable`.

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
2022-06-13 01:06:12 +10:00
Jiahao XU
8ef1e56fcc
Take Receiver by value in ReadableRx::new
It would remove the lifetime and make reasoning the code much easier.

It would also unblock the next commit I am going to make.

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
2022-06-13 01:05:07 +10:00
Jiahao XU
b4e61161f2
Derive strum_macros::Display on TarBasedFmt
sp that it can be printed.

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
2022-06-13 00:53:08 +10:00
Jiahao XU
94c77c32b4
Make debug! message more consistent
in `extract_compressed_from_readable`

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
2022-06-12 22:38:00 +10:00
Jiahao XU
225cf74cd9
Refactor: Ret impl Stream in create_request
Since both `download*` function takes a `impl Stream` and the
`Response::bytes_stream` takes `Response` by value, thus there is no
lifetime issue and we can return `impl Stream` instead of `Response`

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
2022-06-12 20:01:43 +10:00
Jiahao XU
baf9784b82
Update doc of mod async_extracter
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
2022-06-12 19:52:03 +10:00
Jiahao XU
3a1038c80b
Optimize binary size/compilation time
by reducing generics monomorphization using `Box<dyn Trait>`.

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
2022-06-12 19:44:59 +10:00
Jiahao XU
bd39ce754f
Fix Vfs::add_path: Use to_string_lossy
instead of `to_str` to be compatible with the implementation in
`cargo_toml`:

https://docs.rs/cargo_toml/0.11.5/src/cargo_toml/afs.rs.html#24

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
2022-06-12 19:23:03 +10:00
Jiahao XU
b88e384f95
Fix Vfs::add_path: Add insert filename instead of path
into the `HashSet<Box<Path>>`

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
2022-06-12 19:17:06 +10:00