Jiahao XU
e18ac6e117
Run Manifest parsing in block_in_place mode
...
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
2022-06-23 19:55:09 +10:00
Jiahao XU
784a24577b
Refactor: Rm extract_impl
...
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
2022-06-18 18:37:50 +10:00
Jiahao XU
db22d7d041
Fix codestyle in async_extracter
...
Use consistent codestyle for specifing trait bounds.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
2022-06-18 18:35:01 +10:00
Jiahao XU
a5879e3d65
Rm unnecessary to_owned call in extract_*
...
It was called before because `spawn_blocking` requires that, but we now
switches to `block_in_place` which no longer needs this.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
2022-06-18 18:24:01 +10:00
Jiahao XU
9de8a4841f
Update doc of mod async_extracter
...
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
2022-06-18 18:12:48 +10:00
Jiahao XU
b152358175
Rm unused trait bound in extract_impl
...
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
2022-06-18 18:12:13 +10:00
Jiahao XU
621a641529
Rm unused items in mod helpers
...
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
2022-06-18 18:10:22 +10:00
Jiahao XU
c15d99c6f0
Run downloader & extracter on the same thread
...
This have the following advantage:
- Remove the mpsc channel, which:
- Remove synchronization required for mpsc.
- Remove the internal buffering of the mpsc channel, which avoid potentially OOM situation.
- Improve data locality since it no longer needs to be sent over thread.
- It uses `block_in_place` to avoid creating an additional blocking
thread.
The disadvantages would be that the downloader can no longer be run in parallel to the extracter.
If the bottleneck is the decompressor, then the downloader should also pause and wait
for the decompressor to consume the data.
But if the bottleneck is the network, then that might be an issue.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
2022-06-18 18:07:46 +10:00
Jiahao XU
9eb1128f9f
Rm unused DummyVisitor in extract_tar_based_stream
...
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
2022-06-13 19:32:55 +10:00
Jiahao XU
2091345ce0
Refactor: Mv TarEntriesVisitor to mod async_extracter
...
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
2022-06-13 01:14:22 +10:00
Jiahao XU
467f7f6834
Refactor: Call create_tar_decoder directly
...
in `extract_tar_based_stream*`.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
2022-06-13 01:12:21 +10:00
Jiahao XU
8ef1e56fcc
Take Receiver by value in ReadableRx::new
...
It would remove the lifetime and make reasoning the code much easier.
It would also unblock the next commit I am going to make.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
2022-06-13 01:05:07 +10:00
Jiahao XU
baf9784b82
Update doc of mod async_extracter
...
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
2022-06-12 19:52:03 +10:00
Jiahao XU
3a1038c80b
Optimize binary size/compilation time
...
by reducing generics monomorphization using `Box<dyn Trait>`.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
2022-06-12 19:44:59 +10:00
Jiahao XU
e376b71cf4
Simplify extract_tar_based_stream_and_visit
...
Rm unused param `path` and the unnecessary
`fs::create_dir_all` since the tar will not be extracted to disk.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
2022-06-12 17:05:35 +10:00
Jiahao XU
17fcac7e63
Refactor: Simplify untar with new enum Op
...
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
2022-06-12 17:03:39 +10:00
Jiahao XU
f82890cba3
Rm download_and_extract_with_filter
...
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
2022-06-12 16:42:32 +10:00
Jiahao XU
f8c8c66f57
Impl new fn helpers::download_tar_based_and_visit
...
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
2022-06-11 20:38:11 +10:00
Jiahao XU
90a96cabc9
Rewrite untar to take a visitor & simplify
...
signature of `download_and_extract_with_filter`
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
2022-06-11 20:31:46 +10:00
Jiahao XU
5a43ee2681
Simplify extract_compressed_from_readable impl
...
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
2022-06-11 20:16:41 +10:00
Jiahao XU
57b40d809e
Cleanup mod async_extracter
...
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
2022-06-11 20:15:17 +10:00
Jiahao XU
7b52eaad5b
Rewrite AsyncExtracter: Extract fmt logic as callback fn
...
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
2022-06-11 20:15:17 +10:00
Jiahao XU
cbd57a1bce
Refactor async_extracter: Create multi extracters
...
dedicated to different tasks
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
2022-06-11 20:15:17 +10:00
Jiahao XU
bd68613448
Refactor and add new enum PkgFmtDecomposed
...
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
2022-06-11 20:15:17 +10:00
Félix Saparelli
f53680c497
Merge pull request #174 from NobodyXu/fix/find_crate_cratesio
2022-06-11 14:54:50 +12:00
Jiahao XU
1d139324c7
Rewrite untar: Takes a filter fn instead of array
...
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
2022-06-10 16:35:51 +10:00
Jiahao XU
32ad530329
Rm use of AutoAbortJoinHandle in AsyncExtracter
...
Since there is no way to abort a blocking thread, using
`AutoAbortJoinHandle` does not add any value.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
2022-06-10 13:54:43 +10:00
Jiahao XU
1c3a672108
Rm scopeguard creatd for mpsc::Receiver
...
Since `Receiver` always closes on drop, there is no need to call
`Receiver::close`, which is there so that you can close the mpsc channel
without dropping `Receiver`.
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
2022-06-10 13:52:11 +10:00
Jiahao XU
cadf045d0a
Refactor: Abstract away AsyncExtracter by new fn
...
`extract_archive_stream`
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
2022-06-09 22:29:33 +10:00
Jiahao XU
00242a40c6
Update doc for AsyncExtracter::new
...
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
2022-06-09 22:22:39 +10:00
Jiahao XU
8a812c8d22
Rename AsyncExtracter::write to feed
...
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
2022-06-09 22:09:46 +10:00
Jiahao XU
432376224f
Update doc of AsyncExtracter
...
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
2022-06-09 22:09:02 +10:00
Jiahao XU
945687c281
Rename AsyncFileWriter to AsyncExtracter
...
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
2022-06-09 22:00:00 +10:00
