Commit graph

22 commits

Author SHA1 Message Date
Jiahao XU
8e08c65946
Fix fallback to cargo-install: Use the default target unless user override (#1430)
* Fix fallback to `cargo-install`: Use the default target

which is guaranteed to work, instead of providing a custom `--target`
from `detect-targets`.

The one from `detect-targets` might report musl even if the `rustc`
reports gnu, since OSes like NixOS could use a custom glibc location
that prevent glibc binaries built for other OSes be run for them.

However, if we are compiling the crate and `rustc` defaults to glibc,
then it should be guaranteed to work.

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Impl fn `DesiredTargets::get_initialized`

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Fix fallback to `cargo-install`: Use user-provided override targets

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

---------

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
2023-09-30 00:22:21 +00:00
Jiahao XU
b56a8c3579
Fix detect-targets glibc detection (#1421)
* Fix `detect-targets` glibc detection

Fixed #1420

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Fix `detect-targets` glibc detection for Fedora

Fedora 37-39 has glibc installed in `/lib64` and `/usr/lib64` instead of
`/lib` or `/usr/lib`.

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Improve caching in `detect-targets-alpine-test`

Make sure it reuses cached artifacts instead of recompiling everything.

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Mv `test-detect-targets-musl.sh` into `.github/scripts/`

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Refactor `ci.yml`: Extract new job `detect-targets-build`

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* FIx `detect-targets-ubuntu-test`

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Avoid building the entire workspacein `detect-targets-build`

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Fix `detect-targets-*-test`: `chmod +x detect-targets`

Also fixed testing on Alpine, to make sure it exits with status 1 on
assertion failure.

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Fix typo

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Cache all crates in `detect-targets-build`

Since `detect-targets` rarely changes and is quite small, it is also
reasonable to cache it.

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Add job `detect-targets-more-glibc-test`

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Add `detect-targets-nix-test`

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Fix `detect-targets-nix-test` executing `detect-targets`

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Fix workflow: Add all `detect-targets-*` jobs to be dep of `tests-pass`

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

---------

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
2023-09-29 23:46:03 +00:00
Tamir Duberstein
2db8e254bc
Investigate incorrect target detection (#1376)
* detect-targets: add debug tracing

See #1375.

* Add new feature `tracing` & `cli-tracing` to `detect-targets

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Fix clippy lints

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

---------

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
Co-authored-by: Jiahao XU <Jiahao_XU@outlook.com>
2023-09-23 10:45:44 +00:00
Jiahao XU
efbd20857b
Fix detect-targets on ubuntu 20.04, glibc 2.31 (#1379)
* Fix `detect-targets` on ubuntu 20.04, glibc 2.31

Fixed #1375 and fixed #1378

glibc 2.31 does not support `--version`, so we need to detect and
fallback to passing an executable linked with that glibc to check if it
is indeed glibc.

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Fix compilation faillure on ubuntu-20.04

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

---------

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
2023-09-22 23:55:12 +00:00
Jiahao XU
76a692224d
Fix detect-targets on Linux and add CI testing (#1344)
* Testing: Add `detect-targets/src/main.rs`

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Fix `detect-targets` linux: `guess_host_triple` could return wrong libc info

so it has to check it manually instead of simply providing alternatives
like other OSes.

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Fix `get_ld_flavor` for Alpine's gcompat glibc

Its output is different from regular glibc, so we need to hardcode that
particular cases.

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Fix detection of alpine specific musl target

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Add ci testing for Alpine

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Add CI test for detect-targets on ubuntu

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Refactor `get_ld_flavor`

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Fix shellcheck

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Add more CI test for ubuntu

and fixed typo in it

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Rm distro specific target as it breaks `cargo-install` fallback

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Make sure all binaries are built in CI

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Add `package.metadata.binstall` for `detect-targets`

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Fix justfile

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Fix `detect-targets-{alpine, ubuntu}-test`

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Fix `detect-targets-ubuntu-test`

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Fix `debug-targets-ubuntu-test`

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* `set -exuo pipefail` in `detect-targets-ubuntu-test`

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Simplify `detect-targets-*-test`: Use `Swatinem/rust-cache@v2` directly

instead of using `just-setup`

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Rm dup steps in `detect-targets-ubuntu-test`

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Add `ls` to detect-targets-alpine-test for debugging

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* FIx `detect-targets-alpine-test`

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Fix `get_ld_flavor`

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Fix `linux::detect_targets` on ubuntu & glibc system

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* FIx `linux::detect_targets` glibc checking

Check dynlib suffix

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

---------

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
2023-09-06 14:17:43 +00:00
Jiahao XU
0fa315758b
feat detect-targets: Improve support of non-std glibc/musl (#1343)
* feat `detect-targets`: Improve support of non-std glibc/musl

Fixed #1329

 - Refactor: Create `linux::detect_alternative_targets` to reuse code
   from other targets
 - Run `/lib/ld-linux-{cpu_arch}.so.1 --version` for checking glibc
   support instead of running `ldd --version` since it could be non-std
   glibc installation and does not provide
   `/lib/ld-linux-{cpu_arch}.so.1`
 - Check for non-std glibc and add fallback target
   `{cpu_arch}-{distro_name}-linux-gnu{abi}`
 - Add `{cpu_arch}-{distro_name}-linux-musl{abi}` fallback for musl
   libc, specially for Alpine since it has a
   `/lib/ld-musl-{cpu_arch}.so.1`
 - For unknown libc flavor, check for the target provided before
   fallback to musl

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* feat `detect-targets`: Support glibc on musl target

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* feat `detect-targets`: Unify `Libc::{Gnu, Musl}` checks

since we can't really tell if we are on gnu or musl

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

---------

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
2023-09-02 23:19:34 +00:00
Jiahao XU
a84844938d
Fix aarch64-apple-darwin fallback to x86_64h (#1278)
Run `arch -arch x86_64h /usr/bin/true` to decides whether fallback
is feasible.

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
2023-08-11 01:02:03 +00:00
Jiahao XU
d5549ce99e
detect-targets: Add fallback to x86_64-apple-darwin (#1233)
Fallback to `x86_64h-apple-darwin` if supported.

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
2023-07-28 07:59:23 +00:00
Jiahao XU
1293314100
feature: Add fallback x86_64h-apple-darwin and fallback for it (#1228)
Now `detect_targets` would fallback to `x86_64h-apple-darwin` on
`aarch64-apple-darwin`.

It also provides fallback to `x86_64h-apple-darwin` to
`x86_64-apple-darwin` and universal targets.

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
2023-07-23 04:28:59 +00:00
Jiahao XU
e87e3534a8
detect-targets: Add fallback to windows (#650)
Fixed #642

* Add new dep windows-sys v0.42.0 for win only
* Add new dep windows-dll v0.4.1 for win only
* Add x86_64 fallback targets for windows
* Add x86 fallback targets for windows
* Add arm32 fallback targets for windows
* Add arm64 fallback targets for windows
* Add gnu/gnu-llvm fallback targets for windows

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
2023-06-04 03:58:05 +00:00
Jiahao XU
06b89b2d18
Add alias universal2-apple-darwin to universal-apple-darwin (#919)
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
2023-03-17 22:30:55 +11:00
Jiahao XU
a22cb3a332
Fix x86_64 fallback for aarch64 MacOS (#875) 2023-03-10 22:09:16 +11:00
Jiahao XU
d72abde8e9
Fixed #704: Fix get_target_from_rustc for alpine (#705)
On Alpine, rustc returns host triple `x86_64-alpine-linux-musl`, so we
have to adopt code from
[`convert_custom_linux_target`](5519aeb5c2/build-common.rs (L1-L13)) to fix this.

* Optimize `get_target_from_rustc`: Avoid frequent String allocation

caused by `Cursor::new(stdout).lines()` which calls `BufRead::lines` and
returns `Lines` iterator, which returns `Item = io::Result<String>`.

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
2023-01-16 10:57:53 +11:00
Félix Saparelli
aea9df602c
Migrate CI and builds to Just, add "full" builds (#660) 2023-01-08 16:27:36 +11:00
Jiahao XU
42672ccf40
Fix detect_targets_linux: Detect arch/libc at runtime using guess_host_triple (#649)
- Enable `guess_host_triple` on all targets
 - Fix `detect_targets_linux`: Detect runtime arch/libc using
   `guess_host_triple` instead of using the one specified in `TARGET`.
 - Add support for android in `detect_targets`

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
2023-01-04 14:32:13 +13:00
Jiahao XU
59d79d1573
Refactor detect-targets (#636)
* Enable dep guess_host_triple for any non-linux OS
* Refactor: Rm `{windows, macos}::detect_target_*`
* Refactor `detect_targets`

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
2022-12-29 01:24:28 +00:00
Jiahao XU
31c9d4de1d
Verify target triple retrieved from rustc is valid (#637)
Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
2022-12-29 13:42:54 +13:00
Jiahao XU
62e350eba9
detect-targets: Add support of MacOS universal binary (#552)
* Add support for MacOS universal bin
* Refactor: Extract new fn `macos::detect_alternative_targets`

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
2022-11-21 22:16:28 +13:00
Jiahao XU
ec2bdb551e
Use CARGO env variable if present (#453)
* Run `$CARGO -vV` in `detect-targets` if env `CARGO` present
* Improve crate doc for `detect-targets`
* Use env var `CARGO` in `install_from_source` if present
* Test use of `CARGO` env in `tests.sh`

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
2022-10-02 04:52:25 +13:00
Jiahao XU
89869db239
Fix doc of desired_targets (#337) 2022-09-04 15:11:28 +00:00
Jiahao XU
b330a18d40
Change typeof Args::targets to Option<Vec<String>> (#327)
* Change typeof `Args::targets` to `Option<Vec<String>>`

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
2022-08-31 16:48:15 +12:00
Jiahao XU
62f9450d2d
Refactor: Extract new crate detect-targets and improve code quality (#307)
* Refactor: Extract new crate `detect-targets`

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Refactor: Extract new mod `detect` for `detect-targets`

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Refactor: Extract `desired_targets` in crate `detect-targets`

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Refactor: Extract `detect::linux` in crate `detect-targets`

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Refactor: Extract `detect::macos` in crate `detect-targets`

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Refactor: Extract `detect::windows` in crate `detect-targets`

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Add new dep cfg-if v1.0.0 for `detect-targets`

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Refactor: Simplify mod declaration in `detect` using `cfg_if!`

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Refactor: Simplify `detect_targets` using `cfg_if!`

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Add crate doc for `detect-targets`

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Enable feature "macros" of tokio in `detect-targets`

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Enable feature "io-util" of dep tokio

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Rm unused feature "io-util" & "macros" of dep tokio

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Set stdin & stderr to null in `get_target_from_rustc`

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Improve doc of `get_desired_targets`

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Improve `detect_targets_linux`: Run `ldd` with stdin set to null

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Fix potential panic in `windows::detect_alternative_targets`

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* FIx fmt of `detect_targets_linux`

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Do not re-export dep `detect-targets` in `crates/lib`

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Fix typo in crate doc for `detect-targets`

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Enable feature "macros" of tokio in dev mode

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Add example to crate doc of `detect-targets`

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

* Improve API `get_desired_targets`: Take `Option<&str>`

instead of `&Option<String>`

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>

Signed-off-by: Jiahao XU <Jiahao_XU@outlook.com>
2022-08-21 22:21:33 +10:00