Skip to content

Commit 8baf2e9

Browse files
committed
Add example of a dependency linked twice.
1 parent 7216b13 commit 8baf2e9

34 files changed

+1426
-32
lines changed

.dockerignore

+2
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,6 @@
11
.git
2+
.github
23
LICENSE
34
*.md
45
docker-*.sh
6+
rustc-build

.github/workflows/docker-build.yml

+22-1
Original file line numberDiff line numberDiff line change
@@ -29,10 +29,31 @@ jobs:
2929
--tmpfs /home/dev/build:exec,size=512m \
3030
--tmpfs /home/dev/.gradle:exec,size=512m \
3131
--tmpfs /home/dev/.android:size=1m \
32-
--tmpfs /home/dev/.cargo/registry:size=16m \
32+
--tmpfs /home/dev/.cargo/registry:size=32m \
3333
--tmpfs /home/dev/.rustup/toolchains:exec,size=1m \
3434
--entrypoint /home/dev/build-application.sh \
3535
android-rust-simd
36+
- name: Build Android application (re-linked library)
37+
run: |
38+
docker run \
39+
--rm \
40+
--cap-drop=all \
41+
--security-opt no-new-privileges \
42+
--read-only \
43+
-u dev \
44+
--memory=3g \
45+
--memory-swap=3g \
46+
--memory-swappiness=0 \
47+
--env USER=dev \
48+
--tmpfs /tmp:size=16m \
49+
--tmpfs /home/dev/build:exec,size=512m \
50+
--tmpfs /home/dev/.gradle:exec,size=512m \
51+
--tmpfs /home/dev/.android:size=1m \
52+
--tmpfs /home/dev/.cargo/registry:size=32m \
53+
--tmpfs /home/dev/.cargo/git:size=1m \
54+
--tmpfs /home/dev/.rustup/toolchains:exec,size=1m \
55+
--entrypoint /home/dev/build-application-relinked.sh \
56+
android-rust-simd
3657
- name: Build Android application (stage 1 Rust)
3758
run: |
3859
mkdir -p rustc-build;

.github/workflows/format.yml

+2
Original file line numberDiff line numberDiff line change
@@ -12,5 +12,7 @@ jobs:
1212
components: rustfmt
1313
- name: Check Rust formatting (android-simd)
1414
run: cargo fmt --verbose --manifest-path src/android-simd/Cargo.toml -- --check
15+
- name: Check Rust formatting (relinked)
16+
run: cargo fmt --verbose --manifest-path src/relinked/Cargo.toml -- --check
1517
- name: Check Rust formatting (flamedisk)
1618
run: cargo fmt --verbose --manifest-path tools/flamedisk/Cargo.toml -- --check

Dockerfile

+20-3
Original file line numberDiff line numberDiff line change
@@ -132,6 +132,7 @@ USER root
132132

133133
COPY \
134134
scripts/build-application.sh \
135+
scripts/build-application-relinked.sh \
135136
scripts/build-application-stage1.sh \
136137
scripts/setup.sh \
137138
scripts/clone-rustlang-head.sh \
@@ -153,9 +154,16 @@ COPY \
153154
scripts/launch-app-debug.sh \
154155
scripts/launch-app-release.sh \
155156
scripts/flamedisk.sh \
157+
scripts/android-runner.sh \
158+
scripts/build-relinked.sh \
159+
scripts/script-relinked.sh \
160+
scripts/bench.sh \
161+
scripts/bench-haraka.sh \
162+
scripts/bench-horcrux.sh \
156163
/home/dev/
157164
RUN chmod 555 \
158165
/home/dev/build-application.sh \
166+
/home/dev/build-application-relinked.sh \
159167
/home/dev/build-application-stage1.sh \
160168
/home/dev/setup.sh \
161169
/home/dev/clone-rustlang-head.sh \
@@ -176,11 +184,20 @@ RUN chmod 555 \
176184
/home/dev/emulator.sh \
177185
/home/dev/launch-app-debug.sh \
178186
/home/dev/launch-app-release.sh \
179-
/home/dev/flamedisk.sh
187+
/home/dev/flamedisk.sh \
188+
/home/dev/android-runner.sh \
189+
/home/dev/build-relinked.sh \
190+
/home/dev/script-relinked.sh \
191+
/home/dev/bench.sh \
192+
/home/dev/bench-haraka.sh \
193+
/home/dev/bench-horcrux.sh
180194

181195
COPY --chown=1000:1000 cargo-config.toml /home/dev/.cargo/config
182-
COPY --chown=1000:1000 rustbuild-config.toml /home/dev/
183-
COPY --chown=1000:1000 stdarch.patch /home/dev/
196+
COPY --chown=1000:1000 \
197+
rustbuild-config.toml \
198+
stdarch.patch \
199+
horcrux.dynamic_detect.patch \
200+
/home/dev/
184201

185202
COPY --chown=1000:1000 src /home/dev/src
186203
COPY --chown=1000:1000 tools /home/dev/tools

README.md

+17-2
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,8 @@ Additionally, this Rust library demonstrates detecting support for SIMD instruct
66
You can find more information in the following blog posts:
77

88
- [Compiling Rust libraries for Android apps: a deep dive](https://gendignoux.com/blog/2022/10/24/rust-library-android.html),
9-
- [Detecting SIMD support on ARM with Android (and patching the Rust compiler for it)](https://gendignoux.com/blog/2022/11/09/rust-simd-detect-arm-android.md).
9+
- [Detecting SIMD support on ARM with Android (and patching the Rust compiler for it)](https://gendignoux.com/blog/2022/11/09/rust-simd-detect-arm-android.md),
10+
- [Testing SIMD instructions on ARM with Rust on Android](https://gendignoux.com/blog/2023/01/05/rust-arm-simd-android.html).
1011

1112
## Usage
1213

@@ -16,6 +17,8 @@ To build a Docker container containing all the build tools and the demo applicat
1617
$ sudo ./docker-build.sh
1718
```
1819

20+
### Building and running the demo Android application
21+
1922
You can then launch this Docker container to build the application.
2023
This comes in various configurations:
2124

@@ -35,6 +38,7 @@ You can build various flavors of the Rust library.
3538
- `script-rust-nightly-nostrip.sh`: same, but without stripping debug symbols.
3639
- `script-rust-stage1.sh` and `script-rust-stage1-nostrip.sh`: using a locally built `stage1` Rust compiler (see below).
3740
- `script-rust-default.sh` and `script-rust-default-nostrip.sh`: using the default Rust toolchain (stable).
41+
- `script-relinked.sh`: more advanced library, which bundles another library linked twice (see the corresponding [blog post](https://gendignoux.com/blog/2023/01/05/rust-arm-simd-android.html#mixing-dynamic-and-static-detection-re-linking-a-dependency)).
3842

3943
Then, multiple ways are provided to build the Java part of the Android app.
4044

@@ -44,7 +48,7 @@ Then, multiple ways are provided to build the Java part of the Android app.
4448

4549
You can then spawn an Android emulator with `emulator.sh`, and use the `launch-app-debug.sh` or `launch-app-release.sh` scripts to install+launch the application via ADB to either the emulator or a real device connected via USB.
4650

47-
## Building and using a patched Rust compiler
51+
### Building and using a patched Rust compiler
4852

4953
This repository also shows how to patch and build the Rust compiler.
5054
You can launch the Docker container in various scenarios:
@@ -65,3 +69,14 @@ You'll also find the following tool to generate a flame graph of the disk space
6569

6670
- `tools/flamedisk`: small Rust tool to generate an input suitable for `flamegraph.pl`,
6771
- `scripts/flamedisk.sh`: driver script to generate the flame graph.
72+
73+
### Running unit tests and benchmarks on an Android device
74+
75+
This repository shows how to run Rust unit tests and benchmarks directly on an attached Android device (physical device via USB or emulator), without using a full Android application.
76+
The `android-runner.sh` script tells Cargo how to do that.
77+
78+
Within the Docker container, you can run the following benchmarks:
79+
80+
- `bench.sh`: demo application, located in `src/android-simd`,
81+
- `bench-haraka.sh`: my implementation of the Haraka hash function (https://github.com/gendx/haraka-rs),
82+
- `bench-horcrux.sh`: my implementation of Shamir's Secret Sharing (https://github.com/gendx/horcrux).

cargo-config.toml

+4
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,15 @@
11
[target.aarch64-linux-android]
22
linker = "/home/dev/opt/android-sdk/ndk/25.1.8937393/toolchains/llvm/prebuilt/linux-x86_64/bin/aarch64-linux-android30-clang"
3+
runner = "/home/dev/android-runner.sh"
34

45
[target.armv7-linux-androideabi]
56
linker = "/home/dev/opt/android-sdk/ndk/25.1.8937393/toolchains/llvm/prebuilt/linux-x86_64/bin/armv7a-linux-androideabi30-clang"
7+
runner = "/home/dev/android-runner.sh"
68

79
[target.i686-linux-android]
810
linker = "/home/dev/opt/android-sdk/ndk/25.1.8937393/toolchains/llvm/prebuilt/linux-x86_64/bin/i686-linux-android30-clang"
11+
runner = "/home/dev/android-runner.sh"
912

1013
[target.x86_64-linux-android]
1114
linker = "/home/dev/opt/android-sdk/ndk/25.1.8937393/toolchains/llvm/prebuilt/linux-x86_64/bin/x86_64-linux-android30-clang"
15+
runner = "/home/dev/android-runner.sh"

docker-run-all-rustc.sh

+1
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,7 @@ docker run \
2222
--tmpfs /home/dev/.gradle:exec,size=512m \
2323
--tmpfs /home/dev/.android:size=3g \
2424
--tmpfs /home/dev/.cargo/registry:size=512m \
25+
--tmpfs /home/dev/.cargo/git:size=1m \
2526
--tmpfs /home/dev/.rustup/toolchains:exec,size=1m \
2627
--volume /tmp/.X11-unix:/tmp/.X11-unix \
2728
--volume ${PWD}/rustc-build:/home/dev/rustc-build \

docker-run-all.sh

+2-1
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,8 @@ docker run \
1515
--tmpfs /home/dev/build:exec,size=512m \
1616
--tmpfs /home/dev/.gradle:exec,size=512m \
1717
--tmpfs /home/dev/.android:size=3g \
18-
--tmpfs /home/dev/.cargo/registry:size=16m \
18+
--tmpfs /home/dev/.cargo/registry:size=32m \
19+
--tmpfs /home/dev/.cargo/git:size=1m \
1920
--tmpfs /home/dev/.rustup/toolchains:exec,size=1m \
2021
--volume /tmp/.X11-unix:/tmp/.X11-unix \
2122
--device /dev/bus/usb \

docker-run-basic.sh

+2-1
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,7 @@ docker run \
1414
--tmpfs /home/dev/build:exec,size=512m \
1515
--tmpfs /home/dev/.gradle:exec,size=512m \
1616
--tmpfs /home/dev/.android:size=1m \
17-
--tmpfs /home/dev/.cargo/registry:size=16m \
17+
--tmpfs /home/dev/.cargo/registry:size=32m \
18+
--tmpfs /home/dev/.cargo/git:size=1m \
1819
--tmpfs /home/dev/.rustup/toolchains:exec,size=1m \
1920
android-rust-simd

docker-run-emulator.sh

+2-1
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,8 @@ docker run \
1515
--tmpfs /home/dev/build:exec,size=512m \
1616
--tmpfs /home/dev/.gradle:exec,size=512m \
1717
--tmpfs /home/dev/.android:size=3g \
18-
--tmpfs /home/dev/.cargo/registry:size=16m \
18+
--tmpfs /home/dev/.cargo/registry:size=32m \
19+
--tmpfs /home/dev/.cargo/git:size=1m \
1920
--tmpfs /home/dev/.rustup/toolchains:exec,size=1m \
2021
--volume /tmp/.X11-unix:/tmp/.X11-unix \
2122
--device /dev/dri \

docker-run-rustc-1cpu.sh

+1
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,7 @@ docker run \
1818
--env USER=dev \
1919
--tmpfs /tmp:size=256m \
2020
--tmpfs /home/dev/.cargo/registry:size=512m \
21+
--tmpfs /home/dev/.cargo/git:size=1m \
2122
--tmpfs /home/dev/.rustup/toolchains:exec,size=1m \
2223
--volume $PWD/rustc-build:/home/dev/rustc-build \
2324
android-rust-simd

docker-run-rustc-2cpus.sh

+1
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,7 @@ docker run \
1818
--env USER=dev \
1919
--tmpfs /tmp:size=256m \
2020
--tmpfs /home/dev/.cargo/registry:size=512m \
21+
--tmpfs /home/dev/.cargo/git:size=1m \
2122
--tmpfs /home/dev/.rustup/toolchains:exec,size=1m \
2223
--volume ${PWD}/rustc-build:/home/dev/rustc-build \
2324
android-rust-simd

docker-run-rustc-4cpus.sh

+1
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,7 @@ docker run \
1818
--env USER=dev \
1919
--tmpfs /tmp:size=256m \
2020
--tmpfs /home/dev/.cargo/registry:size=512m \
21+
--tmpfs /home/dev/.cargo/git:size=1m \
2122
--tmpfs /home/dev/.rustup/toolchains:exec,size=1m \
2223
--volume ${PWD}/rustc-build:/home/dev/rustc-build \
2324
android-rust-simd

docker-run-rustc-8cpus.sh

+1
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,7 @@ docker run \
1818
--env USER=dev \
1919
--tmpfs /tmp:size=256m \
2020
--tmpfs /home/dev/.cargo/registry:size=512m \
21+
--tmpfs /home/dev/.cargo/git:size=1m \
2122
--tmpfs /home/dev/.rustup/toolchains:exec,size=1m \
2223
--volume ${PWD}/rustc-build:/home/dev/rustc-build \
2324
android-rust-simd

docker-run-usb.sh

+2-1
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,8 @@ docker run \
1414
--tmpfs /home/dev/build:exec,size=512m \
1515
--tmpfs /home/dev/.gradle:exec,size=512m \
1616
--tmpfs /home/dev/.android:size=1m \
17-
--tmpfs /home/dev/.cargo/registry:size=16m \
17+
--tmpfs /home/dev/.cargo/registry:size=32m \
18+
--tmpfs /home/dev/.cargo/git:size=1m \
1819
--tmpfs /home/dev/.rustup/toolchains:exec,size=1m \
1920
--device /dev/bus/usb \
2021
android-rust-simd

horcrux.dynamic_detect.patch

+91
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,91 @@
1+
diff --git a/horcrux/src/gf2n.rs b/horcrux/src/gf2n.rs
2+
index 5bc0221..501d455 100644
3+
--- a/horcrux/src/gf2n.rs
4+
+++ b/horcrux/src/gf2n.rs
5+
@@ -294,13 +294,9 @@ fn mul_clmul_u64<const NWORDS: usize, const A: usize, const B: usize, const C: u
6+
GF2n::<u64, NWORDS, A, B, C>::propagate_carries(words, carry)
7+
}
8+
9+
-#[cfg(all(
10+
- feature = "clmul",
11+
- target_arch = "aarch64",
12+
- target_feature = "neon",
13+
- target_feature = "aes"
14+
-))]
15+
-fn mul_clmul_u64<const NWORDS: usize, const A: usize, const B: usize, const C: usize>(
16+
+#[cfg(all(feature = "clmul", target_arch = "aarch64"))]
17+
+#[target_feature(enable = "neon", enable = "aes")]
18+
+unsafe fn mul_clmul_u64<const NWORDS: usize, const A: usize, const B: usize, const C: usize>(
19+
x: &GF2n<u64, NWORDS, A, B, C>,
20+
y: &GF2n<u64, NWORDS, A, B, C>,
21+
) -> GF2n<u64, NWORDS, A, B, C> {
22+
@@ -316,7 +312,7 @@ fn mul_clmul_u64<const NWORDS: usize, const A: usize, const B: usize, const C: u
23+
for j in 0..NWORDS {
24+
let yj = y.words[j];
25+
// Safety: target_feature's "neon" and "aes" are available in this function.
26+
- let clmul: u128 = unsafe { vmull_p64(xi, yj) };
27+
+ let clmul: u128 = vmull_p64(xi, yj);
28+
let low: u64 = clmul as u64;
29+
let high: u64 = (clmul >> 64) as u64;
30+
31+
@@ -540,12 +536,7 @@ impl<W: Word, const NWORDS: usize, const A: usize, const B: usize, const C: usiz
32+
target_feature = "sse2",
33+
target_feature = "pclmulqdq"
34+
),
35+
- all(
36+
- feature = "clmul",
37+
- target_arch = "aarch64",
38+
- target_feature = "neon",
39+
- target_feature = "aes"
40+
- )
41+
+ all(feature = "clmul", target_arch = "aarch64")
42+
))]
43+
fn propagate_carries(mut words: [W; NWORDS], carry: [W; NWORDS]) -> Self {
44+
if NWORDS == 1 {
45+
@@ -672,19 +663,11 @@ impl<W: Word, const NWORDS: usize, const A: usize, const B: usize, const C: usiz
46+
type Output = Self;
47+
48+
fn mul(self, other: &Self) -> Self {
49+
- #[cfg(any(
50+
- all(
51+
- feature = "clmul",
52+
- target_arch = "x86_64",
53+
- target_feature = "sse2",
54+
- target_feature = "pclmulqdq"
55+
- ),
56+
- all(
57+
- feature = "clmul",
58+
- target_arch = "aarch64",
59+
- target_feature = "neon",
60+
- target_feature = "aes"
61+
- )
62+
+ #[cfg(all(
63+
+ feature = "clmul",
64+
+ target_arch = "x86_64",
65+
+ target_feature = "sse2",
66+
+ target_feature = "pclmulqdq"
67+
))]
68+
if W::NBITS == 64 {
69+
// Safety: W == u64 when NBITS == 64.
70+
@@ -696,6 +679,21 @@ impl<W: Word, const NWORDS: usize, const A: usize, const B: usize, const C: usiz
71+
let result: &Self = unsafe { std::mem::transmute(&tmp) };
72+
return *result;
73+
}
74+
+ #[cfg(all(feature = "clmul", target_arch = "aarch64"))]
75+
+ if W::NBITS == 64
76+
+ && std::arch::is_aarch64_feature_detected!("neon")
77+
+ && std::arch::is_aarch64_feature_detected!("aes")
78+
+ {
79+
+ // Safety: W == u64 when NBITS == 64.
80+
+ let x: &GF2n<u64, NWORDS, A, B, C> = unsafe { std::mem::transmute(&self) };
81+
+ // Safety: W == u64 when NBITS == 64.
82+
+ let y: &GF2n<u64, NWORDS, A, B, C> = unsafe { std::mem::transmute(other) };
83+
+ // Safety: target_feature's "neon" and "aes" are available in this block.
84+
+ let tmp: GF2n<u64, NWORDS, A, B, C> = unsafe { mul_clmul_u64(x, y) };
85+
+ // Safety: W == u64 when NBITS == 64.
86+
+ let result: &Self = unsafe { std::mem::transmute(&tmp) };
87+
+ return *result;
88+
+ }
89+
self.mul_as_add(other)
90+
}
91+
}

scripts/android-runner.sh

+24
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
#!/bin/bash
2+
3+
set -eu
4+
5+
echo "####################"
6+
echo "# android-runner.sh invoked with: $@"
7+
echo "####################"
8+
9+
# The binary to upload and run is the first argument.
10+
BINARY_PATH="$1"
11+
BINARY=`basename ${BINARY_PATH}`
12+
# Remove the first parameter.
13+
shift
14+
15+
# Push the test binary on the device via ADB.
16+
adb push "${BINARY_PATH}" "/data/local/tmp/$BINARY"
17+
adb shell "chmod 755 /data/local/tmp/$BINARY"
18+
19+
# Run the test binary, forwarding the remaining parameters, so that benchmarks,
20+
# test filtering, etc. work.
21+
adb shell "/data/local/tmp/$BINARY $@"
22+
23+
# Cleanup.
24+
adb shell "rm /data/local/tmp/$BINARY"

scripts/bench-haraka.sh

+14
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
#!/bin/bash
2+
3+
set -eux
4+
5+
export CARGO_UNSTABLE_SPARSE_REGISTRY=true
6+
7+
cd build/
8+
git clone https://github.com/gendx/haraka-rs
9+
cd haraka-rs/
10+
11+
RUSTFLAGS='-C target-feature=+aes' cargo +nightly bench --target aarch64-linux-android
12+
RUSTFLAGS='-C target-feature=+aes' cargo +nightly bench --target armv7-linux-androideabi
13+
cargo +nightly bench --target aarch64-linux-android
14+
cargo +nightly bench --target armv7-linux-androideabi

0 commit comments

Comments
 (0)