Platform support and `auto` scripts improvements #124

bavshin-f5 · 2025-01-25T00:55:37Z

The whole thing now kind of works on Windows¹. Also on NetBSD, OpenBSD and Illumos.
Getting closer to the "Tested OS and platforms" of nginx.

In the course of debugging, examples/config accumulated too many subtle details so I split it into a reusable examples/auto/rust. The idea is to drop the file into your project, copy config.make and write a minimal config.

TODOs:

Documentation updates
Copyright header in auto/rust to allow redistribution.
Decide if it's worth renaming NGX_OBJS to NGINX_BUILD_DIR

Things to address later:

Detect NGX_ALIGNMENT redefined via --with-cc-opt and add a compile-time assertion for insufficient alignment, Use CFLAGS from the Makefile #118
Figure out how to disentangle configure/make commands from nginx-sys/build/vendored.rs
Windows CI?

with a couple of patches: nginx#457 and sed -i -e 's/^$#define _WIN32_WINNT$.*/\1 0x0600/' src/os/win32/ngx_win32_config.h ↩

xeioex · 2025-01-25T07:10:31Z

Copyright header in auto/rust to allow redistribution.

Sound good.

Decide if it's worth renaming NGX_OBJS to NGINX_BUILD_DIR

I believe we are in the early days of the project. The project is still experimental.

`std::fs::canonicalize` always converts paths to UNC on Windows. However, a plenty of software lacks support for this naming convention, notably Microsoft's C/C++ Compiler. `dunce::canonicalize` is a commonly used solution that picks the most compatible path representation on Windows and calls `fs::canonicalize` on any other platform.

Previously, we assumed that the source directory is a parent of `NGX_OBJS`. This isn't true, `--builddir` can point to an absolute path, and source directory can be read-only. Specifying an explicit source path and resolving includes agains it fixes the build for such configurations.

The `NGX_OBJS` name was taken from the nginx buildsystem, but it is an internal detail there, so the name wasn't really meaningful. Now that we have `NGINX_SOURCE_DIR`, there's a benefit in more consistent naming. Technically, not a breaking change, as we haven't had a release with the old variable name.

A new file, `examples/auto/rust`, can be used in the module projects to simplify the build and hide most of the platform-specific details. `examples/config` and `examples/config.make` are reimplemented using the library.

xeioex · 2025-01-25T06:59:13Z

examples/auto/rust

+if [ "$NGX_DEBUG" = YES ]; then
+    ngx_cargo_default_profile=ngx-debug
+else
+    ngx_cargo_default_profile=ngx-release


When building without --with-debug I get

... objs/ngx_modules.o \ -L/media/psf/workspace/nginx/nginScript/quickjs-ng/build -lpthread -lcrypt objs/ngx_rust_examples/ngx-release/examples/libawssig.a -lm objs/ngx_ru st_examples/ngx-release/examples/libcurl.a objs/ngx_rust_examples/ngx-release/examples/libupstream.a objs/ngx_rust_examples/ngx-release/examples/l ibhttporigdst.a -lpcre2-8 -lssl -lcrypto -lpthread -lz \ -Wl,-E /usr/bin/ld: objs/ngx_rust_examples/ngx-release/examples/libcurl.a(curl-de3e6fe882b906f0.core-f5a882967048065e.core.ecb3f420ec5f4bcf-cgu.0.rcgu.o. rcgu.o): in function `core::num::from_str_radix_panic_rt': /rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14/library/core/src/num/mod.rs:1399: multiple definition of `core::num::from_str_radix_panic_rt'; obj s/ngx_rust_examples/ngx-release/examples/libawssig.a(awssig-22b3aae1e52dce40.core-f5a882967048065e.core.ecb3f420ec5f4bcf-cgu.0.rcgu.o.rcgu.o):/rus tc/f6e511eec7342f59a25f7c0534f1dbea00d01b14/library/core/src/num/mod.rs:1399: first defined here /usr/bin/ld: objs/ngx_rust_examples/ngx-release/examples/libcurl.a(curl-de3e6fe882b906f0.core-f5a882967048065e.core.ecb3f420ec5f4bcf-cgu.0.rcgu.o. rcgu.o): in function `... u.0.rcgu.o.rcgu.o): in function `core::fmt::num::<impl core::fmt::LowerHex for i8>::fmt': /usr/bin/ld: objs/ngx_rust_examples/ngx-release/examples/libcurl.a(curl-de3e6fe882b906f0.core-f5a882967048065e.core.ecb3f420ec5f4bcf-cgu.0.rcgu.o. rcgu.o): in function `core::fmt::num::<impl core::fmt::UpperHex for i8>::fmt': /rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14/library/core/src/fmt/num.rs:155: multiple definition of `core::fmt::num::<impl core::fmt::UpperHex for u8>::fmt'; objs/ngx_rust_examples/ngx-release/examples/libawssig.a(awssig-22b3aae1e52dce40.core-f5a882967048065e.core.ecb3f420ec5f4bcf-cgu.0. rcgu.o.rcgu.o):/rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14/library/core/src/fmt/num.rs:155: first defined here /usr/bin/ld: objs/ngx_rust_examples/ngx-release/examples/libcurl.a(curl-de3e6fe882b906f0.core-f5a882967048065e.core.ecb3f420ec5f4bcf-cgu.0.rcgu.o. rcgu.o): in function `core::fmt::num::imp::<impl core::fmt::Display for u64>::fmt': /rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14/library/core/src/fmt/num.rs:313: multiple definition of `core::fmt::num::imp::<impl core::fmt::Dis play for usize>::fmt'; objs/ngx_rust_examples/ngx-release/examples/libawssig.a(awssig-22b3aae1e52dce40.core-f5a882967048065e.core.ecb3f420ec5f4bcf -cgu.0.rcgu.o.rcgu.o):/rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14/library/core/src/fmt/num.rs:313: first defined here /usr/bin/ld: objs/ngx_rust_examples/ngx-release/examples/libcurl.a(curl-de3e6fe882b906f0.core-f5a882967048065e.core.ecb3f420ec5f4bcf-cgu.0.rcgu.o. rcgu.o): in function `core::fmt::Formatter::write_str': /rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14/library/core/src/fmt/mod.rs:1625: multiple definition of `<core::fmt::Formatter as core::fmt::Writ e>::write_str'; objs/ngx_rust_examples/ngx-release/examples/libawssig.a(awssig-22b3aae1e52dce40.core-f5a882967048065e.core.ecb3f420ec5f4bcf-cgu.0. rcgu.o.rcgu.o):/rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14/library/core/src/fmt/mod.rs:1625: first defined here /usr/bin/ld: objs/ngx_rust_examples/ngx-release/examples/libcurl.a(curl-de3e6fe882b906f0.core-f5a882967048065e.core.ecb3f420ec5f4bcf-cgu.0.rcgu.o. rcgu.o): in function `core::fmt::num::<impl core::fmt::LowerHex for i64>::fmt': /rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14/library/core/src/fmt/num.rs:155: multiple definition of `core::fmt::num::<impl core::fmt::LowerHex for usize>::fmt'; objs/ngx_rust_examples/ngx-release/examples/libawssig.a(awssig-22b3aae1e52dce40.core-f5a882967048065e.core.ecb3f420ec5f4bcf-cgu .0.rcgu.o.rcgu.o):/rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14/library/core/src/fmt/num.rs:155: first defined here /usr/bin/ld: objs/ngx_rust_examples/ngx-release/examples/libcurl.a(curl-de3e6fe882b906f0.core-f5a882967048065e.core.ecb3f420ec5f4bcf-cgu.0.rcgu.o. rcgu.o): in function `core::fmt::num::<impl core::fmt::UpperHex for i64>::fmt': /rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14/library/core/src/fmt/num.rs:155: multiple definition of `core::fmt::num::<impl core::fmt::UpperHex for usize>::fmt'; objs/ngx_rust_examples/ngx-release/examples/libawssig.a(awssig-22b3aae1e52dce40.core-f5a882967048065e.core.ecb3f420ec5f4bcf-cgu .0.rcgu.o.rcgu.o):/rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14/library/core/src/fmt/num.rs:155: first defined here /usr/bin/ld: objs/ngx_rust_examples/ngx-release/examples/libcurl.a(curl-de3e6fe882b906f0.core-f5a882967048065e.core.ecb3f420ec5f4bcf-cgu.0.rcgu.o. rcgu.o): in function `core::fmt::num::<impl core::fmt::LowerHex for i64>::fmt': /rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14/library/core/src/fmt/num.rs:155: multiple definition of `core::fmt::num::<impl core::fmt::LowerHex for u64>::fmt'; objs/ngx_rust_examples/ngx-release/examples/libawssig.a(awssig-22b3aae1e52dce40.core-f5a882967048065e.core.ecb3f420ec5f4bcf-cgu.0 .rcgu.o.rcgu.o):/rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14/library/core/src/fmt/num.rs:155: first defined here /usr/bin/ld: objs/ngx_rust_examples/ngx-release/examples/libcurl.a(curl-de3e6fe882b906f0.core-f5a882967048065e.core.ecb3f420ec5f4bcf-cgu.0.rcgu.o. rcgu.o): in function `core::fmt::num::<impl core::fmt::UpperHex for i64>::fmt': /rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14/library/core/src/fmt/num.rs:155: multiple definition of `core::fmt::num::<impl core::fmt::UpperHex for u64>::fmt'; objs/ngx_rust_examples/ngx-release/examples/libawssig.a(awssig-22b3aae1e52dce40.core-f5a882967048065e.core.ecb3f420ec5f4bcf-cgu.0 .rcgu.o.rcgu.o):/rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14/library/core/src/fmt/num.rs:155: first defined here collect2: error: ld returned 1 exit status make[1]: *** [objs/Makefile:310: objs/nginx] Error 1 make[1]: Leaving directory '/media/psf/workspace/nginx/nginx' make: *** [Makefile:10: build] Error 2 (747 of 929): multiple definition of `core::num::from_str_radix_panic_rt'; objs/ngx_rust_examples/ngx-release/examples/libawssig.a(awssig-22b3aae1 e52dce40.core-f5a882967048065e.core.ecb3f420ec5f4bcf-cgu.0.rcgu.o.rcgu.o):/rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14/library/core/src/num/mod .rs:1399: first defined here

Removing lto = "thin" from profile.ngx-release solves the issue.

xeioex · 2025-02-05T06:10:10Z

examples/auto/rust

+#     ngx_module_name=ngx_http_example_module
+#     ngx_module_incs=
+#     ngx_module_deps=
+#     ngx_module_libs=


After thinking about it a bit more, I agree that ngx_rust_ prefix for auto/module variables make the script more complicated.

Closing this remark.

xeioex · 2025-02-05T05:53:28Z

examples/auto/rust

+
+ngx_rust_module () {
+    ngx_addon_id=$(echo "$ngx_addon_name" | sed -e 's/[^A-Za-z0-9_]/_/g')
+    ngx_rust_target_name=${ngx_rust_target_name:-ngx_module_name}


if $ngx_rust_target_name is not defined, $ngx_rust_target_name becomes "ngx_module_name"

xeioex · 2025-02-05T06:07:28Z

examples/auto/rust

+
+
+# Registers a module in the buildsystem.
+# Expects two variables to be set:


"Expects two variables to be set:"
->
"In addition to auto/module variables expects two variables to be set:"?

bavshin-f5 force-pushed the portability branch from ee02bb9 to 5cb033e Compare January 25, 2025 08:45

bavshin-f5 added 3 commits January 29, 2025 15:45

bavshin-f5 force-pushed the portability branch 2 times, most recently from c19590d to e6426f9 Compare January 31, 2025 00:08

bavshin-f5 added 2 commits February 3, 2025 14:11

feat: library for nginx auto/configure integration

e79df6c

A new file, `examples/auto/rust`, can be used in the module projects to simplify the build and hide most of the platform-specific details. `examples/config` and `examples/config.make` are reimplemented using the library.

doc(sys): document NGINX_SOURCE_DIR and NGINX_BUILD_DIR

f8bff78

bavshin-f5 force-pushed the portability branch from e6426f9 to f8bff78 Compare February 3, 2025 22:35

bavshin-f5 requested a review from xeioex February 3, 2025 22:49

xeioex suggested changes Feb 5, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Platform support and `auto` scripts improvements #124

Platform support and `auto` scripts improvements #124

bavshin-f5 commented Jan 25, 2025 •

edited

Loading

xeioex commented Jan 25, 2025

xeioex Jan 25, 2025

xeioex Feb 5, 2025

xeioex Feb 5, 2025

xeioex Feb 5, 2025

xeioex Feb 5, 2025



		# Registers a module in the buildsystem.
		# Expects two variables to be set:

Platform support and auto scripts improvements #124

Are you sure you want to change the base?

Platform support and auto scripts improvements #124

Conversation

bavshin-f5 commented Jan 25, 2025 • edited Loading

Footnotes

xeioex commented Jan 25, 2025

xeioex Jan 25, 2025

Choose a reason for hiding this comment

xeioex Feb 5, 2025

Choose a reason for hiding this comment

xeioex Feb 5, 2025

Choose a reason for hiding this comment

xeioex Feb 5, 2025

Choose a reason for hiding this comment

xeioex Feb 5, 2025

Choose a reason for hiding this comment

Platform support and `auto` scripts improvements #124

Platform support and `auto` scripts improvements #124

bavshin-f5 commented Jan 25, 2025 •

edited

Loading