-
Notifications
You must be signed in to change notification settings - Fork 555
Remove unimplemented!("Unsupported target OS");
to allow _running_ on Linux and WSL
#1842
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Comments
I would love to simplify this. I think the current approach was originally proposed by @roblabla to support cross-compilation (e.g. targeting Windows from Linux) but if that still works with this change I'd love to go for it. |
It works, and I'm using it right now 🥳 @roblabla if you can test the linked branch and confirm whether this doesn't break your use-case, I'll wrap it up in a nice PR with the above summarized. EDIT: That said we do have to think about the proposed "Alternative approach". |
Hah, @kennykerr and I were talking about this during some recent GNU toolchain work. I don't think any of these configuration attributes are needed, for the reasons you stated (i.e. cross compilation targets windows). |
What does this refer to? We support cross-compilation on macOS, Linux, and Windows today and those scenarios are tested in CI. |
I think this is the original PR: #830 |
@riverar The CI can build tests but it cannot run |
@MarijnS95 Oh those tools, got it. Thanks! |
It looks like this only moved the Possibly for docs.rs? Perhaps this wasn't possible back then, but nowadays - and you're making use of it already: windows-rs/crates/libs/windows/Cargo.toml Lines 14 to 16 in b8d83c2
|
It was to allow the Rust compiler devs (who didn't use Windows at the time) to build and profile the generated code. Specifically, profile the compiler while building and generating the code. |
@kennykerr Is that still ongoing? How do you think we should proceed?
In the end it comes down to what this project values most:
|
I suggest we just drop the |
That seems reasonable, I can work around it with a custom library to link against. I can clean up the linked branch and open a PR with that.
Thanks (Microsoft in general) for trying so hard to make this crate unusable for me though, while at the same time pulling the plug on EDIT: IIRC the previous objection was the |
I thought you said that would work for you. 😅 My understanding was that cross compilation was important to most folks and that dropping cfg wouldn't be a problem for you since the linker would only try to resolve imports for functions you're actually calling. Perhaps I missed something, but its not because we're trying to make your life difficult. 😜 |
I was confused as well, so I whipped up a dxcore sample to test the end-to-end dev experience here and identified several blockers, most of which Marijn mentioned:
Did I miss anything @MarijnS95? Do you agree with these proposed fixes? |
I think it would be helpful to clarify that this issue is really "support cross-platform for dxcore" rather than "simplify link cfg guards". If that's not too disruptive, as @riverar mentioned, and can be tested via yml, then this seems doable. |
I wonder if this misunderstanding persevered over the past year that I tried to contribute to this repository, that'd explain a whole lot. I'm not using DXCore, which serves Graphics/Compute Adapter enumeration and obviously requires a physical Windows system with graphics drivers etc. I'm not cross-compiling any of these crates (from a non-Windows OS) for consumption on a Windows OS. I'm using DXC - short for DirectXShaderCompiler. DirectXShaderCompiler provides libraries and binaries that run natively on Linux. Said library exposes its functionality through a COM-like interface, also on Linux (C++ vtables, I'm currently maintaining a crate that provides Rust bindings into this library. It leverages the ancient
But there's one caveat. As it stands it doesn't run natively on Linux (or other non-Windows targets). And it seems to be the exact same situation as ±1 year ago: this is trivially fixable, to make the There are effectively only 3 smaller issues that I see:
Now, back to our regular programming of answering individual replies:
Dropping the
As explained in the issue even if no function from an Alternatively we can perhaps turn this into a custom feature/
Understood, I hope the above makes it more clear that I'm not interested in cross-compiling but running the crate itself directly on Linux (and MacOS). And by extent
(Drop the As above: this'd help
It is, in essence, simplify link cfg guards, and that'd implicitly bring us closer to "support cross-platform DirectXShaderCompiler Rust bindings". We just have to bikeshed the semantics:
By YML, you mean GitHub Actions, canonically "the CI"? |
Just here to confirm that the proposed simplification would work for my (cross-compilation) use-case. The cross-compilation MR added this |
Oh my mistake for the dxcore/DirectXShaderCompiler mix up. But they're effectively the same in this case. (Both run on Linux and do not need a Windows system.) Can you confirm that these exact fixes would resolve this? I'm trying to wrap this up this thread. In the interest of moving forward, please keep your answers terse thanks! |
Right, but
Sometimes one has to write a one-off elaborate bit of text to get a point across properly and completely. As I seem to have failed that thus far, this couldn't come in terse. Not intending to do that more often but do hope it helps in understanding! |
Yep, targeting
Oh I wasn't suggesting to guard
I like your counter proposal of dropping
Understood. I was very conflicted with myself when I wrote that. It just felt like I was reading walls of text repeating a lot of the same information. But I understand there were some errors in the thread and you needed to get everyone on the same page. |
Perfect, thanks for confirming!
Ah, yes, we actually have two issues here! One is use of I don't think my use-case for
The only downside is that you don't get to specify what library to pull the symbol from, but that's fine. It also allows for my use-case of of one-off At the same time it gets rid of the linker errors in
And I was very conflicted with myself as well, initially writing "Now, back to our regular programming of answering individual replies, leaving out anything that has already been described above:". The bold part was removed in post-editing as I did end up restating a lot of the same, you're not mistaken there 😬 |
Note that dropping this would change the compiled code on Windows due to the way imports work. In C++ terms using
This would avoid the above issue. |
@ChrisDenton What's the advantage of https://docs.microsoft.com/en-us/cpp/build/importing-into-an-application-using-declspec-dllimport?view=msvc-170 isn't very clear and merely states:
|
To sum it up briefly, in x86/x64 assembly calling an imported function requires an instruction that is larger than the one for directly calling a function in the binary. Therefore when you don't use |
@ChrisDenton Note that this shouldn't perform extra jumps to acquire a symbol resolved/linked at runtime, the libraries are statically linked. Doesn't the linker turn this into a cheap |
The |
@MarijnS95 Can you provide some context to this?
I was looking around DXC for evidence of this fundamental string encoding difference but couldn't immediately find one. Can you point me the right direction? Is there a set of DXC APIs that take in differently encoded text based on platform? (Sounds terrible if so.) |
@riverar DXC delegates to This type is typically 32-bit except on Windows where it represents UTF16:
EDIT: Alas, I said I wanted to discuss this in a separate issue :) |
@MarijnS95 Got it, thanks. I learned something new today. @ChrisDenton Good point re: the less efficient indirection. I would say that disqualifies the dropping-cfg option until we crunch some numbers and see if there's meaningful impact there. (LTO may narrow any gap?) So I think we can move forward for now with the changes listed above. If you agree we've settled on something usable, I can pester Kenny for sign-off. 🙃 |
We'd opt for point 3, as an alternative to point 2 for now until investigating whether dropping the Then regarding my long post, do we opt for solution 2/3/4?
Let's wait for @kennykerr to voice his preference! |
I believe @kennykerr is away for the week and so it might take a bit longer to hear from him. As for my 2¢, it seems @ChrisDenton provides an argument for why you would want to use If not, then option #2 sounds like the most reasonable option. This also leaves open the possibility of moving to option #3 in the future in a backwards compatible way. |
@rylev Just in case, I think the most backwards-compatible option to move to would be #4 (sorry, I quoted point 2-4 my post because 1 is "out the door"): I'm also curious, but perhaps our most non-destructive option for now is remove EDIT: Should we wait for Kenny before going ahead with the changes / PR? |
Can someone succinctly describe what the issue is because this thread is long and it's not yet clear which of the many issues in this thread is the actual problem, or first problem, folks want to solve. And "simplify x" is not a problem, it's a vague solution to an undefined problem. Merely simplifying some code gen would not require so many comments. 😉 |
#[link]
cfg
-guards into cfg_attr
unimplemented!("Unsupported target OS");
to allow _running_ on Linux and WSL
@kennykerr there, it's all in the title now. You'll have to catch up with the comments though to understand the alternative solutions to solve a subsequent "issue" around |
To be honest, I'm unclear what it means for a Windows crate to "support" Linux. Do you just want to share some types with windows-rs? Or are you planning to implement some Windows functions on Linux/WSL? |
@ChrisDenton I've explained my use-case many times, even @riverar took a stab at it. Everyone seems to be fed-up with the amount of comments in this issue already, so no: I won't re-re-re-re-re-re-re-re-explain it again, you'll just have to scroll up or click #1842 (comment). |
So to sum it up succinctly, the end goal is to provide Rust bindings to DirectXShaderCompiler on Linux systems. Windows-rs can provide the needed COM auto-generation (vtables, AddRef/Release) and associated types (e.g. BSTR). Though there are some differences between platforms (e.g. Linux BSTR uses 32-bit code units whereas Windows uses 16-bit). The problem this specific issue aims to address is that the current |
Exactly, and it doesn't seem to be limited to that. @riverar proposed bindings to
For |
@kennykerr DirectX has a few cross-platform APIs and very minimal changes are being proposed to make @MarijnS95 Can you submit a PR? Let's move forward with the changes listed in #1842 (comment). (Whether you want to use |
❤️ @riverar thanks, I'll submit it shortly. I'll go for |
Hm we don't want any new features at this time, those are hard to change/use. Let's stick to what's in the linked comment and we can re-evaluate afterwards. |
The "problem"
Currently the
windows
crate is allowed to compile on non-windows
systems thanks to a rather large block of code, replicated across every function:I remember discussing this long ago: it was needed for CI but seems to have never been removed, as there wasn't much reason to run on non-
windows
anyway.Rust is supposed to omit linkage when functions aren't used: any
extern "system" { pub fn blah(); }
doesn't show up in the symbol table if it isn't called into. However, this isn't the case for#[link]
:-lwindows
is added to the linker invocation even if no functions from theextern "system"
block it is annotating are called.That is evident in the
windows-sys
crate which doesn't carry anycfg(windows)
around#[link]
attributes: it is inconveniently used in themetadata
crate that's subsequently used by all the tools, not allowing any of them to be ran on non-Windows (= note: mold: fatal: library not found: windows
) even though they don't reference a single function, just some typedefs.Proposed solution
Now, combine those two points and we can turn the above into:
(Dis)advantages
Disclaimer: most of the below applies to non-
windows
only, which perhaps still isn't "in scope" of the project - but I do hope we can discuss and consider this, especially in light ofcom-rs
being officially deprecated! 1This is much shorter and simpler (multiplied by thousands of functions), but there's an obvious catch: whenever this function is used, even in conditional code (consider for example
error.rs
using various functions for retrieving and stringifying/prettifyingError
), linking fails.Perhaps that's (much) better than an unsuspecting
unimplemented!()
being reached in an optional codepath (e.g.Error
handling).However, there's an immediate solution too. And I've alluded to / requested this in earlier issues: define your own fallback.
(This goes anywhere in a user's codebase, could even be a dependent crate)
(Badly chosen example, this'd be better on e.g.
SysStringLen
.)Then the linker error is gone again, and the user has the ability to provide the necessary functions themselves at no additional design cost to
windows-rs
🎉.Proof of concept
You can find the end result here: https://github.com/MarijnS95/windows-rs/compare/simplify-cfg-windows (
Showing 429 changed files with 71,379 additions and 157,268 deletions
). As bonus this includes the samecfg_attr
inwindows-sys
to allow running the tools on non-Windows.Alternative (even smaller) approach
Perhaps this isn't necessary (read: remove the
#[cfg(windows)]
entirely) at all if a blanketlibwindows.so
is provided by the user, possibly built straight from a Rust crate with aforementioned fallback implementations. This can be done directly throughcargo
by providing a:Such a crate can be specified directly as a dependency in another crate even if no Rust symbols are referenced and it's not explicitly linked against:
"windows-fallback"
here still builds and itslibwindows.so
result ends up in a path that's already part of the search path to cover the automatically-inserted-lwindows
🎉 (may be a bit fragile to depend on though?).That however doesn't provide a solution to the metadata/tools currently being only compilable on Windows, and I'm not suggesting to provide this crate within
windows-rs
!Footnotes
I am of course again trying to find a proper, maintained, clean Rust alternative for
DirectXShaderCompiler
COM bindings. This is perhaps one of the few - or only? - binaries that has a functioning COM API outside of Windows. The proposed Quality-of-Life improvements will shorten generated code and allow me to usewindows-rs
outside of Windows at no additional maintenance cost here. ↩The text was updated successfully, but these errors were encountered: