Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fs: avoid contents copy for Vec<u8>, String, Bytes write #7199

Merged
merged 3 commits into from
Apr 8, 2025

Conversation

paolobarbolini
Copy link
Contributor

@paolobarbolini paolobarbolini commented Mar 7, 2025

Motivation

As discovered by #7183, tokio::fs::write always copies the whole buffer, even when it's not necessary (the buffer is Send + 'static).

Solution

This issue could have been fixed if we had proper specialization, but without it and with references involved we can't use Any from std directly. However, David made the typeid crate for comparing TypeIds of non-'static types. The bincode folks made their own crate, unty crate, based on this same idea. This gives us poor's man specialization, allowing us to avoid the copy for Vec<u8>, String and Bytes. Box[<u8>], Arc<[u8]> and many other Send + 'static types could also be added in the future. Yesterday Armin also made a crate for doing something similar to this PR https://lucumr.pocoo.org/2025/3/23/from-string/.

Compiler explorer shows good enough output and correct proper functioning https://rust.godbolt.org/z/ef5GaooxW.

Fixes: #7183

@Darksonn Darksonn added A-tokio Area: The main tokio crate M-fs Module: tokio/fs labels Mar 7, 2025
@paolobarbolini paolobarbolini marked this pull request as ready for review March 24, 2025 06:49
@paolobarbolini
Copy link
Contributor Author

I've removed the dependency on unty. Meanwhile I've discovered that David also has a crate for doing this called typeid, so I guess this is hack is here to stay. It makes sense to me anyway, unless providing a different generic value to a type (PhantomData in this case) doesn't get assigned a different TypeId in the future...

Copy link
Contributor

@Darksonn Darksonn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I asked about this particular hack on zulip and I am ok with using this hack.

@Darksonn Darksonn merged commit 817fa60 into tokio-rs:master Apr 8, 2025
82 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-tokio Area: The main tokio crate M-fs Module: tokio/fs
Projects
None yet
Development

Successfully merging this pull request may close these issues.

tokio::fs::write clones the data
2 participants