Responder::respond_to not async, so unsure how implementing etags is possible?

[scullionw]

use std::io::Cursor;

use rocket::{
    response::{self, Responder},
    Request, Response,
};

use pollster::FutureExt as _;

pub struct Etag<T>(pub T);

#[rocket::async_trait]
impl<'r, T> Responder<'r, 'static> for Etag<T>
where
    T: Responder<'r, 'static>,
{
    fn respond_to(self, request: &'r Request<'_>) -> response::Result<'static> {
        let mut response = self.0.respond_to(request)?;

        let bytes = {
            let bytes_future = response.body_mut().to_bytes(); // <--- this is a future, but this isn't an async function

            bytes_future.block_on().unwrap() // <--- i use pollster here to get it to compile..
        };

        let etag = format!("{:x}", md5::compute(&bytes));

        Response::build_from(response)
            .raw_header("ETag", etag)
            .sized_body(bytes.len(), Cursor::new(bytes))
            .ok()
    }
}

^ this compiles because I use pollster to block on the future, which is less than ideal obviously.

not sure how else to build this? I was trying to reuse rocket's serialization of the responder, to avoid serializing twice.

i see in the docs that an async constructor is recommended, but i don't think this can apply since i need access to the serialized response in order to create an etag (which is the entire point)

[SergioBenitez]

Rocket is purposefully making it very hard for you to do this. You should never read the entire response body data. It is not only inefficient but also logically incorrect as the data is potentially infinite. This makes it a trivially exploitable DoS attack vector. Even if the data isn't infinite, it could still be problematic. For example, if a large file is being returned, your implementation will read the entire file into memory. This is not only another example of a DoS vector, but also very slow.

I would suggest a more Rockety / Rusty approach. Here you're taking any response type T and attempting to hash it to generate an ETag. A better strategy would be to parameterize the generic on a trait that provides the hash. That is: impl<T: Taggable> ETag<T>. You can then provide efficient implementations, and use async if you must. For example:

enum Tag {
    Weak(..),
    Strong(..),
}

trait Taggable {
    async fn tag(&self) -> Tag;
}

impl<T: Taggable> ETag<T> {
    async fn new(item: T) -> Self {
        let tag = item.tag().await;
        ETag(tag, item)
    }
}

A good implementation for File (and NamedFile) would be to use the hash of the modification time and/or other metadata. This gives you a hard hash. For strings, you could decide to use a weak hash of its length or actually hash the string - but at least the string is already in memory and you don't create a second copy of it to do so. With specialization, you could also make this work efficiently for a lot of types with very little effort.

[scullionw]

Yeah the trait is almost what I had going on in my first pass, but for types that can't use a modification time, i eventually need to compute the md5. That is when I ran into the issue of serializing twice. Would you see any way around that duplicated work?

The other option I am seeing is to convert my type into a buffer inside an Etag::new(). Then in the impl Responder I can calculate the hash and send the bytes. That works. The problem with that solution is I lose all the headers/things the inner Responder was setting. that the Responder types don't implement serialize.

edit: This almost works but still serializes twice. Would there be a way to prevent rocket from serializing since I am setting the sized_body myself?

pub struct Etag<T>(pub T);

// Impl Etag::new for types that Implement responder and EtagHash trait
#[rocket::async_trait]
impl<'r, T> Responder<'r, 'static> for Etag<T>
where
    T: Responder<'r, 'static>,
    T: Deref,
    T::Target: Serialize,
{
    fn respond_to(self, request: &'r Request<'_>) -> response::Result<'static> {
        let bytes = serde_json::to_vec(&*self.0).unwrap();

        let response = self.0.respond_to(request)?;

        let etag = format!("{:x}", md5::compute(&bytes));

        Response::build_from(response)
            .raw_header("ETag", etag)
            .sized_body(bytes.len(), Cursor::new(bytes))
            .ok()
    }
}

If there is a way I will be able to generalize with the Taggable in your example and do the work in the constructor possibly.

[the10thWiz]

As presented, you have to serialize twice, since they could be serialized to different formats. This is trivially true for Etag<MsgPack<T>>, which would wind up with an Etag that represents hash of the same data in JSON. If you can specialize your implementation to know what the final format is, you can avoid the second serialization by using RawJson(bytes) rather than self.0 (a Json(T)), which will automatically set all the headers and content type associated with JSON, but doesn't actually serialize the contents.