Generic flat response implementation

[D0zee]

No description provided.

[anarthal]

I've reviewed the flat_response_impl part. It looks what I'd expect. Marcelo is more suited to review the rest, since I don't really know how adapters work. Thanks for submitting the patch.

[D0zee]

@anarthal It's not really ready, just wanted to make sure that I'm on right direction. Thank you for your review!

[mzimbres]

Hi @D0zee, I've looked at this PR again and think it has a good direction so I endorse further development. The two issues I talked about in the issue are minor and can be addressed in the future if at all, i.e.

1. offset_node vs offset_string.
2. node_view at(std::size_t) vs node_view const& at(std::size_t)

Thanks again for the time invested.

[D0zee]

Hi @mzimbres, I'm going to look at suggested issues in details and implement them to enhance performance and make the structure more user-friendly. Thank you for your comments in the issue

[mzimbres]

I think we can just replace generic_response with generic_flat_response in all occurrences.

[D0zee]

It will be good test for API of new class because I already see some absent methods like clear().

[mzimbres]

I would like to avoid the creation of two std::function for each request (i.e. async_exec calls) since each one represents a potential dynamic allocation. In this case however I think we can't do much better because inside async_exec we only have access to the type erased adapter i.e. std::function and therefore we can't call adapter.set_done() for example. I think the prepare_done_fn callback is not critical though since it is small and the dynamic allocation is likely to be optimized away with SOO (small object optimization). @anarthal What do you think?

[D0zee]

sizeof(generic_flat_response*)=8
sizeof(generic_flat_response&)=64
sizeof([](){})=1

• If response type is generic_flat_response we will return lambda with captured reference ([res]()mutable{}). The size of this lambda is 64 bytes, std::function will definitely keep it on heap. I suggest to capture and pass the pointergeneric_flat_response*. Its size is 8 bytes and in this case SOO will take place during creation of std::function in any_adapter. As a result we have no heap allocations in this case.

• If the response type is any other SOO will take a place by default due to the fact that size of empty lambda is 1 byte.

Source: https://stackoverflow.com/a/57049013

[mzimbres]

I have been thinking about this and there seems to be a simpler way of avoiding the extra std::function that I haven't considered earlier. First we define a special node value in node.hpp

constexpr resp3::node_view done_node{resp3::type::invalid, -1, -1, {}};

then call the adapter with this node when the parser is done here

template <class Adapter>
bool parse(resp3::parser& p, std::string_view const& msg, Adapter& adapter, system::error_code& ec)
{
   while (!p.done()) {
      ...
   }

   // ---> New adapter call here to inform parsing is finished.
   adapter(std::make_optional<resp3::parser::result>(done_node), system::error_code());

   return true;
}

Then call set_views() on the adapter when this node is detected here

template <>
class general_aggregate<result<flat_response_value>> {
private:
   result<flat_response_value>* result_;

public:
   template <class String>
   void operator()(resp3::basic_node<String> const& nd, system::error_code&)
   {
      // ---> Check whether done here.
      if (nd == done_node) {
         result_->value().set_views();
         return;
      }

      ...
   }
};

I totally missed this possibility earlier but it looks cleaner than adding a new std::function? Do you mind trying this?

[D0zee]

Yes, it looks much better and we don't have to care about heap allocations. I will try it!

[D0zee]

Am I correct that this case must be handled by others adapters as well?

[mzimbres]

Hi @D0zee, yes other adapters will have to handle this as well, for example

if (nd == done_node)
    return;

I forgot to say this in my previous comment.

[mzimbres]

Hi again, there is still one problem to solve. The way I suggested above will cause the set_view to be called multiple times, once for each request in the pipeline. That is because the same flat response can be used to store the response of multiple requests. I am still not sure what is the simplest way to avoid that, it can be solved by adding state to set_views() so that it does not traverse what has already been traversed.

Or perhaps there is a way to wrap the adapters at other location such as here

   template <class T>
   static auto create_impl(T& resp) -> impl_t
   {
      using namespace boost::redis::adapter;
      auto adapter = boost_redis_adapt(resp);
      std::size_t size = adapter.get_supported_response_size();
      return {std::move(adapter), size};
   }

or here

   adapter_ = [this, adapter](resp3::node_view const& nd, system::error_code& ec) {
      auto const i = req_->get_expected_responses() - remaining_responses_;
      adapter(i, nd, ec);
   };

so that only the last call to adapter(done_node, ec) triggers a set_view(). I think however this might be too complicated and perhaps it just simpler to let each call to set_view to traverse only what has not been set yet, as suggested above.

If I have time I will review all of this to see if there is any simplification possible.

[anarthal]

I might be late to the discussion, but the new API looks more inconvenient than what was proposed initially, leaking much many implementation details and requiring workarounds in the adapters. I guess there's a strong reason performance-wise to go this way. Do we have measures on how faster this approach is vs. the more convenient one?

[mzimbres]

@anarthal There are two problems to solve

1. The generic_flat_response api
2. Where to construct the string_views from the offsets.

Number 1. is the easier part and I would like to have it equal to generic_response. The current PR is almost there but actually requires calling resp.view(). I am still not sure whether this is ok but it is something easy to change.

Number 2. is what is causing problems. I am still trying to understand what is cleaner from the design and performance point of view and in a way that doesn't break existing code. We have already two callbacks associated with async_exec, the adapter (and its wrapper) and the set_done_callback. The latter is the best place where we can call flat_resp.set_view() but type erasure is preventing that. Adding a new callback is a poor workaround which makes the code more messy. The third option is to let parser notify the adapter it is done.

I think the third option is the only clean and sound option because it makes sense to know when parsing is complete from the adapter. But once we have that the set_done_callback existence become perhaps unnecessary.

Given this complexity I think it would be simpler to split this PR in two. @D0zee works only on 1. and I will work on part 2. so he has a sane way of calling set_views() that does not mess with the code. After I am done he can rebase and it should work.

[D0zee]

@mzimbres Let me please know when your part is ready. I think I will finish my part (generic_flat_response API) tomorrow and on weekends

[mzimbres]

@D0zee I will. I am currently finishing this PR and after it is merged I will investigate how to solve this one.

[anarthal]

We have already two callbacks associated with async_exec, the adapter (and its wrapper) and the set_done_callback.

Note that the done callback currently belongs to the I/O world (it interacts with a channel) rather than the parser world. IMO long-term the callback should be replaced by a reader action (as I think was your intention according to this comment).

Semantically, having adapters support a "done" callback looks sound to me. But as you said, there is the type erasing issue.

If you want, I can try to write a type-erased adapter type that encapsulates both functions, something like:

class any_adapter_impl {
    // stores the underlying adapter, akin to what std::function does for functions
public:
    void on_node(std::size_t, resp3::node_view const&, system::error_code&);
    void on_finished();
};

Then you can use this type in parser.

[mzimbres]

Note that the done callback currently belongs to the I/O world (it interacts with a channel) rather than the parser world. IMO long-term the callback should be replaced by a reader action (as I think was your intention according to this comment).

Yeah, that is an important realization. Extending the adapter with on_finished is only meant to aid building the response from the wire protocol.

Semantically, having adapters support a "done" callback looks sound to me.

I think we need three functions on_star(), on_node() and on_finish(). We actually have on_start() that is currently called on_value_available.

But as you said, there is the type erasing issue.

There is a simple way to deal with that. We can add a new parameter to the adapter, currently we have

template <class String>
void operator()(resp3::basic_node<String> const&, system::error_code&);

which could be changed to

enum class event { start, node, finish};

template <class String>
void operator()(event ev, resp3::basic_node<String> const& nd, system::error_code& ev)
{
   switch (ev) {
      case start: adapter.on_start(); return;
      case node: adapter.on_node(nd, ev); return;
      case finish: adapter.on_finish(); return;
   }
}

That would be a breaking change but probably nobody is writing adapters.

If you want, I can try to write a type-erased adapter type that encapsulates both functions, something like:

You are welcome, the adapter module has its complexity so feel free to ask. Also, please create a sub-issue in the corresponding ticket.

class any_adapter_impl {
// stores the underlying adapter, akin to what std::function does for functions
public:
void on_node(std::size_t, resp3::node_view const&, system::error_code&);
void on_finished();
};

Note that on_finished has to be called when each response is finished and not only once when the all responses were received. For example

request req
req.push("COMMAND1", ...);
req.push("COMMAND2", ...);
req.push("COMMAND3", ...);

response<T1, T2, T3> resp;

co_wait conn.async_exec(request, resp);

// on_finish will have been called three times when we get here, once
// for each response element.

I am noticing how much background knowledge this implementation requires, I should not have expected @D0zee to go through all these details, apologies.

[D0zee]

Hi @mzimbres, I've resolved the comments above and added api methods required for tests/examples. However I realized that examples don't work because we have to do the same trick as with set_done_callback in multiplexer here.

With adapter extension mentioned below we will be able to get rid of prepare_done callback in any_adapter and trick I mentioned below will be not needed. Looking forward to the implementation, it would unblock me. Thank you in advance!