Let JsonFormatter implement Serializer

[laeubi]

Once in a while I need to write json output in some of my plugins as well but it is surprisingly hard to do so as the API to do so is all internal and non trivial to implement.

On the other hand the JsonFormatter already has everything we need here, so this lets JsonFormatter now implement Serializer to conviently support this use-case

@mpkorstanje what do you think? I just (again) encountered the problem that my custom hack for this broke the Eclipse-Cucumber plugin, having a semi official API for that would be great and it does not exposes to much of the internals but still allow to reliable write out the JSON (e.g. to a file or websocket) in custom plugins.

[mpkorstanje]

Would the MessageFormatter be something that you could use?

Though admittedly I'm a bit confused about what your use case is exactly.

Once in a while I need to write json output in some of my plugins as well but it is surprisingly hard to do so as the API to do so is all internal and non trivial to implement.

Here it sounds like you want to write arbitrary json. But the Serializer interface only permits writing cucumber messages (in an envelope) to json. So as proposed it seems the pull request doesn't meet your own requirements.

[laeubi]

Would the MessageFormatter be something that you could use?

Good catch, that would be a better place to implement that interface

But it has the same limitations as JsonFormatter:

1. No way to write an Envelope directly I need to somehow "trick" it by injecting an EventPublisher to get access to the actual write method
2. The code takes control over the OutputStream (e.g. closing it in some cases)
3. There is no way to write individual frames unless I create a new MessageFormatter + Output stream for each message (what would work but feels a bit wasted)
4. No way to extend as everything is (package) private

Here it sounds like you want to write arbitrary json.

No, I want exactly that:

But the Serializer interface only permits writing cucumber messages (in an envelope) to json.

So this hopefully makes it more clear:

1. I have an endpoint (actually a socket) that requires a special frame format (here the number of bytes as an int + a payload that is a Json encoded cucumber Envelope)
2. I start cucumber with a special plugin that simply listen to message (like MessageFormatter does) and when I receive one I convert it to the Json into a byte array and send the actual size+bytes over the wire
3. Then on the endpoint side I decode it again (for what yet I have also not found a good way in cucumber directly)

This all is used to observer the test-run of a forked cucumber test in eclipse-cucumber plugin.

[mpkorstanje]

Unfortunately Java doesn't come with a builtin JSON serializer/deserializer. The MessageToNdjsonWriter and NdjsonToMessageIterable are designed with that in mind by requiring that users bring their own. And eventually Cucumber will require that too (#3034).

I don't mean to XY-problem you, but it really seems like you need a JSON serializer/deserializer. What is the reason you can't bring your own serializer?

I start cucumber with a special plugin that simply listen to message (like MessageFormatter does) and when I receive one I convert it to the Json into a byte array and send the actual size+bytes over the wire

I did see you were working on something similar for JUnit (junit-team/junit-framework#5096, ota4j-team/open-test-reporting#728). There it seems writing to an output stream is sufficient?

But for what it is worth, the MessageFormatter writes '\n delimited output. You could create an OutputStream that buffers the output until a \n is seen. Something like:

ByteArrayOutputStream delegate = new ByteArrayOutputStream();

OutputStream outputStream = new OutputStream() {
    private final ByteArrayOutputStream buffer = new ByteArrayOutputStream();

    @Override
    public void write(int b) throws IOException {
        if (b == '\n') {
            delegate.write(buffer.size());
            buffer.writeTo(delegate);
            buffer.reset();
        } else {
            buffer.write(b);
        }
    }
};

MessageFormatter messageFormatter = new MessageFormatter(outputStream);

[mpkorstanje]

Alternatively would it help if messages implemented the serializable interface?

[laeubi]

Unfortunately Java doesn't come with a builtin JSON serializer/deserializer. The MessageToNdjsonWriter and NdjsonToMessageIterable are designed with that in mind by requiring that users bring their own.

That's why I found it quite smart if the ones here implement the Serializer interface (same for HTml of course by the way).

I don't mean to XY-problem you, but it really seems like you need a JSON serializer/deserializer. What is the reason you can't bring your own serializer?

The plugin should be as minimal as possible (as I don't know what people are using) and I don't want to bring in something new, and as cucumber already support this and I want exactly this it seems wasted effort anyways.

I did see you were working on something similar for JUnit (junit-team/junit-framework#5096, ota4j-team/open-test-reporting#728). There it seems writing to an output stream is sufficient?

The protocol requires an ACK (so I can hold on the remote process for debugging steps), the formatters only write and forget (what is good!) so I really need to know the boundaries. Also i probably want to send other commands over the channel and the writer possibly do not give me enough control (e.g about flush).

But for what it is worth, the MessageFormatter writes '\n delimited output. You could create an OutputStream that buffers the output until a \n is seen. Something like:

So as said its all possible somehow but not very convenient and I'm bound to how the internal implementation works and I'm not sure I want to put to much burden on it. To be concrete, would you say it will always be the case now and in the future?! And always a single \n ... so implementing an official interface from the messages bundle seems a much more stable and future proof contract to me.

Alternatively would it help if messages implemented the serializable interface?

Not really as then it would require exact same class versions on client/server.

[mpkorstanje]

Would this be sufficient?

public final class ObjectToJsonWriter {
    
    @Override
    public void writeValue(Writer writer, Object value) throws IOException {
        Jackson.OBJECT_MAPPER.writeValue(writer, value);
    }
}

[laeubi]

@mpkorstanje yes that would of course be suitable as well, I just noticed that in this case an output stream might be more flexible? (and one maybe want to use that in the Message/Jsonformater then as well for consistency).

it would of course be great to have the reverse as well for symmetry:

public class JsonReader {
	public <T> T readValue(Class<T> type, byte[] buffer, int offset, int length) throws IOException {
		return Jackson.OBJECT_MAPPER.readerFor(type).readValue(buffer, offset, length);
	}

	public <T> T readValue(Class<T> type, InputStream inputStream) throws IOException {
		return Jackson.OBJECT_MAPPER.readerFor(type).readValue(inputStream);
	}
}

Thinking further in the sense of the API consistency it might be better to simply have a class implement

• io.cucumber.messages.MessageToNdjsonWriter.Serializer
• io.cucumber.messages.NdjsonToMessageIterable.Deserializer

this would also make clear that we write/get an Envelope here...

[mpkorstanje]

I'm going to think about it for a bit.

Providing a json serializer/deserializer has a lot of complexity attached. And I'm worried that it could open a maintaince sinkhole.