Support for serialization and deserialization of JSON

[willemolding]

Given the strong type requirements of AS it would be ideal to be able to parse a JSON string directly into a given class. A decorator could be used to generate the serialize/deserialize functionality on the class at compile time.

@serializable
@deserializable
class T {
  x: i32
  y: string
}

let obj: T = JSON.parse<T>(jsonStr)

This would return an error code if jsonStr is invalid or not deserializable to the class.

[dcodeIO]

Looks like a nice API to me, yeah. Maybe the decorators should be on the serializable fields instead, in case just some fields are meant to be serialized. Classes could even be serializable by default (all fields if none are specifically annotated) due to the fact that parse<T> is statically typed (would be a builtin that could then build the encoders/decoders once it becomes compiled).

Might also be possible to start prototyping something like this right away as a custom transform.

[willemolding]

Oh yeah you are right about the fields. There is also the possibility that the field names are different to the key strings in the JSON string you want to decode so maybe something like:

class Address {
  @deserializable('house_number')
  houseNumber: i32
...
}

As you say it would be possible to infer which classes need encoders/decoders to be generated at compile time by which ones are used as a generic in a call to parse<>.

In terms of implementation I feel like it would be best to leverage as much of an existing JSON parsing library as possible. Even one written in another language that can compile to wasm such as rust Serde or C++ RapidJSON. Any thoughts on what an interface to this might look like?

[dcodeIO]

Even one written in another language

I'd prefer one written in TypeScript so it can just be part of the standard library. Porting a fast and compact one seems like an ideal option.

Edit: The implementations you mentioned seem to have their own implementations of dtoa, regexpes etc. for example, that could actually be shared with other parts of the standard library.

[willemolding]

I started working on an event based JSON parser written in AS based on RapidJSON (https://github.com/willemolding/asm-json-parser). This is what convinced me that using a third party might be a better option. Writing a parser that fully complies with JSON standard is non-trivial and since it all compiles to wasm anyway possibly unnecessary

[dcodeIO]

possibly unnecessary

The disadvantage there is that we'd get duplicate code for similar things (parsing and serializing of float literals is on example), just because standalone parsers like these naturally can't reuse what's already provided by the strandard library. This doesn't matter much for a desktop application, but reducing code size makes sense on the web.

I agree though that it should become easier to integrate non-AS code, if one wants to.

[MaxGraey]

@dcodeIO It will be great if we can resolve generics by returning value as well. Rust could do this. Just for example:

fn parse<T: Default>(_arg: &str) -> T {
    println!("{}", unsafe { std::intrinsics::type_name::<T>() });
    T::default()
}
fn main() {
    let _foo: i32 = parse("abc");
    // print "i32"
}

It would be allow us use JSON.parse without specify typename in brackets:

let obj: Foo = JSON.parse(fooStr);
// or
let obj = JSON.parse(fooStr) as Foo;

Instead:

let obj = JSON.parse<Foo>(fooStr);

which also possible but looks less natural for TypeScript

[dcodeIO]

If JSON.parse were a builtin, it could do type inferrence like other builtins as well. If it isn't a builtin, there'd have to be another way as you propose.

[MaxGraey]

Hmm, I think resolving generics in that way will be useful in any case. But if you see this hard for now making JSON as builtin also great

[dcodeIO]

The ideal case could be to do all of this with proper reflection APIs, like getting a list of field descriptors (name, type, if optional, default value etc.) and working with that. Involves GC-managed descriptor objects, of course, which might not be super efficient right now.

[MaxGraey]

Sounds promising and it seems this allow creating custom decorators without post-processing via --transform. But also require a lot of efforts for now

[willemolding]

From our perspective (using AS for writing holochain apps) we would prefer not to require GC and have everything happen at compile time

[willemolding]

In the interest of keeping code size small and reusing existing float parsing functions etc. what do you think about either using or porting this lib (https://github.com/zserge/jsmn) ?

It seems to do a nice job decoupling the actual parsing of values from the delimiting of values from the JSON string.

[dcodeIO]

That one looks easy to port, yeah. I like its simplicity.

[MaxGraey]

Unfortunately jsmn is very-very slow