Feature: Registering type extensions

[aseigo]

I am using exqlite with the SpatiaLIte extension. This works so long as the inputs and outputs are textual, but this also leaves parsing geometry returned up to the user, which is unfortunate. It would be nice to be able to use the geo library to handle these en/decodings.

This works with PostGIS as libraries such as geo_postgis are able to register type extension modules with postgrex which allow the data being used in queries to be encoded and decoded into native Elixir types, in this case enabling the transparent use of the structs found in the geo library.

I initially thought it might make sense to having something similar in exqlite, whose encoding/decoding is not extensible. Considering that sqlite really only has a static set of types for storage, however, it may make more sense to do the en/decoding in the ecto adapter instead via loaders/dumpers.

While writing the loader and dumper for geometries should be straight-forward, a way to make those available to the adapter is still missing.

With this issue, I'm looking to start a dialogue as to what this can/should look like. With an acceptable design and approach defined, I can take a swing at it with a PR .. but I'd rather start with something that has a chance of being accepted :)

[warmwaffles]

This has me thinking a little bit. SQLite has this https://www.sqlite.org/rtree.html that could be leveraged to enable geo spacial stuff. I do think it would be okay to put these extensions in this library as an optional data type to use. Perhaps under the module Ecto.SQLite3.Extensions But that's a lot to expect people to type 😞

Right now exqlite has no way to define custom types like Postgrex. And I think a similar interface could be implemented here. I didn't do that initially because I wanted something working and out the door.

[aseigo]

Achieving something that works first is absolutely the "right" approach :) The code can evolve once it is working, after all!

But that's a lot to expect people to type

If it is used (mostly) internally, then that's fine IMHO. There's always alias, too, of course. But mostly these should "just work" as Ecto maps the data types to loaders/dumpers .. which would mean that any time a %Geo.*{} struct appears in an Ecto.Schema, it would Just Work(tm) without the user having to do anything (or so the theory goes; it's how it works with the PostGIS postgrex extensions)

these extensions in this library as an optional data type to use

Just to make sure I am understanding this correctly: you'd be interested in a PR that adds Geo data types to the library?

I suppose it could be made a compile-time option, though I'm wary of (too many of) those in libraries I rely on, as it often makes the library a bit more complicated and those optional bits often get less testing...

Let me see how it could look in practice, and I'll return with a branch to look at ... and then we can discuss direction with something more concrete in hand.

Thanks for the prompt and supportive response!

[warmwaffles]

Feel free to experiment and open a PR. I can pull it down and play with it as well. Geo spatial support seems pretty useful in sqlite.

[aseigo]

Opened #167 as a draft PR. It works (though Spatialite is giving me separate issues) ... if this is a path forward that you approve of, then I can add some unit tests, etc.

[aseigo]

@warmwaffles Is there anything that needs attention in the PR? If not, do you have a rough timeline for merging it into a release?

No pressure, I'm just trying to figure out my own schedule :)

[warmwaffles]

Oh sorry @aseigo it's already deployed in the current release.

https://github.com/elixir-sqlite/ecto_sqlite3/releases/tag/v0.20.0

[aseigo]

Hm.. are you sure? #167 is still showing as not merged, and I don't see the TypeExtension module in the main git repo, nor in v0.20?

[warmwaffles]

Oh 🤦🏻 I thought I merged that one

[aseigo]

hehe. no worries, it happens. thanks for your efforts on this library! :)

[warmwaffles]

Deployed under v0.21.0