Data collection: supply a UUID through environment variables

[Akarys42]

Components

Kord Extensions (Core)

Summary

Spinning up persistent filesystem storages can be quite complex - and costly - in highly containerized environments. A much easier object to persist are environment variable. As a matter of fact, this is already doable through Kubernetes by mapping a configmap to a folder, although this would allow to expand this (unintented) feature to other less feature complete container management tools.

Current Behaviour

The UUID is only gotten from the data folder.

Target Behaviour

UUID can also be supplied through an environment variable.

Implementation Tips

No response

[Akarys42]

Turning on readOnlyRootFilesystem is also good security practices in Kubernetes

[gdude2002]

The UUID actually comes from the server, it's just stored in a properties file right now - I've been thinking about this today and I think the obvious solution is to move it over to data adapters.

Though, they still need a rewrite, I'm not sure what order to do things in since I have no idea what migrating from the old system might look like?

[Akarys42]

I can't really think of a migration that wouldn't require end user input - maybe some parameter when configuring the data collection that contains the old UUID? I feel like it'll necessarily include "data loss" for quite a few users, but is it that big of a deal?

[gdude2002]

It's probably not a huge deal, especially for simple data like this, yeah. I can't imagine most bots would need to swap out their data adapters often or anything either, though the lack of portability there has always been a major downside.

I dunno, I'll think of something, I'm sure.

[Akarys42]

Having a migration system would be huge, if it doesn't already exist. I'm a huge fan of Django's and Alembic's designs. It can also be designed in a way where each module (and the core) can provide their own migration, so the end user doesn't even have to think about them. Though in this case it still wouldn't help that much since you have to introduce data into the system which only a developer can do.

[gdude2002]

Yeah, I'm not sure really, database migrations in general are still pretty tough on the JVM, and I feel like the data adapter system ultimately is probably too big to be just part of KordEx Core, but that's a plan to make later down the line probably

[gdude2002]

Data collection now uses storage units and data adapters as of Kord-Extensions/kord-extensions@ba7e250

This will migrate from the properties file (if it's present) and then delete it.

---

As for supplying the UUID early - the server will generate a new UUID if you supply one that isn't already in the database. I did this to make sure you couldn't re-use someone else's UUID if you happened to get your hands on one, as I figured they might be useful for support or identification down the line, but I'm not too sure if this is a realistic problem?

[Akarys42]

UUIDs are 128 bits long, so the odds of generating the same one is used / 128^2. For all applications, a random UUID is considered unique (assuming the random implementation isn't flawed).

[gdude2002]

I meant moreso if a user obtains someone else's UUID somehow

[Akarys42]

I don't see how you can prevent impersonation that way. It seems very unlikely someone would investvtime and effort into that, anyway.

[gdude2002]

Yeah, I guess you're probably right.

[gdude2002]

As of the latest build, a version 4 UUID may be supplied via the dataCollectionUUID system property or the DATA_COLLECTION_UUID env var. This will also stop the data collection system from trying to store any data.

Thanks for the discussion!

[gdude2002]

As this has been resolved without comment for two weeks, I'll close the discussion. Feel free to open a new one if you need anything else!