Upgrading to latest version?

[guidopetri]

I'm upgrading from a very old version of stringer, back when we only had one secret token.

To do that, I'm:

• dumping database to file
• starting the new image in Docker, connected to a new database
• (attempting to) eventually copy data from old db -> new db.

Unfortunately it seems like there's a few issues with starting the new image:

• /usr/local/bin/supercronic doesn't have +x and supervisor returns supervisor: couldn't exec /usr/local/bin/supercronic: ENOEXEC
• the add_api_key_to_user migration fails, because constantize now returns AddAPIKeyToUser but it is currently named AddApiKeyToUser
• AddNullFalseToAssociations migration fails, due to NoMethodError: undefined method 'id' for nil:NilClass
• potentially more, since this is as far as I've gotten.

I'm running this on a raspberry pi (arm64 edit: arm - see below), so I'm unable to use the latest dockerhub images, and have to build from the github branch myself. Any suggestions on how to fix these issues? For the API one, I've managed to circumvent it by just running s/Api/API in that file, but I'm not a ruby developer and don't know how to fix the null ID one.

Additionally, it seems like upgrading from the old encryption method to the new encryption method isn't very simple. I used to be able to dump the entire db and restore it, but it seems that now, with the different encryption, it must somehow be unencrypted and then re-encrypted. Is there support for this within the stringer app?

[guidopetri]

I just remembered there's an export/import function at the bottom of the main page. Guessing that's the intended way of migrating up? Except it doesn't keep starred/unread stories, it just keeps track of which feeds exist, which isn't ideal (I'd like to keep my starred/unreads).

Additionally, I tried resetting to approximately the time the dockerhub 0.2 tag image was created and this image seems to be working. So something in the ~163 commits since has changed the behavior of the app.

[guidopetri]

Another update: it seems the supercronic issue was due to a hardcoding of amd64 in the download, whereas I was building the image for arm. Switching to the arm version seems to have worked - without this, the feeds would not refresh at all.

This may be something worth exploring if there's ever any interest in an ARM/ARM64 build.

[mockdeep]

@guidopetri thanks for reporting in! I'm going to have a look at the migrations and will hopefully be able to get them fixed up today. At an initial glance, it looks like that error would happen if didn't have a user in the database, which is a little surprising if you are starting with data from a previous instance. I can add conditional checks for that, but I'd like to understand a little more about your data setup.

I'm not great with Docker, so it might take me some time to get that sorted out. If you've got any thoughts on it, I'd be happy to hear it. PRs would be helpful, too, even if only to update the docs.

Regarding the encryption, the migration should automatically handle that for you. If it doesn't, I'll do what I can to fix it.

[guidopetri]

At an initial glance, it looks like that error would happen if didn't have a user in the database, which is a little surprising if you are starting with data from a previous instance. I can add conditional checks for that, but I'd like to understand a little more about your data setup.

That's what I'd have expected, too, but it seems the behavior for some library used changed in the past ~160 commits - the exact same database worked just fine with the old version I linked above in my fork.

For my data setup: I have a Postgres DB running on Docker on a VPS, and stringer set up as a local user, a version from ~2 years ago almost exactly: 0b64dc0 . The postgres db is version 14.1(-alpine, if it matters). Do you want/need details on the tables themselves?

I'm migrating to a server at home, with Postgres 14.1(-alpine) running in Docker, and setting stringer up within Docker as well. I had experimented with this a bit before on a second VPS and it seemed like the migration from one db to another had to work like:

• copy schema from old-psql to new-psql
• drop database and recreate, set owner to stringer user
  • this was because somehow, it was trying to run db migrations on an already-migrated db, and this let the stringer container do whatever it needed
• launch stringer service in docker (empty db)
• stop stringer container
• copy over data from old-psql to new-psql
• start stringer container

It seems that the migrations step in the empty db is not necessary anymore, but I'm not confident about that. And obviously with the new encryption setup there is definitely a requirement for following the migrations with the data already in the db.

---

On Docker, I am happy to help out, though my time is a bit constrained atm. I'm still migrating other services from VPS -> home server and just found out that my home server has a 64 bit kernel with a 32 bit userland, which is... inconvenient, and I'll have to migrate stuff again and reinstall the OS :'( but I'd be happy to at least extend the docker image to support ARM/ARM64 as well. We could set it up so that a docker image (in each of the architectures) gets built and pushed to dockerhub, maybe?

---

Finally, on the encryption migration, I'm guessing the easiest way for me to run this is to set up the latest commit as a separate container and launching it pointing to the same db? (I believe this is equivalent to bringing the service down, doing git pull, and then bringing it back up on a vps) I'll give it a shot once I have some more time on my hands, and can report back if I run into any trouble.

[mockdeep]

Okay, I think I see what's going on with the migrations. I think rather than running the migrations on a new instance, you should be able to run rails db:schema:load to load the latest version of the schema. As a rule of thumb, if you have a database with existing data in it, you can run rails db:migrate to update the schema. If you're starting with a fresh database, you can run rails db:schema:load (or rails db:setup if you don't even have the database yet).

The problem with that, though, is that when you import the data, the encryption won't be updated, so you'd also need to manually run the commands from the migration. Since you do have data you want in your database, maybe instead you can do something like this:

1. dump the full database from you old instance (schema + data)
2. restore the full database into the new instance
3. run the migrations

Ideally, I think we'd have a setup where the data is stored outside the docker image so that you don't need to go through this process. It's a bit of a pain.

---

I'd be happy to at least extend the docker image to support ARM/ARM64 as well. We could set it up so that a docker image (in each of the architectures) gets built and pushed to dockerhub, maybe?

This would be very helpful! I'm pretty sure I've seen other requests for better ARM support, so I think others would benefit from it as well.

---

Finally, on the encryption migration, I'm guessing the easiest way for me to run this is to set up the latest commit as a separate container and launching it pointing to the same db?

That sounds about right. In principle, if you were on a VPS without Docker you should be able to simply git pull the latest main, set the new env variables, run rails db:migrate, and restart the app. Docker complicates things since the data is stored in a transient container.

[mockdeep]

Alright, I've updated the migrations so they should hopefully run smoothly now. I also opened a new issue about data exports, though that's probably a larger effort.

[guidopetri]

Thanks so much! I think we can close this issue and open up a new one for Docker/ARM then? If you want to, you can assign that to me.

For the migrations, I'll try launching a new container with the latest code pointing to the same db - my understanding is that the db migrations are performed automatically at the beginning (since I didn't do anything to start them last time) so hopefully it Just Works (tm).

(fwiw, the postgres data is external to the stringer container)

[mockdeep]

Okay, I've made a new issue for the Docker stuff. If anything else comes up, please let me know!