Multiplayer/Ingress force retool as backend

[davidaparicio]

Hello!

I deployed the latest stable version (3.114.2-stable) with the penultimate version of the HELM Chart (retool-6.2.14).

But the chart can't permit to redirect the path /api/multiplayer/ to the serviceName: retool-multiplayer-ws.

In fact, the lines 40/44 have the full name of the application so it's not possible to overwrite to another value (in order to target the multiplayer webservice). :-/

Kind regards,
David

[davidaparicio]

An example from an opensource helm chart:
https://github.com/codecentric/helm-charts/blob/master/charts/keycloak/values.yaml#L308

[davidaparicio]

I retried today without sucess..
I deployed the latest stable version (3.114.5-stable) with the ultimate version of the HELM Chart (retool-6.3.0).

[jjlgao]

Hi David,

Apologies for the late response here.

Could you clarify what error you're getting? I believe our chart does redirect /api/multiplayer/ to the service retool-multiplayer-ws, I take it Helm isn't letting this happen. Is there anything that's outputted when you're trying this that could help narrow down the cause?

[davidaparicio]

Hi @jjlgao,

Yes, I can share you the current overview on our retool namespace

❯ k get pods,deployments,services,ingress,secrets,configmaps -n retool
NAME                                          READY   STATUS    RESTARTS   AGE
pod/retool-9yy-8xx                            1/1     Running   0          150m
pod/retool-9yy-rxx                            1/1     Running   0          152m
pod/retool-code-executor-7d56-9bn4t           1/1     Running   0          152m
pod/retool-jobs-runner-9b9c-tcrn6             1/1     Running   0          152m
pod/retool-multiplayer-ws-6468-v78m9          1/1     Running   0          139m
pod/retool-workflow-backend-b9bc8-b7d57       1/1     Running   0          152m
pod/retool-workflow-worker-698b-hpvll         1/1     Running   0          152m

NAME                                      READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/retool                    2/2     2            2           261d
deployment.apps/retool-code-executor      1/1     1            1           261d
deployment.apps/retool-jobs-runner        1/1     1            1           261d
deployment.apps/retool-multiplayer-ws     1/1     1            1           139m
deployment.apps/retool-workflow-backend   1/1     1            1           261d
deployment.apps/retool-workflow-worker    1/1     1            1           261d

NAME                              TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)             AGE
service/retool                    ClusterIP   10.xx.xx.xx     <none>        3000/TCP            261d
service/retool-code-executor      ClusterIP   10.xx.xx.xx     <none>        80/TCP              261d
service/retool-multiplayer-ws     ClusterIP   10.xx.xx.xx     <none>        80/TCP              139m
service/retool-workflow-backend   ClusterIP   10.xx.xx.xx     <none>        80/TCP              261d
service/retool-workflow-worker    ClusterIP   10.xx.xx.xx     <none>        3005/TCP,9090/TCP   261d

NAME                               CLASS    HOSTS              ADDRESS         PORTS     AGE
ingress.networking.k8s.io/retool   <none>   retool.domain.com  1xx.1xx.1xx.xx   80, 443   261d

NAME                                   TYPE                 DATA   AGE
secret/letsencrypt-prod                Opaque               1      262d
secret/retool                          Opaque               5      262d
secret/retool-tls                      kubernetes.io/tls    2      134m
secret/retool.tls                      kubernetes.io/tls    2      262d
secret/sh.helm.release.v1.retool.v13   helm.sh/release.v1   1      122m
secret/sh.helm.release.v1.retool.v7    helm.sh/release.v1   1      84d

NAME                         DATA   AGE
configmap/kube-root-ca.crt   1      262d

❯ k describe ingress.networking.k8s.io/retool -n retool
Name:             retool
Labels:           app.kubernetes.io/managed-by=Helm
                  helm.sh/chart=retool-6.3.0
Namespace:        retool
Address:          1xx.1xx.1xx.xx
Ingress Class:    <none>
Default backend:  <default>
TLS:
  retool.tls terminates retool.domain.com
Rules:
  Host              Path  Backends
  ----              ----  --------
  retool.domain.com /   retool:3000 (1xx.1xx.1xx.xxx:3000,1xx.1xx.1xx.xxx:3000)

As you can see, there is only one rule..
If I added /api/multiplayer in the line#151, the redirection created was to the retool backend and not the multiplayer (I think, Ican't change it). I followed this values.yaml file.

I need something like this one

  - path: /api/multiplayer/
    pathType: Prefix
    backend:
      service:
        name: retool-multiplayer-ws
        port:
          number: 80

Did I make a mistake somewhere?
Because your Google Docs "Retool multiplayer for self-hosted (beta)" doesn't present the installation for Helm..
(only requirements>Web proxy)

Thank you very much in advance for your help!

Kind regards,
David

[jjlgao]

Thanks for the extra context. I apologize, I made a mistake looking through our chart, and you are correct that our helm chart does not currently set up this extra ingress rule or have a way to do so. I'm checking internally to get an answer for the best practice way to enable multiplayer; in the meantime, does multiplayer work at all if this endpoint is routed to the Retool backend instead?

[davidaparicio]

You're welcome! Yes, exactly, the current Helm chart doesn't handle it :)

Manually, I created this one:

❯ k describe ingress.networking.k8s.io/retool-multiplayer-ingress -n retool
Name:             retool-multiplayer-ingress
Labels:           <none>
Namespace:        retool
Address:          1xx.1xx.1xx.1xx
Ingress Class:    nginx
Default backend:  <default>
TLS:
  retool-tls terminates retool.domain.com
Rules:
  Host                      Path  Backends
  ----                      ----  --------
  retool.domain.com
                            /api/multiplayer/   retool-multiplayer-ws:80 (1xx.1xx.1xx.1xx:3001)
Annotations:                cert-manager.io/cluster-issuer: cluster-issuer
                            nginx.ingress.kubernetes.io/ingress.class: nginx
                            nginx.ingress.kubernetes.io/proxy-body-size: 8m
                            nginx.ingress.kubernetes.io/proxy-http-version: 1.1
                            nginx.ingress.kubernetes.io/proxy-request-buffering: off
                            nginx.ingress.kubernetes.io/proxy-set-headers: ingress-multiplayer-headers
                            nginx.ingress.kubernetes.io/rewrite-target: /
                            nginx.ingress.kubernetes.io/tls-acme: true

When I went to the endpoint /api/multiplayer with my browser, I got "Route not found for /" without the rewrite-target.
Now, with this annotation, I have

 curl https://retool.domain.com/api/multiplayer
{"success":false,"message":"Authentication failure. Missing access token","triggerOauth2SSOLoginAutomatically":false,"triggerSamlLoginAutomatically":false}

I asked my team to test the multiplayer again, but it still doesn't work.

[davidaparicio]

Hello @jjlgao, have some update regarding this issue ? (:

[jjlgao]

Hi David,

Apologies for the late update, it took some time to gather information internally. I confirmed that we don't have a way to route multiplayer requests in this helm chart, and in the past we've expected folks to set up the ingress themselves to do so, independently of our helm chart.

With that being said, we'll be making a change to make this possible this week or next week, to make it part of our helm chart so you can enable it without having to set up an independent ingress.

Thanks,
Jason

[jjlgao]

Hi David,

We're working on the change in PR #205. We actually don't have the greatest test coverage on the ingress component of our helm chart, so if you're able to help us test this by pulling the change and referencing the local helm chart, that would be much appreciated.

Separately, while making this change I discovered that we actually have an extraPaths section in the ingress that you may be able to use in place of the PR changes: https://github.com/tryretool/retool-helm/blob/main/charts/retool/values.yaml#L164

Regardless of whether this works for you, we'll likely move forward with the PR to make sure others in the future who want to enable multiplayer don't need to add custom ingress snippets to properly enable it.

You may be able to add a section like this in your ingress section of your values.yaml file:

    extraPaths:
      - path: /api/multiplayer
        backend:
          service:
            name: retool-multiplayer-ws
            port:
              name: http-server
        pathType: ImplementationSpecific

Apologies for the sparse responses here, it took me some time to gather this information. Could you check these options and let me know if either the PR changes or the extraPaths configuration will work for you?

Thank you!
Jason

[jjlgao]

Hey David! Just a heads up that I'll be out for a few weeks, so expect responses to be slow. If you're able to help test, let me know, otherwise I'm happy to merge the PR so that you can use it!

[davidaparicio]

Hello @jjlgao,

I just update my values.yaml with your recommendation.
I did something like this, also, a month ago.

    extraPaths:
      - path: /api/multiplayer
        backend:
          service:
            name: retool-multiplayer-ws
            port:
              name: http-server
        pathType: Prefix

But, you got this error message:

Error: UPGRADE FAILED: release retool failed, and has been rolled back due to atomic being set: cannot patch "retool-multiplayer-ws" with kind Deployment: Deployment.apps "retool-multiplayer-ws" is invalid: spec.template.spec.containers[0].env[13].name: Invalid value: "0": a valid environment variable name must consist of alphabetic characters, digits, '_', '-', or '.', and must not start with a digit (e.g. 'my.env-name',  or 'MY_ENV.NAME',  or 'MyEnvName1', regex used for validation is '[-._a-zA-Z][-._a-zA-Z0-9]*')

Have you an idea to fix this ?
Maybe your PR #205 can resolve the issue 😉

Thank you!
David

[jjlgao]

That error is new to me! If all you did was change the extraPaths, then it seems like it's possible that it wasn't inserted correctly (not necessarily something you did, it could be an issue with the way the variable is inserted via Helm).

Let me merge #205 and see if that will help you.

[jjlgao]

Please try v6.3.6 and let me know if it works for you!

[davidaparicio]

I can't install the version 6.3.6..

Error: chart "retool" version "6.3.6" not found in https://charts.retool.com repository

And when I go to artifacthub.io, I saw only 6.3.5.

[davidaparicio]

Now, I have another error, when I retried.
But, It's unrelated with the precedent one.

Error: UPGRADE FAILED: template: retool/templates/_helpers.tpl:14:14: executing "retool.fullname" at <.Values.fullnameOverride>: nil pointer evaluating interface {}.fullnameOverride

Even with these variables in the values.yaml

nameOverride: ""
fullnameOverride: ""

The error persists..

And, this issue doesn't appear on 6.3.0 or in the latest 6.3.5. Have you an idea @jjlgao ?

[jjlgao]

Regarding the helm chart version, try again -- it looks like everything's in place, but it might take some time for our cache to expire.

Regarding the name override, that's unfamiliar to me. Were you able to downgrade back to 6.3.5 successfully?

[davidaparicio]

Can you try this on your side, if the error appears or not ?

helm upgrade --install retool retool \
 -f values.yaml \
 --repo https://charts.retool.com \
 --namespace retool \
 --version "6.3.6" \
 --atomic \
 --dry-run

Because with 6.3.5, it's working properly 👍
But not, with 6.3.6

As you can see on my screenshot

[jjlgao]

Apologies, I see the issue, there is a legitimate fix to make. It's not reproducible in our setups, so it slipped through testing. Will push up a new version, sorry about that!

[davidaparicio]

Okay, great. I will setup the 6.3.7 👍
In fact, we want to play and use the multiplayer asap =D

[davidaparicio]

And with the version 6.3.5, when I tried to set the domain environment variable,

multiplayer:
  enabled: true
  replicaCount: 1
  resources:
    requests:
      cpu: "200m"
      memory: "256Mi"
    limits:
      memory: "2048Mi"
  environmentVariables:
    - name: MAIN_DOMAIN
      value: retool.domain.com

I got this error message :/
Pointing this code line for the 13th element

Error: UPGRADE FAILED: release retool failed, and has been rolled back due to atomic being set: cannot patch "retool-multiplayer-ws" with kind Deployment: Deployment.apps "retool-multiplayer-ws" is invalid: spec.template.spec.containers[0].env[13].name: Invalid value: "0": a valid environment variable name must consist of alphabetic characters, digits, '_', '-', or '.', and must not start with a digit (e.g. 'my.env-name',  or 'MY_ENV.NAME',  or 'MyEnvName1', regex used for validation is '[-._a-zA-Z][-._a-zA-Z0-9]*')

In order to fix it, you need to change the example in values.yaml.

  # Set environment variables for multiplayer pods, e.g. defining which origin to use
  # environmentVariables:
  #   - name: MAIN_DOMAIN
  #     value: retool.foo.com

Instead

  # Set environment variables for multiplayer pods, e.g. defining which origin to use
  environmentVariables:
    MAIN_DOMAIN: "retool.domain.com"
    OTHER_VALUE: "42"

I will do a PR 👍

[jjlgao]

Hey @davidaparicio , apologies, I totally forgot to finish merging my follow up. The new chart version will be 6.4.1, it has been published.

[davidaparicio]

I removed my manual ingress, I updated the cluster to the latest stable release.
I deployed via the HELM chart 6.4.3 (13 Mar, 2025), the multiplayer works properly with manual actions 👍
Thanks a lot

PS: the version 6.4.4 is not currently on the Helm repository, cf Artifacthub

[davidaparicio]

I confirmed, with the latest HELM chart v6.4.4, it's also working.
Thanks a lot @jjlgao, great job!!