NuoDB admin does not start up with "Shutting down server due to mismatching initial membership" during upgrade to NuoDB 4.0.7

[adriansuarez]

Describe the bug
The bootstrapServers label and nuodb.com/bootstrap-servers annotation, which allow an initial membership to be defined containing an arbitrary number of members, breaks upgrade from deployments prior to 2.4.0 of nuodb-helm-charts or from NuoDB releases prior to 4.0.7 to 4.0.7 with 2.4.0 of the nuodb-helm-charts.

Context
NuoDB Version: 4.0.6 to 4.0.7
Helm Charts Version: 2.4.0
Kubernetes Version: any
Environment: any

To Reproduce
Steps to reproduce the behavior:

1. Create domain with NuoDB 4.0.7 and helm charts 2.3.1
2. upgrade to newer helm charts

Expected behavior
NuoDB upgrades as expected

[adriansuarez]

The workaround is to explicitly set admin.bootstrapServer=0 in any cluster that did not define it when it was bootstrapped.

[adriansuarez]

@jleslie85 , @vegichan : Proposed release note:

The admin.bootstrapServers property introduced in 2.4.0 allows an initial membership of an arbitrary size to be defined. It is not possible to upgrade from deployments that do not define admin.bootstrapServers to ones that do define it. Since admin.bootstrapServers=1 is the default setting starting with release 2.4.0, admin.bootstrapServers=0 has to be explicitly set when upgrading from deployments that do not define the property, which includes all deployments created on releases prior to 2.4.0.

[mkysel]

The error message that can be seen in the admin log is:

2020-08-28T18:39:30.311+0000 WARN admin-ycd7ry-nuodb-cluster0-0 com.nuodb.nagent.AgentMain main shutdown server evictLocalProcesses=false reason: Shutting down server due to mismatching initial membership.
Durable membership:
{
  "admin-ycd7ry-nuodb-cluster0-0" : {
    "transport" : "<>",
    "version" : "3-18"
  }
}
Membership from config:
{
  "admin-ycd7ry-nuodb-cluster0-0" : {
    "transport" : "<>",
    "version" : "0:10000"
  }
}
2020-08-28T18:39:30.311+0000 INFO  com.nuodb.runner.Runner main Shutting down server...

Notice the mismatched "version"

[github-actions]

This issue is stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed in 7 days