Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rabbitmq 'failed to merge schema: bad cookie' #378

Open
guzzijones opened this issue Nov 3, 2023 · 3 comments
Open

rabbitmq 'failed to merge schema: bad cookie' #378

guzzijones opened this issue Nov 3, 2023 · 3 comments

Comments

@guzzijones
Copy link
Contributor

I was getting this error in rabbitmq pods greater than 0.

Failed to merge schema: Bad cookie in table definition rabbit_user_permission rabbitmq

I ended up finding this solution

Step1: Downscale the statefulset it will not delete the PVC.

kubectl scale statefulsets stackstorm-ha-rabbitmq --replicas=1

Step2: Access the RabbitMQ Pod.

kubectl exec -it stackstorm-ha-rabbitmq-0 -- /bin/bash

Step3: Reset the cluster

rabbitmqctl stop_app
rabbitmqctl force_boot

Step4: Rescale the statefulset

  kubectl scale statefulsets stackstorm-ha-rabbitmq --namespace teps-rabbitmq --replicas=3
@cognifloyd
Copy link
Member

Is there something we can change in this chart to help prevent/resolve this issue? Should this just be added to documentation somewhere? Or something changed in the values passed to the rabbitmq subchart?

@ericssonanders
Copy link

This was a really hard one for me. I did also have some success with rescaling to 1, allow for the pods to terminate than scale up again. I guess the mnesia db:s was in a split-brain context for me. I ended up recreating the cluster. But, obviously this is nothing you would want happening in a production scenario. Using OpenShift.

@guzzijones
Copy link
Contributor Author

It looks like is the solution. I am not sure if we can override these settings from bitnami, though.

Combining podManagementPolicy: parallel with a simple "single node" readiness probe mentioned in the docs would be a much safer solution that does not abuse a feature created as a last resort fo

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants