Skip to content

Sync in the Data Entitties is not happening after the cross namespace recovery  #1682

@mavenzer

Description

@mavenzer

To test the resilience of our deployment patterns we have deployed ODD in many namespaces in Kube Cluster. So to validate the recovery from one namespace to another (say from Dev to Production) we have used cronjob as the backup mechanism. Since we are using Bitnami Postgres as the database its pretty easy to write the cronjob from the database and test it out.

PGPASSWORD=$POSTGRES_PASSWORD pg_dumpall -U $POSTGRES_USER -h $POSTGRES_HOST -p $POSTGRES_PORT > /backups/all-backups-$TIMESTAMP.sql

We can easily do the recovery in the same namespace but the problem starts with the different namesapce(From DEV to PROD) i.e. we wanted to recover the data from the dev namespace to prod namespace using the .sql file from the dev namespace.

Steps which we have followed for the recovery :

  • Scaled down the ODD collector and ODD Platform
  • Copied the .sql file from DEV to PROD
  • Recovered the data using PSQL shell.
  • Updated the data-source in the collector
UPDATE data_source
SET oddrn = REPLACE(oddrn, 'odd-postgresql.dev-cluster.svc.cluster.local', 'odd-postgresql.prod-cluster.svc.cluster.local')
WHERE namespace_id = 1;
  • Added the same token in the Collector POD which was in the Dev Namespace.
  • Started the ODD platform and ODD collector

So what we have found is exact 2x number of data entities in the ODD platform, because there are two copies of data entities one of the DEV NS and one for the PROD ns. And the data is there for the dev ns not for the Prod ns.

data-prod
data-dev

Metadata

Metadata

Assignees

Labels

kind: bugSomething isn't working

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions