Skip to content

Stabilize certificate rollback partition test - #8132

Open
Amaury Chamayou (achamayou) wants to merge 1 commit into
mainfrom
achamayou/partitions-commit-sync-timeout
Open

Stabilize certificate rollback partition test#8132
Amaury Chamayou (achamayou) wants to merge 1 commit into
mainfrom
achamayou/partitions-commit-sync-timeout

Conversation

@achamayou

Copy link
Copy Markdown
Member

Summary

Use the test network's election-derived timeout while waiting for commit levels to converge after healing the partition in test_rolled_back_node_certificate.

This is deliberately scoped to this call site. The generic 3-second default remains unchanged for tests that do not intentionally tear down node-to-node connectivity.

Why

The first attempt of CI run 31189649906 failed after the certificate-renewal partition was removed:

  • node 0 rolled back its conflicting view-6 suffix and stored the view-7 suffix through 7.96;
  • node 0 reported 7.94 committed while nodes 1 and 3 reported 7.96 for the full 3-second wait_for_node_commit_sync() default;
  • the network's configured election timeout was 4 seconds;
  • just after that 3-second wait failed, node 0 reopened node-to-node connections, started view 8, and became leader with 7.96 in its log.

Deleting the iptables rules and reaching primary unanimity do not guarantee that node-to-node transport has recovered immediately. This is the same class of timing behaviour described in #7687 and handled elsewhere by #7601.

The preceding wait_for_commit() in this test already uses network.election_duration * 4. Applying the same bound to commit synchronization gives transport reconnection and any resulting election time to complete, without adding an unbounded wait or weakening assertions.

Attempt 2 passed unchanged, including the partitions step, confirming that the original failure was intermittent rather than a deterministic consensus regression.

Validation

  • scripts/ci-checks.sh
  • cmake -S . -B build -GNinja -DCMAKE_BUILD_TYPE=Debug
  • cmake --build build
  • ./tests.sh --timeout 360 --output-on-failure -L partitions -C partitions (passed in a user network namespace with NET_ADMIN)

Copilot AI lite review requested due to automatic review settings August 7, 2026 17:16
@achamayou
Amaury Chamayou (achamayou) requested a review from a team as a code owner August 7, 2026 17:16

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR stabilizes the test_rolled_back_node_certificate partitions test by aligning the post-partition commit-synchronization wait with the test network’s election-derived timing, reducing intermittent CI failures when node-to-node transport recovery lags behind primary unanimity.

Changes:

  • Extend wait_for_node_commit_sync() timeout at the rollback verification point to network.election_duration * 4 (instead of the 3s default).

Custom instructions used:

  • .github/copilot-instructions.md
  • .github/instructions/reviewing.instructions.md

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants