Skip to content

HDDS-10315. Speed up TestContainerReplication#10858

Open
cychiu8 wants to merge 4 commits into
apache:masterfrom
cychiu8:HDDS-10315
Open

HDDS-10315. Speed up TestContainerReplication#10858
cychiu8 wants to merge 4 commits into
apache:masterfrom
cychiu8:HDDS-10315

Conversation

@cychiu8

@cychiu8 cychiu8 commented Jul 24, 2026

Copy link
Copy Markdown
Contributor

What changes were proposed in this pull request?

Refactors the TestContainerReplication integration test to reduce runtime and duplication.

Reduce runtime:

  • The legacyEnabled parameter is no longer used, so drop the legacyEnabled parameter dimension:
    • createConfiguration(boolean enableLegacy) becomes createConfiguration().
    • containerReplicationArguments() now yields (policyClass) instead of (policyClass, legacyEnabled), halving the parameterized runs.
    • Renamed the test from testContainerReplication to testRatisContainerReReplicationAfterDatanodeShutdown and added a Javadoc describing what it verifies.

Cleanup:

  • Both createTestData overloads reuse the existing TestDataUtil.createVolumeAndBucket(...) helper instead of hand-rolled volume/bucket setup.
  • Replaced OzoneClientFactory.getRpcClient(conf) with cluster.newClient().
  • Replaced the one-off newCluster helper with the inline MiniOzoneCluster builder.
  • Removed lookupKey(...) helper and folded into the existing lookupKeyFirstLocation(...). Dropped unused imports and a stale TODO.
  • Simplify testECContainerReplication datanode setup.
  • fixes a Overiding → Overriding typo.

Original running time (223.7 s)

[INFO] -------------------------------------------------------
[INFO] Running org.apache.hadoop.ozone.container.TestContainerReplication
[INFO] Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 223.7 s -- in org.apache.hadoop.ozone.container.TestContainerReplication
[INFO] org.apache.hadoop.ozone.container.TestContainerReplication.testECContainerReplication -- Time elapsed: 40.69 s
[INFO] org.apache.hadoop.ozone.container.TestContainerReplication.testImportedContainerIsClosed -- Time elapsed: 32.48 s
[INFO] org.apache.hadoop.ozone.container.TestContainerReplication.testContainerReplication(String, boolean)[1] -- Time elapsed: 24.52 s
[INFO] org.apache.hadoop.ozone.container.TestContainerReplication.testContainerReplication(String, boolean)[2] -- Time elapsed: 24.35 s
[INFO] org.apache.hadoop.ozone.container.TestContainerReplication.testContainerReplication(String, boolean)[3] -- Time elapsed: 25.54 s
[INFO] org.apache.hadoop.ozone.container.TestContainerReplication.testContainerReplication(String, boolean)[4] -- Time elapsed: 24.42 s
[INFO] org.apache.hadoop.ozone.container.TestContainerReplication.testContainerReplication(String, boolean)[5] -- Time elapsed: 24.91 s
[INFO] org.apache.hadoop.ozone.container.TestContainerReplication.testContainerReplication(String, boolean)[6] -- Time elapsed: 26.51 s
[INFO] 
[INFO] Results:
[INFO] 
[INFO] Tests run: 8, Failures: 0, Errors: 0, Skipped: 0
[INFO] 
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  04:00 min
[INFO] Finished at: 2026-07-23T10:30:16+09:00
[INFO] ------------------------------------------------------------------------

Running time After Refactor (140.8s): 82.9 s faster

[INFO] Running org.apache.hadoop.ozone.container.TestContainerReplication
[INFO] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 140.8 s -- in org.apache.hadoop.ozone.container.TestContainerReplication
[INFO] org.apache.hadoop.ozone.container.TestContainerReplication.testECContainerReplication -- Time elapsed: 35.28 s
[INFO] org.apache.hadoop.ozone.container.TestContainerReplication.testImportedContainerIsClosed -- Time elapsed: 31.83 s
[INFO] org.apache.hadoop.ozone.container.TestContainerReplication.testRatisContainerReReplicationAfterDatanodeShutdown(String)[1] -- Time elapsed: 24.40 s
[INFO] org.apache.hadoop.ozone.container.TestContainerReplication.testRatisContainerReReplicationAfterDatanodeShutdown(String)[2] -- Time elapsed: 25.11 s
[INFO] org.apache.hadoop.ozone.container.TestContainerReplication.testRatisContainerReReplicationAfterDatanodeShutdown(String)[3] -- Time elapsed: 24.11 s
[INFO] 
[INFO] Results:
[INFO] 
[INFO] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0
[INFO] 
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-10315

How was this patch tested?

Ran the test suite:

mvn -pl :ozone-integration-test test \
-Dtest='org.apache.hadoop.ozone.container.TestContainerReplication' \
-Dsurefire.reportFormat=plain \
-Dsurefire.useFile=false \
-DskipShade -DskipRecon

Result: all tests passed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant