Skip to content

Parallelize collection tests into 4 shards#5041

Open
kenyonj wants to merge 2 commits intomainfrom
parallelize-collection-tests
Open

Parallelize collection tests into 4 shards#5041
kenyonj wants to merge 2 commits intomainfrom
parallelize-collection-tests

Conversation

@kenyonj
Copy link
Contributor

@kenyonj kenyonj commented Feb 13, 2026

Summary

  • Split collection tests across 4 parallel GitHub Actions matrix jobs
  • Each shard tests ~25 collections instead of all ~100
  • Reduces wall-clock time for collection testing by ~4x

Changes

  • test/collections_test_helper.rb: Add shard_collections function that uses modular arithmetic on alphabetically sorted collections. Modified collections to apply sharding when COLLECTION_SHARD and COLLECTION_TOTAL_SHARDS env vars are set
  • .github/workflows/test.yml: Replace single collections matrix entry with 4 sharded entries (shard 0-3). Pass shard env vars to the test step

How sharding works

Collections are sorted alphabetically for deterministic assignment. Each collection is assigned to shard index % total_shards. For example with 4 shards:

  • Shard 0: collections at indices 0, 4, 8, 12, ...
  • Shard 1: collections at indices 1, 5, 9, 13, ...
  • etc.

Notes

  • topics and all jobs are unaffected
  • The shard count can be adjusted by modifying the matrix entries
  • When shard env vars are not set (e.g., local development), all collections are tested

Split collection tests across 4 parallel matrix jobs using modular
arithmetic on alphabetically sorted collection names. Each shard tests
roughly 25 collections instead of all ~100, reducing wall-clock time
by ~4x for collection testing.

Sharding is controlled by COLLECTION_SHARD and COLLECTION_TOTAL_SHARDS
env vars, making it easy to adjust the shard count in the future.
@kenyonj kenyonj requested a review from a team as a code owner February 13, 2026 20:39
Copilot AI review requested due to automatic review settings February 13, 2026 20:39
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR parallelizes collection tests across 4 shards to reduce wall-clock time by approximately 4x. The sharding is implemented using modular arithmetic on alphabetically sorted collections, where each of the 4 GitHub Actions matrix jobs tests roughly 25% of the ~100 collections.

Changes:

  • Added sharding logic to split collection tests across parallel jobs
  • Modified GitHub Actions workflow to create 4 separate collection test jobs
  • Implemented deterministic shard assignment using alphabetical sorting and modulo operation

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File Description
test/collections_test_helper.rb Added shard_collections function to filter collections based on shard environment variables
.github/workflows/test.yml Replaced single collections matrix entry with 4 sharded entries and passed shard environment variables

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

shard = ENV["COLLECTION_SHARD"]&.to_i
total_shards = ENV["COLLECTION_TOTAL_SHARDS"]&.to_i

return all_collections unless shard && total_shards && total_shards > 1
Copy link

Copilot AI Feb 13, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The condition shard && total_shards && total_shards > 1 will fail when shard is 0 because 0 is falsy in Ruby. This means shard 0 (the first shard in the matrix) will incorrectly test all collections instead of just every 4th collection starting at index 0. The condition should check for nil explicitly instead: !shard.nil? && !total_shards.nil? && total_shards > 1.

Copilot uses AI. Check for mistakes.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 198d465 — switched to explicit nil checks: !shard.nil? && !total_shards.nil? && total_shards > 1. This also avoids the falsy-zero bug for shard 0. Thanks for the catch!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant