Skip to content

Own benchmark workers and reject invalid results - #9

Closed
jeregrine wants to merge 1 commit into
mainfrom
fix/benchmark-correctness
Closed

Own benchmark workers and reject invalid results#9
jeregrine wants to merge 1 commit into
mainfrom
fix/benchmark-correctness

Conversation

@jeregrine

Copy link
Copy Markdown
Member

Problem

Local benchmark workers sleep forever without a scenario owner. Stopping Group does not stop those workers, so later scenarios inherit an increasingly large process population. Operation results are ignored, and event-delivery timeouts merely print a warning before reporting throughput for all expected events.

Fix

Give each scenario a worker supervisor and synchronously stop its workers on success or failure. Provision workers before releasing the measured workload so supervisor startup does not become a serial throughput bottleneck.

Require successful operation results, detect worker exits and timeouts, verify registry and membership datasets, and validate each read sample outside its timed interval. Event delivery requires the exact expected registrations, rejecting missing, duplicate, or incorrect events instead of reporting partial work as success.

Add a standalone harness regression suite covering resource ownership, failure paths, cluster-scoped verification, and event validation. Remove monitor subscriptions before teardown so Registry shutdown does not require swallowing exit signals.

Supporting information

Write timings now exclude worker provisioning; event timings include exact event validation. Existing throughput figures are not directly comparable: establish a fresh baseline with the corrected harness.

This change affects the local benchmark harness only. It adds no runtime dependencies and does not change Group's production implementation.

@jeregrine
jeregrine force-pushed the fix/benchmark-correctness branch from 6f4a750 to 102d755 Compare September 10, 2026 12:49
@jeregrine jeregrine closed this Sep 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant