Track events filtered out by MQL/subscription queries in mantis-publish-core - #865
Draft
james-lubin wants to merge 2 commits into
Draft
Track events filtered out by MQL/subscription queries in mantis-publish-core#865james-lubin wants to merge 2 commits into
james-lubin wants to merge 2 commits into
Conversation
Client-publish-side (MRE) counterpart to the mqlEventsFilteredOut counter added on the Mantis source-job side (Netflix-internal PR #949). Tagged by stream and a cardinality-limited subscriptionId (CardinalityLimiters.mostFrequent(50), matching the limit used by MqlEvalStage), with one Counter cached per limited id. Access to the limiter is synchronized since it is backed by a non-thread-safe LinkedHashMap and StreamMetrics is shared across concurrently-processed events.
EventProcessor.process now increments StreamMetrics#getMantisEventsFilteredCounter for each subscription whose Subscription.matches(event) returns false. Matching subscriptions and the matches()-throws path (which still only increments mantisQueryFailed) are unaffected.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What & why
Adds a
mantisEventsFilteredSpectator counter tomantis-publish-core, tracking events that were evaluated against a subscription's query but did not match (and were therefore not published). This is the client-publish-side ("MRE") counterpart to a Netflix-internal fix that added an equivalentmqlEventsFilteredOutcounter on the Mantis source-job side.Without this, operators had no way to distinguish "the query genuinely has low match volume" from "events are being silently filtered out" on the publish-client side —
mantisEventsDropped/mantisEventsSkippeddon't cover this case, since the event is neither dropped from the queue nor skipped due to no active subscriptions; it's evaluated and simply doesn't match.What changed
StreamMetrics#getMantisEventsFilteredCounter(String subscriptionId): a new counter taggedstream+ a cardinality-limitedsubscriptionId(CardinalityLimiters.mostFrequent(50)), with counters cached per limited id. Access to the limiter is synchronized sinceCardinalityLimiters.mostFrequentis backed by a non-thread-safeLinkedHashMapandStreamMetricsis shared across concurrently-processed events.EventProcessor#process(...): increments the new counter for each subscription whosematches(event)returnsfalse. The matching branch and the exception-catch branch (which still only incrementsmantisQueryFailed) are unchanged.Test plan
./gradlew :mantis-publish:mantis-publish-core:buildpasses (compile, tests, license checks)--others--bucket, and theEventProcessornon-match/match/throws branches