feat: add HampelFilter, robust outlier replacement for streams - #7598
Open
alxkm wants to merge 1 commit into
Open
feat: add HampelFilter, robust outlier replacement for streams#7598alxkm wants to merge 1 commit into
alxkm wants to merge 1 commit into
Conversation
Co-authored-by: Oleksandr Klymenko <19151554+alxkm@users.noreply.github.com> Signed-off-by: alxkm <19151554+alxkm@users.noreply.github.com>
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #7598 +/- ##
============================================
+ Coverage 80.83% 80.91% +0.08%
- Complexity 7624 7680 +56
============================================
Files 820 822 +2
Lines 24379 24490 +111
Branches 4783 4800 +17
============================================
+ Hits 19706 19817 +111
Misses 3907 3907
Partials 766 766 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
alxkm
marked this pull request as ready for review
September 8, 2026 17:56
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds the Hampel filter (Hampel identifier), which replaces the samples of a signal that look like outliers and leaves every other sample untouched.
MedianFiltercomes with it: the Hampel filter keeps one as its sliding window and reads the sorted contents through its package-privatecopySortedWindow, so the two cannot be separated.For each sample the filter computes two robust statistics over the window ending at that sample -- the median, and the median absolute deviation
MAD = median(|x_i - median|). The MAD is rescaled by 1.4826, the factor that makes it match the standard deviation of normally distributed data, and the sample is flagged whenFlagged samples are replaced by the window median, the rest pass through unchanged. Both statistics are robust, so a spike cannot inflate the very yardstick it is measured against -- which is what happens with a mean and a standard deviation instead. The window is causal and includes the sample being judged, so there is no delay, and the raw sample rather than its replacement enters the window, which keeps the statistics honest.
A threshold of 3 is the usual starting point, but the false positive rate depends just as much on the window size, because the MAD of a few samples is itself a noisy estimate of the spread. On clean Gaussian noise at threshold 3 the filter flags roughly 7% of samples with a window of 5, 3% with 11 and 0.5% with 101, approaching the 0.3% of a perfect sigma. The Javadoc says so, and
MedianFiltercovers the plain variant that rewrites every sample.Each sample costs O(w) time and nothing is allocated after construction.
HampelFilterTestcovers 25 cases andMedianFilterTest23. Among them: the MAD is checked against a brute force computation over several window sizes, an isolated spike is flagged and replaced by the local median, a clean ramp is left alone because the MAD grows with the slope, a zero MAD makes the identifier maximally strict, anddetectOutliersreports flags without altering the signal.Checklist
clang-format -i --style=file path/to/your/file.java