Skip to content

Outlier-search algorithm based on issue #385#388

Merged
tomeichlersmith merged 11 commits intomainfrom
outlier_scan
Apr 1, 2026
Merged

Outlier-search algorithm based on issue #385#388
tomeichlersmith merged 11 commits intomainfrom
outlier_scan

Conversation

@blazeiro
Copy link
Copy Markdown
Contributor

@blazeiro blazeiro commented Apr 1, 2026

Following the ADC irregularities for charge-injections found in Lund for some boards, I created an algorithm that scans parameter-time-scan data for outliers. The script assumes the irregularities to be less frequent than the "normal" results, and hence requires at least 10 samples per time-point for comparison. Using this data, the algorithm calculates the ADC mode for each time-point, as seen in the figure below.

mode

Next, the script checks the ADC codes above and below the mode, one by one. If a gap of 2 or more is found (2+ ADC codes are not contained by any sample at the specific time-point), the points that follow are marked as an "outlier" or "potential outlier" cluster. Gaps of 1 are ignored to account for natural fluctuations or irregularities that can be found in any board, but this can be adjusted in the code. The resulting analysis can then be saved as a csv, with outlier frequency noted per sample. This can be used for quick board-evaluation, as the false-positive rate seems to be fairly low (about 1% from the tests I've done).
Taking the board from #385 as an example, the samples are split as follows.

clustered_outliers

The algorithm provides additional options for plotting and analysis. The samples can be viewed one-by-one with the "SINGULAR" option for plots:

sample_36_outliers

A phase-analysis function (with plots) is also included:

outlier_phase_analysis

Lastly, a somewhat deprecated method for finding outliers is left as an option. This method (the threshold scan), focuses on the ADC differences between subsequent time-points, marking differences larger than an arbitrary threshold as outliers, and does not require multiple samples. This method is fairly flawed, especially since it requires setting arbitrary thresholds, but I believe it could still be occasionally useful (though I can remove it).

Copy link
Copy Markdown
Member

@tomeichlersmith tomeichlersmith left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you :) plots look good

@tomeichlersmith tomeichlersmith merged commit a16d239 into main Apr 1, 2026
3 checks passed
@tomeichlersmith tomeichlersmith deleted the outlier_scan branch April 1, 2026 15:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants