Add IBL's "noise cutoff" quality metric (again) #4072
Conversation
for more information, see https://pre-commit.ci
There was a problem hiding this comment.
Thanks @huizizhang949!
Looking great! Just some minor comments in the warning message and a value to double check in the docs!
|
|
||
| Expectation and use | ||
| ------------------- | ||
|
|
||
| Noise cutoff attempts to describe whether an amplitude distribution is cut off. | ||
| The metric is loosely based on [Hill]_'s amplitude cutoff, but is here adapted (originally by [IBL]_) to avoid making the Gaussianity assumption on spike distributions. | ||
| Noise cutoff provides an estimate of false negative rate, so a lower value indicates fewer missed spikes (a more complete unit). | ||
| If the distribution is not truncated at the low-end, one would expect cutoff to be less than 5 and ratio less than 0.1. |
There was a problem hiding this comment.
here we say that cutoff should be less than 5, but in the plot a cutoff of 1.961 shows a large cutoff. Maybe this should be 0.5 instead of 5?
There was a problem hiding this comment.
Hello, I think we are following what IBL suggests. @huizizhang949 what IBL arguments are equivalent to our arguments? It's something like they use some fixed number of bins, and they only include one bin in the low-tail? Maybe we can clarify this by saying something like:
A larger value of noise_cutoff suggests that the distribution is cut-off. IBL uses the default values of "..." and use a threshold of 5 to determine whether a unit is cut off or not. In practice, the IBL threshold is quite conservative, and a lower threshold might be better for your data. We suggest plotting the data using the plot_amplitudes widget to view your data when choosing your threshold.
There was a problem hiding this comment.
Hi, IBL's whitepaper suggests 5 for thresholding cutoff and 0.1 for ratio, but their code only computes and outputs cutoff (no ratio) and does not refer to any thresholds. Yes, they use the number of lower bins as the argument, with a default value of 1, which can be unstable. And we found that for the examples here 5 and 0.1 are actually too large. I have modified doc to clarify these.
Co-authored-by: Alessio Buccino <[email protected]>
Co-authored-by: Alessio Buccino <[email protected]>
Co-authored-by: Alessio Buccino <[email protected]>
Co-authored-by: Alessio Buccino <[email protected]>
Co-authored-by: Alessio Buccino <[email protected]>
Co-authored-by: Alessio Buccino <[email protected]>
for more information, see https://pre-commit.ci
for more information, see https://pre-commit.ci
|
Hello, just did some tiny tiny docs formatting. |
Copy of #3984 , before I somehow made a mess of it.