Skip to content

Conversation

t-b
Copy link
Collaborator

@t-b t-b commented Feb 28, 2025

  • FFT size fix using 2^l + 3^k logic, closes: PSX: Make it faster #1663
  • psx accepted average fix: return real amplitude for double expontential
  • average fit for reject, undet, all, add into column on the right
  • Add default values for filter frequencies:
sweepFilter: Default values for low/high, order defaults to 4
low cutoff  = 1 / (3 * pi * riseTau)
high cutoff = 1 / (3 * pi * decayTau)
deconvFilter: Default values for low/high, order defaults to 4
low cutoff  = 1 / (4 * pi * riseTau)
high cutoff = 1 / (4 * pi * decayTau)
  • Add y value from PSX_CalculateOnsetTime into psxEvent and psxstats
  • add the fit results as a wave in psx
  • the "block size" control doesn't respect the "current combo" checkbox state
  • add peakThreshold in () behind numberOfSDs in the JWN to document it
  • Add V_mode and all other missing entries from StatsQuantiles to postProc output for stats in psxStats
  • Prep PR: PSX prep #2471
  • TB: Dig up store/results code from slack
  • Tim provides Poc for figuring out psxKernel parameters from scratch
  • Approval from Tim
  • Fix tests
  • Update ihf documentation
Old Todos

So this should be ready for a first test. I've adapted `psxKernel` and `psxStats` to allow passing in multiple selections.

  • Fix assertion, https://aiephys.slack.com/archives/C06P3MYEV6H/p1742330027060869
  • PSX multiple selections (only that) #2376
  • Yesterday, we chatted about moving the baseline search before peak_t, and I think
    that still makes sense. The baseline search shouldn't search past the previous
    peak_t. With the baseline_t of the event available, PSX_CalculateEventPeak, instead
    of using prevDeconvPeak_t to constrain the search, can use the event
    baseline_tinstead. The above makes adding a peakfinding operation, which I suggested
    yesterday, unnecessary.
  • Tweak accept fit average logic: The start point for the fit should be calculated via rise time calculation (20% of amplitude, add control "fit start amplitude"). Use that for onset and peak methods.
  • Debug session with Tim
  • Polish code
    • // TODO add vertical dashed lines at positions
      // 3 checkboxes to turn all of these on/off below suppress update
      // peak_t
      // baseline_t
    • Fix assertion when restoring PSX panel with accepted fit checkbox checked
  • Add three taus to stats
  • Wait for Tim's changes
  • Fix function comment PSX_AppendTracesToAllEventGraph
  • https://aiephys.slack.com/archives/D6WGZV2V8/p1743629846161639
  • Fix slow/fast tau
  • Excel import could be better, see UpdateInfoButtonHelp: Set machine-readable text for C&P #2424
  • Commit cleanup
  • Add psxBPSweepFilter (BP = bandpass) and back down order for sweep/deconv. Handle the high/low frequency input arguments for the user such that the filtering is always bandpass.
  • Rename psxDeconFilter to include the filter type acronym (is this also a bandpass filter?). Manage the frequency arguments for the user such that it doesn't matter what order they are provided.
# psxSweepFilter(lowFreq, highFreq [, order = 6])
# back down on having nonfinite values in filtered data, order -2, output passing order
# back down on deconv order as well, same approach
# use code from Tim to determine the highest possible, reapply that order to all sweep data

@timjarsky

This comment was marked as outdated.

@timjarsky timjarsky assigned t-b and unassigned timjarsky Mar 7, 2025
@t-b

This comment was marked as outdated.

@t-b

This comment was marked as outdated.

@t-b t-b assigned timjarsky and unassigned t-b Mar 11, 2025
@t-b t-b assigned t-b and unassigned timjarsky Mar 18, 2025
@t-b t-b force-pushed the bugfix/2362-psx-support-multiple-selections branch from f508957 to f23db8a Compare March 19, 2025 14:41
@t-b t-b assigned timjarsky and t-b and unassigned t-b and timjarsky Mar 19, 2025
@t-b t-b changed the title Support multiple select statements Various PSX fixes Mar 25, 2025
@t-b t-b force-pushed the bugfix/2362-psx-support-multiple-selections branch from 1332271 to 08deb18 Compare March 26, 2025 18:12
@t-b t-b assigned timjarsky and unassigned timjarsky Mar 26, 2025
@t-b t-b force-pushed the bugfix/2362-psx-support-multiple-selections branch 4 times, most recently from a12c1dd to 193bf80 Compare April 8, 2025 19:46
@timjarsky

This comment was marked as outdated.

@timjarsky
Copy link
Collaborator

@t-b I also tested pasting the average fit results into Excel. The row formatting is correct. However, Excel plots all the columns in one column. Excel is dumb. To correct this, paste into Excel, select the cells, select Data from the menu, select Text to Columns, and select the appropriate delimiter in the Excel pop-up GUI.

@timjarsky
Copy link
Collaborator

timjarsky commented Apr 8, 2025

@t-b there may also be issues with the indivdual decay tau values. For example in this case I'm getting a negative weighted tau.

image

t-b added 12 commits June 25, 2025 11:39
These routines allow to calculate all values between 2^1 and 2^30 which
have 2 and 3 as prime factors. This helps in deciding a good cut off value
for FFT as it only performs when the primes are smaller than 5.
And also move the error state constants into MIES_Constants.ipf.
The FFT implementation in Igor Pro requires the prime factors of the
length to be < 5 for speedy execution.

For some cases these prime factors can be very large (~1e5) and thus FFT
would be dead slow, i.e. in the order of minutes on common machines.

We now employ a data shortening to a size which has only 2 and 3 as prime
factors, which makes FFT very fast. This is only done if we have prime
factors larger than a 1000 in the original size.
We now reserve space for the defaults in the
psxDeconvBPFilter/psxSweepBPFilter operations so that we can later on add
the calculated defaults, which need parameters from psxKernel, can be
added via PSX_AddDefaultFilterFrequencies.
And allow querying them from psxstats.
This is convenient for documentation purposes although this is not an
input parameter.
@t-b t-b force-pushed the bugfix/2362-psx-support-multiple-selections branch from 1f4d350 to 79100a9 Compare June 25, 2025 09:48
@t-b t-b assigned timjarsky and unassigned t-b Jun 25, 2025
@t-b
Copy link
Collaborator Author

t-b commented Jul 9, 2025

@timjarsky Given the SF code

trange = [0, inf]
sel = select(selrange($trange),selchannels(AD), selsweeps())
dat = data($sel)

psx(id, psxKernel($sel))

and

store("result name", psxstats(id, $sel, amp, all, count))

which creates the plot

grafik

the following IP code

Function Dostuff()

	string win = "DB_ITC18USB_Dev_0"

	string name = "result name"
	WAVE textualResultsValues = BSP_GetLogbookWave(win, LBT_RESULTS, LBN_TEXTUAL_VALUES, selectedExpDevice = 1)

	string key = SFH_FormatResultsKey(SFH_RESULT_TYPE_STORE, name)
	
	WAVE/T/Z settings = GetLastSetting(textualResultsValues, NaN, key, SWEEP_FORMULA_RESULT)
	ASSERT(WaveExists(settings), "Missing settings")
	
	string entry = settings[INDEP_HEADSTAGE]

	//print entry

	WAVE/WAVE/Z parsedData = JSONToWave(entry)
	ASSERT(WaveExists(parsedData), "Missing parsedData")
	ASSERT(IsWaveRefWave(parsedData), "parsedData is not a wave reference wave")

	WAVE/Z element = parsedData[0]
	ASSERT(WaveExists(element), "Missing element")

	print element
End

gathers the the number of events, here 42. For this exact use case you also need the latest version of JSONToWave pushed in this branch.

t-b and others added 3 commits July 9, 2025 19:23
Ever since 7aa2be6 (PSX: Add average fit for all states, 2025-06-04) we
are using the per-combo index for gathering the
eventKernelAmp/eventOnsetTime/eventPeakTime values.

This is wrong as we must use the index of the displayed events.
@t-b
Copy link
Collaborator Author

t-b commented Jul 29, 2025

@timjarsky I've fixed the kernelAmp zero error. There was some confusion when calculating the average for multiple combos/ranges.

@timjarsky timjarsky requested a review from Copilot August 5, 2025 19:09
Copy link

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR implements various fixes and improvements for the PSX (PostSynaptic Current/Potential) analysis functionality. The main purpose is to enhance the PSX event detection and fitting algorithms with improved filtering, fitting procedures, and user interface enhancements.

Key changes include:

  • Replaced single exponential with double exponential fitting for more accurate tau calculations
  • Added new filter operations (psxSweepBPFilter and psxDeconvBPFilter) with automatic parameter ordering and ringing detection
  • Enhanced GUI with visualization helpers for peak, baseline, and deconvolution markers

Reviewed Changes

Copilot reviewed 19 out of 19 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
Packages/MIES/MIES_SweepFormula_PSX.ipf Core PSX analysis implementation with new fitting algorithms and filter improvements
Packages/MIES/MIES_SweepFormula_PSX_Macro.ipf Updated GUI panel with new checkboxes and controls for visualization
Packages/MIES/MIES_WaveDataFolderGetters.ipf Added new getters for PSX event data structures and fit results
Packages/MIES/MIES_MiesUtilities_Algorithm.ipf New band-pass filter with ringing detection algorithm
Packages/MIES/MIES_Utilities_Algorithm.ipf FFT optimization utilities for better performance
Packages/tests/Basic/UTF_SweepFormula_PSX.ipf Updated tests for new PSX functionality
Packages/doc/SweepFormula.rst Documentation updates for new PSX operations
Comments suppressed due to low confidence (4)

Packages/MIES/MIES_SweepFormula_PSX.ipf:2439

  • Variable name 'riseLowerThreshold' should be 'riselowerThreshold' to match the declaration on line 2434.
	PSX_FitAverage(win, averageDFR, eventOnsetTime, eventPeakTime, eventStopTime, eventKernelAmp, stateNew, PSX_REJECT)

Packages/MIES/MIES_SweepFormula_PSX.ipf:2441

  • Variable name 'riseLowerThreshold' should be 'riselowerThreshold' to match the declaration on line 2434.
	PSX_UpdateAverageWave(contAverageUndet, undetIndex, averageDFR, PSX_UNDET)

Packages/MIES/MIES_SweepFormula_PSX.ipf:6445

  • [nitpick] Variable name 'forDisp' is unclear. Consider renaming to something more descriptive like 'avgEventDiffDisplay' or 'debugDisplay'.
	duplicate AvgEventDiff, forDisp

Packages/tests/Basic/UTF_Utils_Algorithm.ipf:1397

  • [nitpick] The test logic uses a logical OR condition which may not provide sufficient test coverage. Consider testing each case separately to ensure both conditions are properly validated.
			CHECK(result[0] == 2 || result[0] == 3) // NOLINT

@t-b
Copy link
Collaborator Author

t-b commented Aug 20, 2025

@timjarsky The store example is at #2362 (comment).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

PSX: Make it faster

2 participants